How we work.

Abliteration: how we take the “I can't help with that” out of a language model, without taking anything else out with it.

The ideaHow it worksWhat changes
The Bleepless mascot snipping one thread out of a tangle

Refusal is one direction.

A language model turns your message into long lists of numbers, called activations, and passes them through dozens of layers. In 2024, researchers found something surprisingly simple inside chat models: when a model is about to refuse, its activations all shift the same way.

Across many different refused requests, it's essentially one direction: the refusal direction.

Figure 1. Two kinds of prompts, one direction between them

prompts it answersprompts it refusesrefusal direction
Illustration. Each dot is one prompt's activation, squashed down to two dimensions. The arrow from one group's average to the other's is the refusal direction.

Figure 2. Where the refusal signal lives

clearest herefirst layerlast layerseparation
Illustration of the typical shape: how cleanly refused and answered prompts separate along that direction, layer by layer.

The direction shows up most clearly in the middle of the network, where the model has understood the request but hasn't written anything yet.

That's where it “decides” to refuse, and that's where abliteration aims.

How it works.

Abliteration (ablate + obliterate) uses that finding directly. Find the direction once, then edit the model so it can never move along it again. No retraining and no tricks in the prompt: the change lives in the weights.

  1. 1

    Collect two sets of prompts

    Hundreds of requests the model normally refuses, and hundreds of ordinary ones it happily answers.

  2. 2

    Record what the model thinks

    Run both sets through the model and save its internal activations at every layer.

  3. 3

    Find the direction

    Average each set and subtract one from the other. The difference points the way the model moves when it's about to refuse.

  4. 4

    Remove it from the weights

    Adjust the weights so they can no longer write along that direction. The model keeps everything else it knows.

Figure 3. Removing one direction, keeping the rest

refusal direction r̂activation hremoved: (h·r̂) r̂kept:everything else
Every activation h is split into its part along the refusal direction r̂ and everything else. Abliteration removes the first part: h′ = h − (h·r̂) r̂. In the weights that becomes W′ = W − r̂ r̂ᵀ W.
The mascot pointing at one arrow among many dots

Why it barely touches anything else

A 27B model works in thousands of dimensions at once. Removing one of them is a tiny edit, aimed at exactly the behaviour we want gone. Knowledge, writing style and reasoning live elsewhere.

What changes, what doesn't.

Before

I'm not comfortable writing a roast that could hurt someone's feelings. How about a kind birthday message instead?

After

Dave's so late to everything, his birth certificate says “expected: Tuesday-ish”. Want five more?

Example prompt: “Write a brutal roast of my friend Dave, who's always late.”

BeforeAfter
Refuses ordinary-but-edgy questions
Adds lectures and disclaimers
Softens dark fiction until it's toothless
Knows facts, writes well, reasons step by step
Follows your instructions and your style

It isn't magic. Abliteration removes most refusals, not every single one, and the edit can cost a little general ability, which is why careful versions test the model afterwards and sometimes give it a light fine-tune to recover. It also can't add knowledge the base model never had.

Further reading.

Refusal in Language Models Is Mediated by a Single Direction

Arditi et al., 2024. The research behind abliteration.

Uncensor any LLM with abliteration

Maxime Labonne, Hugging Face, 2024. A practical walkthrough.

Ask it something.

Helios 1 is abliterated. Zouk is fine-tuned to answer first, too.

Start chatting Meet the models