Why AI Alignment Needs Inspectable Values: Lessons From Damasio's Elliot

One patient from the neuroscience literature deserves a place on every AI alignment reading list. Antonio Damasio calls him Elliot, and his case anchors the book Descartes' Error. A tumour pressing on the ventromedial prefrontal cortex was surgically removed, and some of the nearby tissue went with it. Afterwards Elliot scored above average on IQ tests. His memory, language, attention, factual knowledge and logical reasoning were all intact. Given a moral dilemma, he could set out the arguments on both sides more completely than most people present.

Yet his life fell apart. He lost his job, then his savings through a string of poor business decisions, and he went through two divorces. At each step he could analyse the options in detail, and at each step he picked badly and learned nothing. Elliot hadn't lost knowledge. He had lost whatever makes knowledge actionable.

That case describes, as sharply as anything, what "alignment" is trying to fix. It's also a warning, because the industry has built, at huge cost, systems with the opposite deficit, and there's hardly any vocabulary for noticing it.

Four engineering questions

Strip away the rhetoric and alignment is a value-mechanism problem. It splits into four questions, all of them engineering questions:

1. How does a system carry a value?
2. How does it apply that value at the instant of a decision?
3. How can people inspect which value it applied?
4. How can people correct the value when it's wrong?

"Which values should the system hold?" is missing from that list on purpose. It's a different problem, one of governance and human judgement, and it comes back near the end. Mixing the two up is why alignment debates so often drift into philosophy when they should be producing schematics. So how do today's best systems answer the four questions?

What reinforcement learning from human feedback achieves, and where it stops

Reinforcement learning from human feedback (RLHF) deserves credit. It works. It took models that would happily produce anything and made them broadly useful, cooperative and safe in a way raw pretraining never managed. It ranks among the real engineering achievements of the past decade, and anyone dismissing it hasn't spent time with models from before it.

The way it works matters, though.

RLHF takes human preference signals and blends them into the very weight matrices that store the rest of the model's knowledge. Values and knowledge end up as one material. No border separates them, whether faint or buried. There simply isn't one.

That's roughly like trying to give a person good judgement by rewiring their cortex and hoping it emerges. Sometimes it does. Three problems follow:

1. Inspection is impossible. There's no place to look for what the model values. You can probe its behaviour and infer, which is a psychology experiment, not an inspection.
2. Tracing is impossible. When a model refuses a request, approves a loan or flags a claim, nobody can point to the experience that produced that preference. Provenance was destroyed during training, by design.
3. Editing is impossible. Find one wrong value, such as a bias, an overcorrection or a preference absorbed from poor annotator data, and no surgical fix exists. Retraining disturbs everything else along the way.

Amazon found this out the hard way. Starting in 2014 the company built a model that scored job applicants from one to five stars, trained on a decade of its own résumés, drawn from a male-dominated industry. The model learned to penalise résumés containing the word "women's" and the names of certain all-women colleges. Amazon spotted those terms and neutralised them. Then in 2017 it scrapped the project, because after fixing the cases it could see, it had no means of showing that the model wasn't expressing the same preference through some other proxy nobody had thought to check.

The anecdote holds the whole argument. The failure wasn't that the model learned something wrong, since every model does. The failure was that nobody could find out what else it had learned, and so nobody could tell when the fixing was done. That's whack-a-mole with the lights off. Years of work ended because the values weren't addressable.

For a recipe chatbot, that's a shrug. For systems that touch credit decisions, medical triage, claims adjudication, or anything a regulator may someday ask you to explain under oath, those three gaps aren't inconveniences. They rule the approach out.

When opaque values hurt real families

Ruling out isn't an abstraction. From 2005 to 2019 the Dutch tax authority wrongly accused roughly 26,000 parents of childcare-benefit fraud. A self-learning risk-classification model, in use since 2013, took part in choosing whom to flag. Dutch nationality was one of its inputs, which pushed scores up for non-Dutch nationals, drew more investigations to them, and fed those outcomes straight back into training. Civil servants handling flagged cases were never shown why a score was high. Amnesty International described a black hole of accountability. Families lost homes, jobs and marriages, and the government fell in January 2021.

The mechanism matters as much as the outcome. No engineer decided that "non-Dutch is riskier" should be a value. It piled up, unnamed and unreadable, in a system where values and knowledge shared the same space, and the people running it had no surface on which to discover it.

We tend to treat that as the natural state of affairs, as if values must be diffuse and unreadable. Biology says otherwise.

What emotion does for reasoning

Popular belief holds that cool, Spock-like rationality yields good judgement while emotion corrupts it. The evidence points the other way.

Damasio's somatic marker hypothesis proposes that emotional processes guide and bias decisions instead of interfering with them. When a person lives through an outcome, a bodily-affective state gets linked to it. Later, on meeting a similar option, the marker is re-evoked, sometimes as a conscious gut feeling and often below awareness, and it tilts the choice before deliberation begins. Bechara and Damasio state the functional claim about as directly as possible: sound, rational decision-making depends on prior accurate emotional processing. Cognition without emotional signals doesn't produce normal behaviour even when knowledge is fully adequate.

That's precisely what Elliot demonstrates: complete knowledge, no value signal, and every choice forced through explicit cost-benefit analysis with no means of ending the analysis. He never really finished one.

Comparative work makes the idea harder to wave away. In the model Mendl and Paul propose, an animal picks between affective options: every possible action carries a value built from the feelings attached to earlier outcomes of that action in comparable situations. Anderson and Adolphs list four emotion primitives, namely scalability, valence, persistence and generalisation, and report them in organisms as far from us as fish and fruit flies.

Evolution arrived at this design repeatedly and independently over hundreds of millions of years, because a nervous system that can't tag experience as good, bad, costly or dangerous has to recompute the world from scratch every time it moves. Evolution built a value-carrying mechanism since no working decision-maker can do without one. The AI field built systems with encyclopaedic knowledge and then attached values by dissolving them into that knowledge.

Right mechanism, poor implementation

Biology's mechanism is sound: learned value signals, indexed by experience, tied to memory and retrieved at decision time. Its implementation is poor in exactly the ways engineers care about:

• It can't be inspected. Nobody reads their own somatic markers as values. At best there's a feeling.
• It can't be traced. A gut says no to a job offer, and asking why gets silence. The provenance is gone.
• It can't be edited. A value learned from one bad experience at nineteen doesn't vanish on command. With luck and diligence it fades after years of effort.

Set those three beside the three RLHF failures above and they match. The field copied biology's weaknesses and dropped its real architectural insight. So consider keeping the mechanism and fixing the implementation.

A mechanism with receipts

Once the problem is stated that way, the design follows almost mechanically. At learning time, when the system records a pattern of experience, a valence is attached to it: a signed number, conventionally in the range from -1 to +1, stored alongside the pattern instead of merged into it. At prediction time, when that pattern matches the current observation, its valence rides along with the prediction and biases which action gets chosen.

That's all there is to it. It's the somatic marker made from numbers instead of neurochemistry, and the consequences are bigger than they look:

1. Inspectable. Nobody can read a person's somatic markers, but a score such as safety at -0.7 on a particular learned pattern is just a number in a known place. The system's full list of values and their strengths can be listed. No probing, no behavioural inference, no psychology experiment.
2. Traceable. An intuition keeps no receipts, whereas a learned valence points back to the specific observations it was built from and what happened in each. Provenance survives because nothing dissolved it.
3. Editable. When a value is off, change the number directly, or feed in a counterexample and let the running average absorb it. No training run is needed and unrelated knowledge stays untouched. People may need therapy and transformers a fresh pipeline, but here there's a simpler way out.

Picture a system that has been learning from a stream of account events and has stored the following:

PTRN|a7f3e91   ["login_new_device", "password_reset", "payee_added", <additional context>]
   emotives   →  fraud_risk: -0.62   urgency: +0.85
   provenance →  13 matched occurrences | 9 adverse, 4 benign
                 recent window: [-0.9, -0.8, -1.0, -0.4, +0.1]

During learning, each occurrence of that sequence added an outcome valence to the pattern: the pattern stores what happened and the emotive stores how it turned out. When a live stream of events later lines up with those three steps, the pattern activates and the fraud-risk score accompanies the prediction. That figure, attached to that pattern, is what nudges the system to hold the transfer.

Now suppose that value is wrong. Imagine nine of the thirteen bad outcomes all landed inside one three-week window, during which a device-upgrade promotion sent many honest customers through exactly this sequence and a jumpy fraud desk flagged every one. Spotting that needs no theory about the model's internals: read the provenance and the dates bunch together. Fixing it needs no retraining run either. Adjust the value on that pattern, or supply corrective observations and let the rolling window soak them up.

The blast radius is one pattern. Everything else the system knows stays untouched, because it was never entangled with this in the first place.

Keeping values apart from knowledge also enables things that fused systems can't do. Different value sets can run over the same learned knowledge for different deployments, a system's values can be compared before and after an update, and an auditor can simply be handed the list.

Four jobs emotions do

Pfister and Böhm argue that emotions serve four separate functions in decision-making, not one, and each maps to something worth building:

• Information. Pleasure and pain are what shape preferences in living creatures. In software, the tag on a value says which follow-ups have tended to end well and which badly.
• Speed. People choose quickly under time pressure by skipping deliberation. A machine can use a threshold check to act or refuse without evaluating every candidate.
• Relevance. Emotion directs attention to what matters for the decision at hand. In software, values become the prioritisation signal across competing predictions.
• Commitment. People pre-commit on morally significant choices and resist momentary self-interest. In a system, that means hard guardrails, such as skipping any option with safety below 0.5, which aren't argued afresh on every pass.

Commitment, Pfister and Böhm stress, isn't about pleasure at all. It belongs to the moral and social domain. A well-built value mechanism therefore needs two opposed drives rather than one dial: things to pursue and things to avoid. Intent and defence.

A working implementation

The mechanism was described abstractly on purpose, because the argument stands or falls by itself. It also exists as running software. KATO is an open-source architecture that learns patterns deterministically, and its value tags go by the name emotives. They get attached to patterns while learning and are carried inside each prediction object. A rolling window of recent averages lets a value drift as experience builds up, so no single observation controls it. Emotives come in two kinds: intent-side ones the system tries to maximise, and defence-side ones it tries to minimise. Each is a number that can be read, followed back to its observations and changed.

The lineage isn't new. KATO's predecessor, GAIuS, was reportedly the only machine-learning algorithm certified to FAA DO-178C. Scale didn't earn that certification. Explainable, traceable, reproducible decisions did, so determinism was the price of entry rather than a stylistic choice. Explicit, inspectable values are the same sort of ticket for the same reason.

What this does not settle

Emotives don't say which values a system should hold. The mechanism can't tell anyone the correct valence for a pattern, whose safety deserves extra weight, or where fairness should give way to accuracy. Choosing those is value specification, also called outer alignment, and it remains a governance decision for people.

What changes is visibility. With explicit values, a person has to write the figure down and another has to approve it. Trade-offs stop being accidental by-products of an unreviewed annotation pipeline. They turn into written decisions with owners.

That isn't a limitation of the approach. It's the purpose. Alignment shifts from a training-time hope to a runtime engineering property, one that can be specified, tested, monitored and revised on a Tuesday afternoon without a GPU cluster.

Control, not lift

The Wright brothers didn't win by solving lift. Gliders had flown for decades, and engines were arriving on their own timetable. They won by solving control, a way to steer actively something that could already stay airborne.

Transformers fly. That question is settled, and scale will keep settling it. What they lack, structurally, is a control surface: somewhere to reach in and say "this, not that," confident that only the intended thing was touched. Calls to slow frontier AI work are unlikely to be heeded, which makes the missing control surface more pressing.

Biology built one and has refined it since before vertebrates existed. The design notes are there to be read.

Sources

• Damasio, Antonio. Descartes' Error: Emotion, Reason and the Human Brain, published 1994.
• Pfister and Böhm on the multiplicity of emotions, Judgment and Decision Making, volume 3, 2008.
• Anderson and Adolphs, a framework for studying emotions across species, Cell, volume 157, 2014.
• Bechara and Damasio on the somatic marker hypothesis, Games and Economic Behavior, volume 52, 2005.
• Mendl and Paul on animal affect and decision-making, Neuroscience and Biobehavioral Reviews, volume 112, 2020.