What's this about?
Models detect moral content broadly, but the paper argues detection is a low bar. It asks whether they go further: do they distinguish moral foundations from one another, and organise them in any structured way?
What are "moral foundations"?
A framework from moral psychology proposing that moral intuitions cluster into distinct dimensions — care/harm, fairness, loyalty, authority, sanctity — which different people and cultures weight differently. Applied to a model, the question becomes whether "this is wrong" is one undifferentiated signal internally, or whether the model separately represents *why* something is wrong. Those have very different implications for alignment work.
Why does this matter to you?
- If moral judgement is one flat signal internally, then fine-tuning on it moves everything at once — which explains a lot of observed over-refusal.
- Interpretability of value representations is a prerequisite for steering them deliberately.
- The framework is borrowed from human moral psychology, which may or may not carve the model at its joints — worth reading sceptically.