What's this about?
Reinforcement learning can align diffusion models with human preferences, but a reward given only at the end does not say how any intermediate step should have been different. DiffusionOPSD provides that intermediate signal. 50 upvotes.
What exactly is the problem with endpoint rewards?
A diffusion model builds an image by removing noise over many steps. If you only score the finished picture, you learn "that sequence was good" without learning which step helped — like being told your dinner was bad without knowing which ingredient. Producing a per-step signal is the difference between learning from an outcome and learning from a process.
What's the community saying?
The credit-assignment framing is what draws interest, since it is the same problem that dominates long-horizon agent work — figuring out which of many steps deserves the blame — appearing here in image generation.