What's this about?
Meta released Spark 1.2, the latest in its Spark model family. Simon Willison used his SVG-pelican benchmark to visually compare Spark (April 8), Spark 1.1 (July 9), and Spark 1.2 (today), showing incremental improvement across one family over four months.
What exactly is the "pelican benchmark"?
Willison's informal test: ask a model to draw a pelican riding a bicycle as an SVG (vector image). It's deliberately hard — models must reason about shapes and spatial relationships in code — and because it's the same prompt every time, it makes progress within a model family legible at a glance. Not rigorous, but a consistent visual yardstick.
Why does it matter?
36 likes. Modest news, but a useful reminder that steady within-family iteration (April → July → August) is where a lot of real progress happens, beneath the splashy new-model headlines.