What's this about?
Roboflow, a computer vision company, ran OpenAI's GPT-5.6 lineup through its forthcoming VLM benchmark, which covers detection, counting, OCR and data extraction. Their conclusion: Sol is clearly the best vision model OpenAI has released, with the biggest improvement in object detection and counting, where GPT-5.5 was notably weak. That matters because OpenAI's launch emphasized computer use — agents navigating desktop applications — which depends entirely on visual understanding.
What exactly is a "vision model" or VLM?
A vision-language model takes images as well as text and can reason about both together — describe a photo, read a screenshot, find an object, count things. The term covers a wide range of difficulty that benchmarks tend to blur. Describing a scene is easy. Localizing precisely — this object is at these coordinates — is much harder, and it's what actually matters for an agent controlling a computer, because clicking the wrong place fails silently. Counting is harder still, since models tend to estimate rather than enumerate.
What's the community saying?
- Impressive at hard perception. One commenter had it identify and draw the very faint reflection of what they were wearing, visible only in a small dark region of a brightly lit poster behind glass — while the poster itself contained similar clothing. Another had it correctly quote roughly-highlighted text and identify the nested context (a modal, inside a web app, inside a desktop screenshot).
- "Still as blind as any other model, no taste, no attention to detail." The opposing view, held strongly, mostly by people asking for aesthetic judgment rather than perception.
- Image editing remains bad. A detailed account of two weeks trying to get Codex to outpaint a generated background image at higher resolution while preserving scale and content — repeated failure.
- The benchmark itself got audited. Commenters found Sol marked wrong where it was right — an expected answer containing an error, and a bounding-box result rotated 90 degrees, apparently from an EXIF orientation bug in the harness rather than the model.
- "Why is pill counting even in here?" A fair methodological objection: some of these tasks are solved better and far more cheaply by classical computer vision, and benchmarking a language model on them measures the wrong thing.