dotey's earlier finding: the bottleneck was the JSON

What's this about?

The post that set up the one above. Users reported BaoCut's transcription was slow. He repeatedly asked Codex and Fable to analyse the bottleneck, and each time got a pile of plausible reasons and plausible fixes — open more workers, pre-warm them — that produced ambiguous results. Analysing the packets himself, he found the real cost: the output JSON format was too verbose, making generation and validation slow. Switching to plain text or simple HTML saves tokens but makes parsing far more complex, because you then have to diff the polished output against the original subtitles and map changes back to words. He made the trade anyway: 33 model calls down to 12, 31 minutes down to 18, and a nearly 7-hour interview fully processed in 42 minutes.

What is the general principle here?

There is a dial between "simple program, expensive model output" and "cheap model output, complex program". Structured formats like JSON are chosen because they are trivial to parse — the cost is pushed onto the model, which must generate and validate every bracket and key. Flip the dial and the model emits something minimal while your code does the hard work of reconciling it. Neither end is correct; the point is that it is a choice, and most people never notice they made it.

What's the community saying?

1. The observation that started it: a capable model will optimise hard inside the frame you give it and rarely propose leaving that frame.
2. The reason it kept failing is instructive — every suggested fix was reasonable, cited real mechanisms, and was wrong about this system.
3. His cost datapoint for the non-agent path: full bilingual subtitle translation and proofreading for a one-hour video with DeepSeek v4 Flash, about ¥0.4.