What's this about?
MinerU, a 75k-star open-source tool from OpenDataLab, converts PDFs, images, and Office documents into clean Markdown while preserving tables, layout, and figures. It trended again as the centerpiece of a sharp debate about whether feeding PDFs straight to AI wastes money.
What exactly is the "PDF token" argument?
When you upload a PDF to a chatbot, the model often "sees" each page as an image using its vision system to preserve layout — and lxfater argues that costs roughly 4× the tokens (and thus money) versus plain text. Converting to Markdown first, the claim goes, can save ~80% of tokens. dotey pushed back: multimodal models understand a page-by-page PDF *better* than stripped Markdown+images, and the real token cost in modern agents is skills and repeated tool calls, not input text — plus prompt caching makes input size matter less.
What's the community saying?
- The pro-conversion side: convert everything to Markdown to slash token spend, especially at scale where 4× adds up fast.
- The pro-PDF side (dotey): the token you save is offset by information loss — splitting figures and icons out of context hurts the model's understanding, and for image/chart-heavy docs, just send the PDF.
- A pragmatic middle: for pure-text docs Markdown is fine; for anything with tables, charts, or layout, keep the original and let the multimodal model read it.