What's this about?
Amid the K3 hype, Mollick posted a caution: when he used Kimi K3 Max for complex statistical auditing of his own prior academic work, it "messed up in a bunch of ways" — misapplying statistical methods and executing others badly. GPT-5.6 Pro caught and critiqued the errors, and Mollick agreed with the critique.
What exactly is "statistical auditing"?
Re-checking the numbers in published research: were the right statistical tests used, were they applied correctly, do the conclusions follow? It's exacting, multi-step analytical work where subtle methodological errors invalidate results — a very different skill from writing plausible prose or pretty frontend code, and a good probe of whether a model *reasons* or *pattern-matches*.
What's the community saying?
259 likes, the designated cold-shower post of K3 day
- Leaderboards vs reliability — the same model can top arenas and still be untrustworthy for expert analytical work.
- Cross-model checking works — using a rival frontier model (GPT-5.6 Pro) to audit K3's output is itself a workflow takeaway.
- Pairs with his other posts — closest-to-frontier-yet on shader tests, but "wow does the harness love to loop back over tasks tweaking things at max level."