What's this about?
LLMs increasingly write Infrastructure-as-Code, where one insecure default deploys straight to production. Prior evaluations report raw vulnerability counts, which the paper argues is meaningless without a baseline — hence "compared to what?" The contribution is anchoring against what human-authored IaC actually looks like.
Why does the baseline change the conclusion?
Because "the model produced 12 misconfigurations" is only alarming if humans produce fewer. Human-written Terraform and Kubernetes manifests are famously full of over-permissive defaults, public buckets and wide security groups — the entire cloud-security-posture industry exists because of it. A model that is merely as bad as a median human is a different finding from one that is worse, and both are different from one that is better but fails in unfamiliar ways.
Why does this matter to you?
- IaC is the highest-leverage place a generated mistake becomes a live exposure, and it is squarely in scan-the-repo territory.
- "Compared to what" generalises: raw finding counts without a baseline are the most common way security tooling misleads.
- It suggests reporting model-generated IaC findings against a human-authored control, not in isolation.