Earendil measures AI code sloppiness and erosion
While modern LLMs excel at generating code that passes isolated unit tests, iterative software development with coding agents frequently triggers an explosion of bloated abstractions, duplication, and architectural decay. Sebastian from Earendil examines the challenge of measuring code "slop," dismissing naive LLM-as-a-judge approaches in favor of quantitative structural metrics drawn from the SlopCodeBench benchmark: Verbosity (clone lines and AST-Grep flagged lines relative to total lines of code) and Erosion (the concentration of code mass in high-complexity functions). Across evaluations, agent-generated code exhibited over twice the verbosity (0.33 vs. 0.15) and erosion (0.68 vs. 0.31) of human repositories, while multi-turn checkpoints caused the strict pass rate of top models to drop to 0% as bad architectural decisions compounded.
Vibe coding has hit its architectural hangover: generating green test suites in isolated single shots is trivial, but keeping agent-generated codebases maintainable across multiple iterations remains an unsolved failure mode. LLMs can generate functionally correct code due to clear test-reward signals, but they fail to self-regulate architectural hygiene or resist introducing unnecessary bloat. Brittle LLM-as-a-judge evaluations reinforce the necessity of deterministic structural metrics like AST analysis and cyclomatic complexity. The quantitative definitions of Verbosity and Structural Erosion offer concrete mathematical proxies for architectural degradation that align closely with human developer intuition, while iterative benchmarks expose how compounding technical debt in multi-turn agent workflows causes strict solve rates to collapse.
DISCOVERED
1h ago
2026-09-11
PUBLISHED
4h ago
2026-09-11
RELEVANCE
AUTHOR
doppp