ProgramBench tests coding agent language choices

Evaluating coding agents by forcing specific languages is a brilliant way to uncover compiler and syntax biases in LLMs and identify the most cost-effective target languages for agentic generation.

* Python will likely consume the fewest tokens due to its high density, but compiled languages with strong type safety (like Rust or Go) might yield higher correctness due to rigorous compile-time checks.

* Allowing agents "free choice" often results in them defaulting to Python or JavaScript out of habit, which may not be the optimal choice for rebuilding lower-level system utilities.

* The outcomes could guide developers on how to instruct autonomous agents to write code (e.g., targeting Go instead of C to minimize bugs).