Small Models Show Agent Promise

The real takeaway is that small models can work for agents, but only when the orchestration layer does a lot of the heavy lifting. The failure modes here look less like raw capability gaps and more like instruction-following, schema, and state-retention problems.

• –Nemotron variants repeatedly looped and re-did work, which is disastrous in iterative agent loops.
• –Qwen and OmniCoder were more capable, but JSON schema adherence and latency still became bottlenecks.
• –Jan-v3-4B followed directions better, yet skipped steps and failed to persist outputs, so it wasted prior work.
• –The task design itself is sensible: break work into small JS subtasks, save intermediate files, and constrain the agent’s world tightly.
• –Model-specific prompts may outperform brute-force scaling, especially for consumer-friendly setups on rented 3090s.