
Dark Factory benchmarks mixed-provider AI workflows
Cole Medin’s "lights-out" software factory experiment now supports mixed-provider AI workflows, leveraging Claude 3 Opus for reasoning and Kimi K2.6 for implementation. The project uses GitHub labels as an autonomous state machine to manage the entire development lifecycle from issue triage to automated merge.
The "Dark Factory" approach demonstrates how to bypass the "gaming the system" trap in AI dev by using a validator agent that never sees the original implementation plan. It is a blueprint for cost-effective, continuous software delivery without human oversight.
- –Mixed-provider routing balances reasoning depth with economic throughput by pairing high-level planners with specialized coding models.
- –The "Holdout" pattern ensures rigorous quality gates by decoupling implementation context from validation requirements.
- –Using GitHub labels as a state machine provides a transparent, audit-ready log for every autonomous agent decision.
- –Benchmarking shows specialized model pairings can outperform single-model setups in both cost and fidelity.
- –The project highlights a shift where developers move from writing code to orchestrating complex, autonomous multi-agent systems.
DISCOVERED
1h ago
2026-05-20
PUBLISHED
1h ago
2026-05-20
RELEVANCE
AUTHOR
Cole Medin