SemiAnalysis benchmarks Anthropic, OpenAI on long horizon coding

// 2h agoBENCHMARK RESULT

SemiAnalysis benchmarks Anthropic, OpenAI on long horizon coding

SemiAnalysis purchased various subscription plans from both Anthropic and OpenAI to randomly run long-horizon coding tasks, likely to compare the models' practical performance and reliability over extended interactions. The testing aims to benchmark how these leading models handle complex, multi-step coding scenarios under different subscription tiers.

// ANALYSIS

Benchmarking long-horizon capabilities is crucial as models are increasingly used for complex, autonomous tasks.

–Tests the practical limits of context windows and sustained reasoning.
–Compares the value propositions of different subscription plans from major AI providers.
–Helps developers understand the real-world reliability of models for coding tasks.

// TAGS

aillmbenchmarkcodinganthropicopenaisemianalysis

DISCOVERED

2h ago

2026-06-11

PUBLISHED

2h ago

2026-06-11

RELEVANCE

8/ 10

AUTHOR

pueblokc

// KEEP READING

More AI developer news from the feed

EXPLORE FULL FEED

UPDATE17m ago

DeepSeek V4 hits Vercel AI Gateway via Azure

Vercel has added support for DeepSeek V4 Pro and Flash models on its AI Gateway via Azure integration, offering developers a cost-efficient failover path with implicit prefix caching. Both models feature a 1-million-token context window and native tool use, manageable through a single unified API.

UPDATE43m ago

Claude Managed Agents gain scheduling, environment variables

Anthropic has introduced scheduled deployments and environment variable support for Claude Managed Agents, enabling automated tasks and secure CLI integrations. Configured via Claude Code or the ant CLI, the architecture swaps credentials at the network boundary so agents never have direct access to raw secrets.

LAUNCH44m ago

Web-based Quake 2 multiplayer port launches

Pieter Levels has launched a web-based, multiplayer port of Quake 2 featuring a revived 28-year-old custom map originally created by Javi Lopez. Developed with assistance from Anthropic's Claude Fable 5 model, the project required repacking PAK files to resolve initial texture loading issues.