GDPx Brings Partial Credit to Agent Evals

// 1h agoBENCHMARK RESULT

GDPx Brings Partial Credit to Agent Evals

SemiAnalysis launched GDPx, an agentic benchmark spanning 11 high-value domains, including software engineering, finance, cybersecurity, biology, and chip design. It combines binary and continuous grading, giving models partial credit where all-or-nothing scoring would reduce difficult tasks to zero.

// ANALYSIS

GDPx is a more realistic test of agent capability, but its low scores and nuanced grading show how far models remain from reliably completing expert workflows.

–Finance and legal tasks often score 0% under binary grading, making partial credit essential for measuring incremental progress.
–The benchmark uses realistic environments and long-running tasks rather than self-contained prompts.
–Claude Opus 5.5 leads the leaderboard at just 12%, underscoring the difficulty of frontier-level agent work.
–Partial credit is informative, but a 44% legal score still corresponded to no usable redline, so progress metrics should not be confused with production readiness.
–Safety refusals materially affect rankings in cyber, biology, and alignment tasks.

// TAGS

gdpxbenchmarkevaluationagentresearchtool-use

DISCOVERED

1h ago

2026-11-02

PUBLISHED

2h ago

2026-11-02

RELEVANCE

9/ 10

AUTHOR

AdamHoltererer

// KEEP READING

More AI developer news from the feed

EXPLORE FULL FEED

MODEL23m ago

Deep Principle launches Hydrogen-1.0 science model

Deep Principle has released Hydrogen-1.0, a 753B open-weight model post-trained from GLM-5.3 for chemistry, materials science, and life-science research. It offers a 1M-token context window, MIT-licensed weights, and specialized training on laboratory records and discovery trajectories.

BENCHMARK31m ago

Prime Inference tops Nemotron cache rankings

Prime Intellect reports a 96.9% cache hit rate serving Nemotron 3.5 Super VL, cutting effective input costs to $0.024 per million tokens—roughly 70% below its largest provider rival.

INFRA46m ago

OpenAI's Pando Cuts Git Push Latency

OpenAI built Pando, a new Git backend for its monorepo that preserves existing workflows for engineers and coding agents. It reportedly cuts push latency by about 80% while serving nearly a petabyte of Git data daily.