Anthropic cache TTLs backfire on costs

// 45d agoINFRASTRUCTURE

Anthropic cache TTLs backfire on costs

A developer reports that disabling prompt caching on Anthropic models slightly lowered their bill, suggesting the newer cache TTL economics can be worse than expected for low-reuse workloads. The lesson is that prompt caching only pays when the same context gets reused often enough to offset write overhead.

// ANALYSIS

Prompt caching is an optimization, not a guarantee. If your workload is bursty, one-shot, or has long gaps between turns, the cache write tax can outweigh the savings from cache reads.

–Anthropic’s prompt caching supports 5-minute and 1-hour TTLs, and the longer-lived cache costs more to write
–Long-running agent sessions and repeated codebase lookups are the best fit; sparse request patterns are the worst fit
–Per-session economics matter more than aggregate token counts, so “more caching” can still mean “more spend”
–This is a reminder to measure infra at the workload level before assuming the provider’s default is cost-optimal

// TAGS

anthropicllmpricinginferenceapi

DISCOVERED

45d ago

2026-04-30

PUBLISHED

45d ago

2026-04-30

RELEVANCE

8/ 10

AUTHOR

theo

// KEEP READING

More AI developer news from the feed

EXPLORE FULL FEED

OPEN SOURCE51m ago

ByteDance open-sources Bernini video framework

ByteDance has open-sourced Bernini, a unified framework that combines an MLLM-based semantic planner with a DiT-based renderer for consistent video generation and editing. By decoupling structural semantic planning from pixel generation, the system enables highly controllable, reference-guided edits while mitigating frame flickering and background drift.

OPEN SOURCE51m ago

MiniMax open-sources MSA sparse attention kernel

MiniMax has open-sourced MiniMax Sparse Attention (MSA), a blockwise sparse attention kernel designed to handle million-token context windows efficiently. By combining a two-branch architecture with a co-designed GPU execution path, MSA reduces per-token compute by 28.4×, achieving a 14.2× prefill speedup and 7.6× decoding speedup on H800 GPUs.

OPEN SOURCE51m ago

shadcn releases improve to audit codebases

Created by shadcn, improve is an open-source developer tool that optimizes token consumption in agentic workflows by decoupling planning from code generation. The tool uses premium frontier models to audit codebases and generate execution plans, then delegates coding tasks to cheaper models.