Merge Gateway adds batch inference support

As LLM gateways mature into critical enterprise infrastructure, unified batching and cost-arbitrage features are quickly shifting from luxury add-ons to essential cost-control requirements.

• **Significant Cost Reduction for Heavy Workloads**: Background pipelines, synthetic data generation, and regression evaluations burn millions of tokens; unifying batch execution allows organizations to cut inference bills roughly in half without building bespoke batch clients per provider.

• **Centralized Policy Enforcement**: Running asynchronous workloads through the gateway preserves organization-wide DLP filtering, audit logging, and per-project budget guardrails that direct provider scripts often bypass.

• **Abstraction of Fragmented Provider Batches**: Teams avoid writing bespoke file upload, polling, and formatting logic across divergent provider batch schemas (OpenAI vs. Anthropic vs. Bedrock).

• **Upstream Queue Bottlenecks**: Providers impose strict concurrent batch queue limits, meaning enterprise users will still need to plan around queue-full exceptions and variable turnaround windows during peak provider load.