llama.cpp cold-starts A770 servers on Windows 11

// 45d agoINFRASTRUCTURE

llama.cpp cold-starts A770 servers on Windows 11

A Reddit user reports that a llama.cpp server on Windows 11 with an Intel Arc A770 and dual Xeon C612 platform repeatedly unloads the model after idle periods, even with BIOS and Windows power-saving options disabled. When the next API request arrives, the model can take a long time to reload in chunks, suggesting a cold-start path rather than normal steady-state inference. The only reliable workaround they found was a polling script that sends a tiny completion every 30 seconds to keep the server active.

// ANALYSIS

This looks more like an idle-path or driver/runtime issue than a simple display-output problem.

–The slow “reload in gigabyte increments” points to model state being paged out or reinitialized, not just the GPU entering a light sleep state.
–Intel Arc on Windows plus an older workstation chipset is a plausible edge-case combination for PCIe power management or driver behavior.
–The polling workaround is practical, but it is a keepalive hack, not a fix.
–For operators, the real question is whether llama.cpp is unloading on its own, or whether Windows/driver behavior is forcing the backend to rehydrate VRAM and host memory after idleness.

// TAGS

llama-cppintel arca770windowsgpullm servingapiidlekeepalive

DISCOVERED

45d ago

2026-04-18

PUBLISHED

45d ago

2026-04-18

RELEVANCE

5/ 10

AUTHOR

Turbulent-Attorney65

// KEEP READING

More AI developer news from the feed

EXPLORE FULL FEED

OPEN SOURCE1m ago

Flowsint launches graph-based OSINT platform

Flowsint is an open-source, graph-based investigation platform designed for security analysts, threat intelligence teams, and OSINT researchers to visually map relationships between digital entities. The self-hosted, Docker-compatible system features customizable data transforms that automatically query external sources to enrich investigation graphs.

OPEN SOURCE1m ago

Headroom compresses LLM context by 60–95%

Headroom is an open-source developer tool and proxy designed to compress LLM context—such as tool outputs, logs, files, and RAG chunks—to reduce token consumption by 60% to 95% while maintaining model accuracy. It integrates as a Python/TypeScript library, an MCP server, or a zero-code proxy server compatible with Claude Code, Cursor, and Aider.

OPEN SOURCE1m ago

Hermes WebUI offers a responsive and lightweight web interface for managing and interacting with Hermes Agent sessions.

Hermes WebUI is a community-built web application designed to interface with the Hermes Agent. It provides a clean, mobile-responsive three-panel layout that includes a chat window, session management, and a file browser for the agent's workspace. Users can host the interface on remote servers or local machines and connect to their agent via HTTPS, SSH tunnels, or Tailscale, enabling flexible remote access from phones, tablets, or laptops.

llama.cpp cold-starts A770 servers on Windows 11