Modelship unifies local model serving

Modelship is aiming at a real pain point: local AI stacks are getting broader than “run one chat model,” but most self-hosted tooling still treats every modality as a separate service.

• –The OpenAI-compatible API makes it practical as a drop-in backend for existing SDK-based apps, agents, and home automation tools.
• –Per-model GPU allocation is the sharp feature here, especially for developers trying to fit chat, embeddings, STT, TTS, and image generation onto constrained hardware.
• –Ray Serve gives it a more serious orchestration foundation than a simple wrapper, with isolated deployments, health checks, replicas, and routing.
• –The tradeoff is maturity: the README flags production gaps like no rate limiting, limited health checks, thin test coverage, and no Helm chart.
• –This sits between Ollama-style simplicity and production model-serving platforms, which could be useful if the project keeps tightening operations.