Run any model on your hardware.
Deploy open-weight models to Kubernetes or dstack, fine-tune them on your own data, and serve them through the same OpenAI-compatible API — fully air-gapped, zero external dependencies.
Full control over your model infrastructure.
Kubernetes Deployments
Helm chart for one-command cluster deployment. GPU node scheduling with resource quotas and auto-scaling. Health checks and rolling upgrades with zero downtime.
dstack Support
Native dstack backend for cloud GPU orchestration. Spot-instance cost optimisation built in. Works with AWS, GCP, Azure, Lambda Labs, and CoreWeave.
Model Library
Curated catalogue of open-weight models including Llama, Mistral, Qwen, Phi, and Gemma. One-click deployment from the admin UI with model versioning and rollback.
Fine-Tuning Jobs
LoRA and QLoRA fine-tuning on your own datasets. Job queue with GPU scheduling and progress tracking. Promote fine-tuned checkpoints to the routing layer with one click.
Fully Air-Gapped
No outbound network calls required after initial image pull. License file activation without internet connectivity. All inference, caching, and audit stays on-premises.
Gateway Integration
Deployed models register automatically as routing targets. Per-model rate limits, budgets, and guardrails apply identically. Prometheus metrics and structured logs included.
From cluster to serving in under an hour.
Install the Helm chart
Run one helm install command against your cluster. ManyLayers Deploy configures GPU node affinity, resource quotas, and persistent volumes automatically.
Pick a model from the library
Browse the curated catalogue of open-weight models in the admin UI. Click Deploy, choose a GPU node pool, and the model is serving within minutes.
Route traffic through Gateway
Deployed models appear automatically as routing targets in Gateway. Apply the same fallback chains, budgets, and guardrails as any external provider.
One platform to secure, route, and deploy — no extra integration.
Most self-hosted model platforms require a separate operations team. ManyLayers Deploy ships as part of the same unified platform — on-prem models automatically appear as routing targets in Gateway with zero glue code.
- Unified API surface — on-prem models served at the same /v1/ endpoints as cloud providers
- Fine-tuned checkpoints promoted to production with a single admin action
- GPU utilisation metrics exported to Prometheus with pre-built Grafana dashboards
- Air-gapped license activation — no internet required after the initial image pull
- Rollback to any previous model checkpoint with zero downtime
“We fine-tuned a domain-specific model and had it serving production traffic through the same API surface as OpenAI within a single sprint. The gateway integration made the transition invisible to our applications.”
ML Platform Lead — Healthcare Technology Company