Going air-gapped: a self-hosting checklist

An air-gapped deployment runs on a network with no outbound internet access. Every dependency — software packages, model weights, license validation, telemetry — must either be pre-staged inside the environment or disabled. This checklist walks through every decision point before, during, and after deployment.

Before you start: what “air-gapped” means for ManyLayers

ManyLayers is designed for air-gapped operation as a first-class use case, not an afterthought. The software is statically compiled with no runtime network dependencies. License validation uses an offline license file. The model library can be pre-seeded from a portable bundle. No telemetry is sent.

What you are responsible for: pre-staging all artifacts inside the environment and maintaining the software update workflow manually (or via an internal mirror).

Phase 1: Environment prerequisites

Compute and storage

Confirm your compute targets:

• Kubernetes: v1.28+ recommended. Install the ManyLayers Helm chart from your internal registry mirror. GPU nodes require NVIDIA device plugin v0.14+ or equivalent.
• dstack: v0.18+ for full compatibility with the ManyLayers Deploy module. dstack manages GPU fleet allocation; ManyLayers Deploy sits at the application layer.

Storage requirements depend on the models you plan to deploy:

Model size	Recommended storage per replica
7B parameters	20 GB (fp16) / 8 GB (int4)
13B parameters	35 GB (fp16) / 14 GB (int4)
70B parameters	140 GB (fp16) / 45 GB (int4)

Provision an S3-compatible object store (MinIO is common for on-prem) as the model weight registry. The Gateway and Deploy module use this as the source of truth for model artifacts.

Network topology

Internal services that must be reachable:

• ManyLayers Gateway → model inference endpoints (Deploy module)
• ManyLayers Workspace → Gateway (all AI calls flow through Gateway)
• Platform admin clients → Gateway management API (port 8080)
• Workspace clients → Workspace application server (port 443)
• Deploy module → object storage (model weight registry)

No outbound internet access is required after initial deployment. Confirm with your network team that egress rules block all outbound traffic before go-live — this validates the air-gap rather than assuming it.

Phase 2: Artifact pre-staging

Software packages

Download the following before entering the air-gap:

• ManyLayers Gateway binary (or Helm chart + container image for Kubernetes)
• ManyLayers Workspace image
• ManyLayers Deploy binary or Helm chart
• Any connector images you plan to use (Confluence, GitHub, etc.)
• Embedding model for semantic caching and knowledge base indexing
• PII classifier model for the guardrail pipeline

All images should be pushed to your internal container registry. Update the Helm chart values files to reference internal registry paths before deploying.

Model weights

Use the ManyLayers model bundle tool to package model weights for offline transfer:

manylayers model bundle \
  --models meta/llama-3.1-70b-instruct,mistral/mistral-7b-instruct \
  --output /tmp/model-bundle.tar.gz

Transfer the bundle to your internal object store and seed the model registry:

manylayers model import \
  --source /mnt/transfer/model-bundle.tar.gz \
  --registry s3://internal-model-registry/

The Deploy module pulls weights from this registry on first deployment and caches them on node-local storage for subsequent starts.

License file

Air-gapped licenses are issued as signed JSON files. Request your license file from your account team before the deployment window. Validate it offline:

manylayers license validate --file manylayers-license.json
# Output: License valid. Expiry: 2027-01-15. Features: gateway,workspace,deploy,air-gap

Place the license file at the path configured in your deployment values or environment variable (MANYLAYERS_LICENSE_PATH). The platform checks the license file on startup and at hourly intervals — it does not phone home for validation.

Phase 3: Configuration checklist

Work through each section of the configuration before first boot.

Gateway

• Provider credentials configured for any cloud providers accessible from within your network (if any)
• Self-hosted model endpoints registered as custom targets
• PII firewall enabled with your organization’s policy
• Audit log export configured to an internal Postgres or S3-compatible sink
• Admin API authentication (API key or mTLS client certificate)
• Semantic cache configured with a locally deployed embedding model

Workspace

• SSO / SAML 2.0 connected to your internal IdP (or LDAP if SAML is not available)
• SCIM provisioning configured (or manual team provisioning if SCIM is not available)
• Knowledge base connectors configured for internal sources only
• Default routing alias pointing to self-hosted models
• Connector service accounts created with minimum necessary permissions

Deploy module

• Model registry URL pointing to internal object store
• GPU node labels configured for model scheduling
• Resource quotas set per model deployment
• Health check endpoints and readiness probe timeouts configured
• Log aggregation endpoint configured (internal Loki, Elastic, or similar)

Phase 4: Operational readiness

Update workflow

Software updates in an air-gapped environment require a staged transfer: download the new artifacts on an internet-connected system, verify checksums, transfer to the air-gap boundary, and promote to the internal registry. Document this procedure and test it before go-live — update failure is more disruptive in air-gapped environments because there is no fallback to a hosted service.

Monitoring

Configure internal monitoring for:

• Gateway request rate and error rate (Prometheus endpoint at /metrics)
• Model inference latency and GPU utilization (Deploy module metrics)
• License expiry (alert at 60, 30, and 14 days before expiry)
• Sync failure for any internal connectors

Disaster recovery

Without internet access, you cannot recover a corrupted binary from a CDN. Maintain a local copy of all deployed artifacts at a known-good version. Test the restore procedure annually: provision a clean node, restore artifacts from the backup, boot the Gateway, and verify end-to-end request handling.

Phase 5: Go-live validation

Run the following checks immediately after initial deployment:

1. Send a test request through the Gateway to the self-hosted model and confirm a valid response.
2. Log in to Workspace via SSO and confirm team membership is correct.
3. Trigger a PII firewall test: send a prompt containing a test SSN pattern and confirm the request is blocked and the block is recorded in the audit log.
4. Confirm that the Gateway produces no outbound network connections (check firewall logs for any egress attempt).
5. Export the audit log to the configured sink and confirm the export completes.

An air-gapped deployment requires more upfront preparation than a cloud-connected installation, but it delivers the strongest possible data sovereignty guarantee. Run through this checklist systematically and document each decision — the documentation is as important as the configuration for your compliance reviewers.