Air-gapped AI for regulated industries

About this session

Organizations in healthcare, defense contracting, financial services, and government face a paradox: the productivity gains from large language models are clear, but their compliance frameworks prohibit sending sensitive data to third-party API endpoints. The common response — “we’ll wait until there’s a compliant offering” — has a cost that compounds every quarter as AI-enabled competitors move faster.

This session shows a practical path through that paradox. ManyLayers runs entirely within your network boundary. Model inference happens on your hardware. No prompt, no response, and no user data crosses your perimeter. The session covers the full deployment lifecycle for a regulated environment, from initial architecture through operational handoff.

What you will learn

• How to map your compliance framework (HIPAA, FedRAMP Moderate, ISO 27001, SOC 2 Type II) to specific ManyLayers configuration decisions
• Network architecture for an air-gapped deployment: what must be reachable, what must be blocked, and how to validate the isolation
• Offline license activation and the annual renewal workflow without internet access
• Configuring the audit log for compliance evidence: immutability, retention, export, and chain-of-custody requirements
• PII firewall configuration for regulated data types: PHI, PCI-DSS cardholder data, and CUI
• Model weight distribution in an air-gapped environment: bundle packaging, checksum verification, and internal registry setup
• How to present this architecture to your compliance team, legal counsel, and auditors — including the evidence artifacts the platform generates

Session agenda

0:00 – 8:00 — Compliance landscape overview: what HIPAA, FedRAMP, and SOC 2 Type II actually require from an AI deployment, and where most platforms fall short

8:00 – 18:00 — Architecture walkthrough: network diagram for a fully air-gapped ManyLayers deployment on Kubernetes; how the Gateway, Workspace, and Deploy modules communicate internally; what egress rules block and why

18:00 – 30:00 — Live configuration demo: offline license file setup, PII firewall policy for PHI patterns, audit log export to an internal Postgres sink, and admin API authentication with mTLS client certificates

30:00 – 40:00 — Model deployment in isolation: pulling weights from an internal MinIO registry, deploying Llama 3.1 70B on bare-metal GPU nodes with dstack, and running a readiness check without internet access

40:00 – 48:00 — Audience Q&A: common questions on FedRAMP authorization path, multi-tenant vs single-tenant deployments for regulated workloads, and disaster recovery in isolated environments