Hybrid deployments: control plane vs data plane

Not every organization can go fully air-gapped, and not every organization should. Full air-gap deployments deliver the strongest data sovereignty guarantee, but they also carry the highest operational burden: every software update, every new model, and every configuration change requires a manual transfer into the isolated environment. For many enterprises, the right answer is a hybrid model — sensitive data never leaves the network, but management and orchestration can use cloud connectivity.

This is the control plane / data plane split, and it’s the most common production architecture for regulated enterprises using ManyLayers.

Defining the split

The data plane is the path that prompt and response data travels. In a hybrid ManyLayers deployment, the data plane stays entirely on-premises or within your private cloud. Prompts originate from user applications inside your network, flow through the ManyLayers Gateway running on your infrastructure, and reach model inference endpoints that are also inside your network. No prompt text, no response text, and no user context crosses your network boundary via the data plane.

The control plane is the path that configuration, metadata, and management traffic travels. In a hybrid deployment, the control plane can use outbound connectivity for specific, well-defined purposes: software updates, license validation, model library metadata, and — optionally — the Gateway’s management UI if you choose to use the hosted version.

This split lets you achieve data residency requirements without carrying the full operational overhead of air-gap management.

What crosses the boundary in each direction

Outbound from your network (control plane only):

• Software update checks and binary downloads (to the ManyLayers release CDN or your internal mirror)
• License heartbeat — a signed token exchange that validates your license without transmitting any usage data
• Model library metadata — a manifest of available models, checksums, and recommended configurations
• Telemetry, if enabled (disabled by default; only aggregate counters, never request content)

Never crosses the boundary:

• Prompt text
• Response text
• User identifiers
• Team or organization names
• Embedded document content from knowledge bases
• Fine-tuning datasets

This is enforced architecturally: the Gateway’s data-plane request handling code has no network path to any external endpoint. The control-plane components are separate processes with separate network permissions.

Architecture layout

A typical hybrid deployment has three zones:

Zone 1 — Internal application network: where your applications run. Applications call the ManyLayers Gateway endpoint over an internal network path. No direct access to model providers.

Zone 2 — AI infrastructure network: where the Gateway, Workspace application server, Deploy module, and self-hosted model inference endpoints run. This zone has no outbound internet access for data-plane traffic. Control-plane processes have egress to specific external hostnames on specific ports (allowlisted outbound rules).

Zone 3 — Model provider network: cloud provider API endpoints (OpenAI, Anthropic, etc.) for any models you choose to route to cloud providers. The Gateway in Zone 2 initiates connections to Zone 3; Zone 3 has no inbound access to Zone 2.

Requests flow: Zone 1 → Zone 2 (Gateway) → Zone 3 (cloud provider, if applicable) → Zone 2 → Zone 1. At no point does Zone 3 have any knowledge of Zone 1.

Kubernetes deployment pattern

In Kubernetes, the control/data plane split maps to separate workloads with different network policies:

# Data-plane: no egress except to internal model endpoints
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: gateway-data-plane
spec:
  podSelector:
    matchLabels:
      app: manylayers-gateway
      plane: data
  policyTypes: [Egress]
  egress:
    - to:
        - namespaceSelector:
            matchLabels:
              name: model-inference
      ports:
        - port: 8080

# Control-plane: egress to management endpoints only
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: gateway-control-plane
spec:
  podSelector:
    matchLabels:
      app: manylayers-gateway
      plane: control
  policyTypes: [Egress]
  egress:
    - to: []   # configured via allow-list in gateway config

The Gateway binary runs both the data-plane and control-plane components but they listen on different ports and use different network identities in the pod spec, making network policy enforcement clean.

When to route to cloud providers vs self-hosted

A hybrid deployment doesn’t require that all inference runs on self-hosted models. Common patterns:

• Tier by data sensitivity: route prompts that contain internal document context to self-hosted models; route general-purpose queries to cloud providers.
• Tier by capability requirement: use cloud frontier models (GPT-4o, Claude Opus) for complex reasoning tasks where quality matters most; use self-hosted models for high-volume, latency-sensitive tasks where cost matters most.
• Fallback to cloud: primary inference is self-hosted; if the self-hosted model is unavailable (maintenance, hardware failure), the Gateway falls back to a cloud provider.

All of these patterns are configurable as routing strategies within ManyLayers Gateway — the data sovereignty properties are enforced by network architecture, while the routing logic is managed in the Gateway configuration.

Validating the split

Before certifying a hybrid deployment for production use, validate each direction of the split:

1. Enable Gateway request logging at DEBUG level temporarily.
2. Send a test request containing a synthetic sensitive string to the data-plane path.
3. Confirm the string appears in internal logs (Gateway audit log, model inference log) but not in any external system.
4. Check your network flow logs for outbound connections from the data-plane network — they should show connections to internal model endpoints only.
5. Confirm the control-plane components establish connections only to the allowlisted external hostnames.

This validation produces a documented evidence artifact that you can share with compliance reviewers and legal counsel to support a data residency attestation.