Building a PII firewall for enterprise AI

Most enterprise AI deployments have a gap between their data-handling policies and their LLM usage in practice. Users paste email threads, support tickets, or internal documents into chat interfaces without thinking about what those messages contain. A single slip can send names, phone numbers, account numbers, or health information to a third-party API endpoint — creating compliance exposure that legal teams lose sleep over.

A PII firewall intercepts every prompt before it leaves your network, scans it for sensitive patterns, and either blocks the request, redacts the sensitive fields, or replaces them with synthetic placeholders before forwarding to the model.

The data egress problem in AI workflows

Traditional DLP (data loss prevention) tools were designed for email and file transfers. They inspect MIME attachments, classify document types, and enforce send policies. They were not designed for natural language prompts sent to API endpoints at high volume with sub-second latency requirements.

An LLM gateway creates a new egress channel that most DLP tools don’t cover. Users who would never email an SSN to an external vendor will paste it into a chat window without a second thought. The interface feels internal, even when the underlying API call is crossing a network boundary to a third-party provider.

The architecture fix is to put the firewall at the gateway, not the endpoint. Every AI request flows through the same chokepoint; the firewall applies universally, regardless of which application or user initiated the request.

Where the firewall lives

In the ManyLayers architecture, guardrails are evaluated in the Gateway layer — the single choke point that all model traffic flows through. This means the firewall applies regardless of which model provider the request is routed to, which team made the request, or which internal application initiated it. There is no way to accidentally bypass it by using a different API key or a direct provider endpoint, because the Gateway holds all provider credentials centrally.

Detection patterns

A PII scanner needs to handle both structured patterns (regular expressions work well) and unstructured natural language (where a small classification model is more effective):

Regex-matched structured PII

• Email addresses (user@domain.com)
• Phone numbers in common national formats
• US Social Security Numbers (XXX-XX-XXXX)
• Credit card numbers (Luhn-validated)
• IBAN and routing numbers
• IP addresses (when contextually sensitive)

Model-assisted unstructured PII

• Full names embedded in prose
• Physical addresses across international formats
• Medical record identifiers and diagnosis language
• Passport and national ID references

ManyLayers Gateway’s guardrail pipeline runs regex passes first (microsecond latency), then optionally invokes a lightweight local classifier model for the unstructured patterns. Because the classifier runs inside your infrastructure, its inference adds no data-egress risk of its own.

Redaction modes

Rather than a simple block-or-allow decision, the gateway supports three response modes per guardrail policy:

1. Block — Return a 400 error to the client with a policy-violation code. Use this for high-severity patterns like SSNs or financial account numbers where even synthetic substitution is unacceptable.

2. Redact — Replace matched spans with a type label ([EMAIL], [PHONE], [NAME]). The sanitized prompt continues to the model. The response is unaltered. Useful when the user’s intent is still clear after removal.

3. Substitute — Replace matched spans with plausible synthetic data (a realistic but fake email address, a random phone number). The model sees a coherent prompt and produces useful output. Best for document summarization tasks where structural context matters.

Policy configuration example

guardrails:
  pii_firewall:
    enabled: true
    patterns:
      - type: ssn
        action: block
      - type: credit_card
        action: block
      - type: email
        action: redact
      - type: full_name
        action: substitute
        confidence_threshold: 0.85
    audit:
      log_matched_spans: false   # never log the actual PII values
      log_policy_decisions: true

Setting log_matched_spans: false is important: you want to know that a guardrail triggered and which policy rule matched, but you do not want the audit log itself to become a store of sensitive data.

Handling model responses

PII can appear in model outputs too, particularly when the model is summarizing or extracting structured data. ManyLayers Gateway can apply a response-side guardrail pass that scans the completion before it is returned to the client. This adds a small latency increment (typically under 20ms for a regex pass on a 2,000-token response) but closes the loop on output-side leakage.

Enable response scanning in the guardrail configuration:

guardrails:
  pii_firewall:
    response_scan:
      enabled: true
      patterns: [ssn, credit_card, email]
      action: redact

Response scanning is not a substitute for prompt-side scanning. A model can hallucinate PII-like patterns that match your regexes but don’t correspond to real data — these will be redacted unnecessarily. Tune response-side patterns conservatively to avoid false positives that break the structure of legitimate outputs.

Compliance notes

A PII firewall is a control, not a compliance certification. Implementing one demonstrates due diligence and satisfies specific requirements in GDPR (data minimization, purpose limitation) and HIPAA (technical safeguards for PHI). It does not replace data classification, access controls, or contractual data-processing agreements with your model providers. The audit log that records every guardrail decision is the evidentiary artifact that matters most during an audit — keep it immutable and off-limits to the teams whose traffic it monitors.

With guardrails in place at the gateway layer, your teams can use AI tools productively without each application team building its own safety checks independently — which inevitably leads to gaps.