AI Gateway with PII Redaction for LLM Applications

Why PII Protection Matters for LLMs

Consider a common scenario in a healthcare application:

{
  "model": "gpt-4.1",
  "messages": [
    {
      "role": "user",
      "content": "Summarize this patient record: John Smith, SSN 123-45-6789, DOB 03/15/1982, diagnosed with Type 2 diabetes on 01/10/2025. Prescribed Metformin 500mg."
    }
  ]
}

Without a gateway, this prompt — containing a real name, SSN, date of birth, and medical diagnosis — is sent unmodified to the model provider's servers.

Architecture: AI Gateway with PII Detection

AI ModelGate implements a multi-stage pipeline that inspects every request before it reaches any downstream provider:

The AI Firewall Detection Engine

Unlike simple regex-based filters, the AI ModelGate firewall uses a multi-layered detection engine that combines four techniques to minimize false negatives without sacrificing latency:

This combined approach adds fewer than 50 milliseconds of latency per request for text and approximately 0.5–1 second for image payloads (vision security). Every response includes x-dlp-latency timing headers so you can verify performance in production.

Example: PII Redaction Flow

Supported Entity Types (28 total)

Implementing PII Redaction in an AI Gateway

AI ModelGate is a drop-in replacement for the OpenAI API. Point your existing SDK at the ModelGate endpoint and PII redaction happens automatically — typically a 2-line change: set baseURL and apiKey. Optional bring-your-own-key (BYOK) lets you supply provider credentials while ModelGate still enforces DLP and routing.

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "os_hub_your_key_here",
  baseURL: "https://api.aimodelgate.ai/v1",
});

const response = await client.chat.completions.create({
  model: "oah/gpt-4.1",
  messages: [
    {
      role: "user",
      content: "Summarize this patient record: John Smith, SSN 123-45-6789",
    },
  ],
});

// ModelGate automatically:
// 1. Scans the prompt for PII entities
// 2. Redacts "John Smith" → [PERSON], "123-45-6789" → [US_SSN]
// 3. Forwards the cleaned prompt to the provider
// 4. Returns the response with x-dlp-latency timing header

from openai import OpenAI

client = OpenAI(
    api_key="os_hub_your_key_here",
    base_url="https://api.aimodelgate.ai/v1",
)

response = client.chat.completions.create(
    model="oah/gpt-4.1",
    messages=[
        {
            "role": "user",
            "content": "Summarize this patient record: John Smith, SSN 123-45-6789",
        }
    ],
)

# PII is redacted before the request reaches OpenAI.
# Check response headers for scan timing and violation counts.

curl -X POST https://api.aimodelgate.ai/v1/chat/completions \
  -H "Authorization: Bearer os_hub_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "oah/gpt-4.1",
    "messages": [
      {
        "role": "user",
        "content": "Summarize this patient record: John Smith, SSN 123-45-6789"
      }
    ]
  }'

# Response headers include:
# x-dlp-latency: 12
# x-request-id: req_xxxx
# hub_metadata.entity_types_detected: ["PERSON","US_SSN"]

Violation Response Format

When the firewall detects a BLOCK-level entity (like a prompt injection attempt), it rejects the request immediately with a structured error response:

{
  "error": {
    "message": "Security policy violation: request blocked.",
    "type": "security_violation",
    "code": 400,
    "violations": [
      {
        "entity": "PROMPT_INJECTION",
        "action": "BLOCK",
        "start": 0,
        "end": 47
      },
      {
        "entity": "US_SSN",
        "action": "REDACT",
        "start": 52,
        "end": 63
      }
    ],
    "correlation_id": "req_a1b2c3d4"
  }
}

Supported AI Providers

PII redaction works identically across all providers. Use a single gateway endpoint and virtual model names (oah/*) — ModelGate handles provider routing automatically, including smart cost routing to the least expensive compatible backend when you opt in.

Benefits of an AI Security Gateway

Try It with AI ModelGate

Get started in under five minutes. No credit card required for the free tier — every request is protected from your very first API call. Integration is usually a 2-line OpenAI SDK change (base URL + ModelGate key); use wallet top-ups and per-project dashboards to track usage alongside DLP events.