A Layered Privacy Sanitization Pipeline for LLM Applications

AI applications today chew through huge amounts of unstructured and semi-structured data, and plenty of it is sensitive: personal details, credentials, financial records, confidential business information. Before any of that gets logged, stored, analyzed, sent elsewhere or handed to a downstream system, you need a dependable way to find the sensitive parts and protect them without wrecking the data's usefulness.

Where plain masking falls short

Classic masking leans on regular expressions and predefined patterns. That works well for structured values like emails, phone numbers and credit card numbers. It struggles when sensitivity depends on context. Take this sentence:

The customer's recovery answer is BlueMountain.

Nothing about the string BlueMountain looks sensitive to a pattern matcher. NLP models recognize entities better but can still miss secrets specific to a domain. LLMs understand context, yet running every input through one adds latency, cost and unpredictable behavior. The sensible answer is to layer the detection.

Three layers and a policy engine

The design combines three detection mechanisms:

1. Rules, for patterns that are predictable.
2. NLP, for entities and language-level recognition.
3. An LLM, which classifies contextual and domain-specific sensitive information.

Everything they find goes to a policy engine that picks the right protection action. The aim isn't to make each layer equally smart. It's to use the simplest dependable mechanism for each kind of data.

Architecture

Build it as a reusable privacy sanitization layer that sits between the applications producing data and the systems consuming it.

Rules with Presidio

Microsoft Presidio is a good base for finding structured and known sensitive values. It ships with recognizers built on patterns, others built on NLP, and a way to write your own. Typical targets include emails, phone and credit card numbers, IP addresses, government-issued identifiers, API keys, and any internal business IDs you define. Rule-based detection is fast, deterministic and easy to explain, which makes it the right first layer.

NLP entity detection

Some entities can't be caught reliably with a regex: person names, locations, organizations, dates and domain-specific entities. Consider "Rahul Sharma moved from Mumbai to Bengaluru." An NLP model picks out the names and places even though they follow no fixed format.

Finding an entity doesn't make it sensitive, though. A policy layer still has to decide whether the entity needs protection.

LLM classification for context

Certain information only reveals itself through its surroundings. With "The customer's security answer is BlueMountain", a traditional recognizer may not flag the word, while an LLM can work out its role from the text around it. Ideally the model acts as a classifier and not as a free-form rewriter. Its output might look like this:

{
  "text": "BlueMountain",
  "category": "SECURITY_ANSWER",
  "action": "MASK",
  "confidence": 0.96
}

Structured findings are simpler to validate, audit and feed into deterministic anonymization logic.

Anonymization driven by policy

Detection only surfaces candidates. The policy engine decides what happens to each one, and the action depends on the category:

• Emails get masked and phone numbers only partly masked
• API keys are removed outright
• Customer IDs are swapped for tokens
• Security answers are redacted
• A public company name can pass through untouched

Keeping this mapping separate means privacy rules can change without retraining or swapping any detection model.

Sequential or parallel

There are two usual ways to run the layers. In a sequential setup, the LLM only sees text that earlier layers have already cleaned. That limits how much sensitive data reaches it, though it may strip out context the classifier needed. In a parallel setup, every detector works on the raw input at the same time. Context stays intact, but raw data may reach the LLM classifier.

Which one fits depends on the model's trust boundary, the deployment environment, latency needs and privacy constraints. One way to avoid exposing raw data is to run on-premise models.

Cost, security and usefulness

Sending every input to an LLM is neither affordable nor needed. A tiered setup is more efficient:

• Structured fields go to rule-based detection
• Known entities go to Presidio or NLP
• Ambiguous free text goes to the LLM for classification
• High-risk data falls under strict masking policies

The system also has to avoid over-masking. Removing everything protects privacy but makes logs and traces nearly worthless. A good design balances privacy protection, detection accuracy, performance and data utility.

Worked example

A raw record:

{
  "email": "rahul@example.com",
  "customer_name": "Rahul Sharma",
  "recovery_answer": "BlueMountain",
  "transaction_id": "TXN-123456"
}

And the sanitized version:

{
  "email": "<EMAIL>",
  "customer_name": "<PERSON>",
  "recovery_answer": "<SECURITY_ANSWER>",
  "transaction_id": "<TRANSACTION_ID>"
}

Evaluate the system on precision and recall, false negative and false positive rates, sanitization latency, how often inputs escalate to the LLM, cost per processed record, and how much the data is still worth after masking.

Why layers

No single technique detects sensitive data reliably. Rules bring speed and determinism, NLP improves entity recognition, LLMs add an understanding of context, and the policy engine turns detections into the right protective action. Stacked together, they protect what matters while keeping the rest of the data useful, and that's the actual goal, because masking everything was never it.