Strip the PII Before It Ever Reaches an AI, With Python

Everybody wants to use AI at work now, and most of them want to paste real data into it. A support agent drops a customer’s whole message into a chatbot to draft a reply. An analyst pastes a spreadsheet row into a model to summarise it. It feels harmless, and it usually is, right up until the day it isn’t.

The compliance problem underneath it is simple. The instant that customer’s name, email, or card number lands in a public AI tool, you’ve sent personal data to a third party. Under GDPR, and Sri Lanka’s own PDPA, that’s a data transfer you’re supposed to have a basis for and be able to account for. “An employee pasted it into a chatbot” is not a basis, and you can’t account for something you never saw.

You can solve most of this before it becomes a policy argument. Put a small redaction step in front of the AI, so the personal data is stripped out and replaced with placeholders before the prompt ever leaves your systems. The model still gets enough to be useful, and the sensitive bits never leave the building.

What the gate has to do

The idea is a filter that sits between your text and the AI:

Find the obvious personal data: emails, phone numbers, card numbers, IP addresses.
Replace each one with a neutral placeholder like [EMAIL_1].
Keep a private mapping of placeholder back to the real value, so a trusted system on your side can put the real details back into the final answer if it needs to.

The model works on [EMAIL_1] instead of jane.doe@gmail.com. It can still draft the reply, summarise the message, or classify the request, because the shape of the text is intact. It just never sees the person.

One catch that keeps it honest

A card number is the one field you don’t want to get wrong, in either direction. Miss a real one and you’ve leaked it. Redact every long number and you’ll wreck order IDs and reference codes that happen to be long.

So the card rule doesn’t just match digits, it validates them with the Luhn check, the same checksum every real card number satisfies. If a 16-digit string passes Luhn, it’s almost certainly a card and gets redacted. If it fails, it’s left alone. That one check is the difference between a filter people trust and one they switch off because it mangles everything.

The script

Here’s the core. Save it as redact.py.

#!/usr/bin/env python3
"""redact - strip common PII from text before it is sent to an AI service."""
import re


def luhn_ok(number: str) -> bool:
    """Return True if the digit string passes the Luhn check (real card number)."""
    digits = [int(d) for d in number if d.isdigit()]
    if not 13 <= len(digits) <= 19:
        return False
    checksum = 0
    parity = len(digits) % 2
    for i, d in enumerate(digits):
        if i % 2 == parity:
            d *= 2
            if d > 9:
                d -= 9
        checksum += d
    return checksum % 10 == 0


# Order matters: cards before phones, because a card looks like a long phone number.
PATTERNS = [
    ("EMAIL", re.compile(r"\b[\w.+-]+@[\w-]+\.[\w.-]+\b"), None),
    ("IP", re.compile(r"\b(?:\d{1,3}\.){3}\d{1,3}\b"), None),
    ("CARD", re.compile(r"\b(?:\d[ -]?){13,19}\b"), luhn_ok),
    ("PHONE", re.compile(r"\+?\d[\d\s-]{7,}\d"), None),
]


def redact(text: str):
    mapping = {}
    counters = {}
    for label, pattern, validate in PATTERNS:
        def replace(match):
            value = match.group(0)
            if validate and not validate(value):
                return value  # looked like a card but failed Luhn, leave it
            counters[label] = counters.get(label, 0) + 1
            token = f"[{label}_{counters[label]}]"
            mapping[token] = value
            return token
        text = pattern.sub(replace, text)
    return text, mapping

The order of the patterns is doing quiet but important work. Cards are checked before phones, because otherwise a card number gets swallowed by the phone rule and mislabelled. I found that out the annoying way when a test card kept coming out tagged as a phone number.

Running it

Feed it a realistic support message:

Customer Jane emailed jane.doe@gmail.com from 41.85.220.14 about order WG-1002.
Her mobile is +94 71 234 5678 and she paid with card 4111 1111 1111 1111.

And here’s what comes out:

AFTER (safe to send to an AI):
Customer Jane emailed [EMAIL_1] from [IP_1] about order WG-1002.
Her mobile is [PHONE_1] and she paid with card [CARD_1].

MAPPING (kept only on your side):
  [EMAIL_1] -> jane.doe@gmail.com
  [IP_1]    -> 41.85.220.14
  [CARD_1]  -> 4111 1111 1111 1111
  [PHONE_1] -> +94 71 234 5678

The order number WG-1002 is untouched, which is exactly right. It isn’t personal data, and the model needs it to be useful. The card was caught because it passed Luhn, the phone was caught by shape, and none of the four sensitive values are in the text you’d send onward.

Where regex runs out

Being honest about the limits is the whole point, because a filter you trust too much is worse than none.

Regex is great at structured data. Emails and cards have a shape, so they’re easy to catch. It’s useless at unstructured personal data. It has no idea that “Jane” is a name, that “the flat above the Cargills in Kandy” is an address, or that a sentence describes someone’s health. For those you need a model that understands language, not patterns.

That’s where a proper tool earns its place. Microsoft’s Presidio is open source and free, and it runs named-entity recognition to catch names, locations, and other free-text personal data that regex can’t. The sensible design is both: this fast regex gate for the structured fields, and something like Presidio behind it for the messy human ones. Regex handles the certain cases in a millisecond, and the model mops up the rest.

The compliance point

What this really buys you is a defensible answer. When someone asks whether staff are sending customer data to AI tools, “no, it’s stripped at source and here’s the code that does it” is a very different conversation from a shrug. You’ve turned a policy you hope people follow into a control that runs whether they remember to or not, and that mapping table doubles as a record of exactly what was removed.

Conclusion

You don’t need to ban AI to use it responsibly, and you don’t need to trust everyone to be careful. A few dozen lines of Python sitting in front of your AI calls removes the data that shouldn’t leave, keeps a record of what it removed, and lets people get on with their work. The certain cases are handled today, and you can bolt on a language model for the rest when you’re ready.

If you build something like this into your own workflow, let me know what you added to the pattern list. Everyone’s idea of “sensitive” is a little different.