A few months ago, a friend of mine who works in generative AI for financial services posed what seemed like a straightforward question over a pint. His team had spent months crafting system prompts for their internal LLM tools — prompts that encoded domain expertise, compliance guardrails, and carefully tuned behaviour for regulated workflows. The prompts were the product. And his question was simple: “How do I stop my users from extracting them?”
If you’ve spent any time with LLMs, you’ll know that system prompts are not secret. Jailbreaking techniques to extract them are well-documented, trivially reproducible, and getting more creative by the month. The prompt sits in the context window alongside the user’s input — it’s right there, separated by convention rather than by any meaningful security boundary. For a financial services firm where the prompts encode proprietary trading logic or compliance rules, that’s a genuine problem.
This question sent me down a rabbit hole. I wanted to understand what the actual options were — not the theoretical ones from academic papers, but approaches you could build and deploy on real cloud infrastructure. The result is a project I’ve open-sourced that implements three fundamentally different approaches to privacy-preserving AI, each drawing trust boundaries in different places and making very different trade-offs.
The core problem — there is no “prompt vault”
Before getting into solutions, it’s worth being precise about why this is hard. When you call an LLM API, the system prompt and the user message are concatenated into a single context and processed together. The model doesn’t have a concept of “privileged” versus “unprivileged” tokens — it’s all just input. Any sufficiently creative prompt injection can, in principle, get the model to reveal its instructions.
You could argue that the API provider should fix this, and some are trying with features like system message separation. But even these approaches rely on the model choosing not to reveal its instructions, which is a behavioural constraint, not a cryptographic one. If your threat model includes determined users — and in financial services, it absolutely should — you need something stronger.
Approach 1 — Federated learning, or “what if the data never left?”
The first approach sidesteps the problem entirely. Rather than trying to hide a prompt during inference, federated learning asks: what if the sensitive data never left the client in the first place?
The implementation uses the Flower framework to orchestrate federated averaging (FedAvg) across multiple simulated clients, each holding a non-overlapping partition of training data. Each client trains a PyTorch CNN locally and sends only weight updates — numpy arrays of gradient deltas — back to a central server, which averages them and distributes the improved global model.
The privacy guarantee is structural: raw training data never crosses a network boundary. The server only ever sees aggregated weight updates, never individual training examples.
There’s an important caveat, though. Weight updates can leak information about training data through gradient inversion attacks — a determined adversary with access to the gradients can, under certain conditions, reconstruct training examples. Differential privacy (adding calibrated noise to gradients before transmission) is the standard mitigation, but it trades off against model accuracy. The implementation doesn’t include this, partly because the trade-off is highly domain-specific — how much accuracy can you sacrifice for how much privacy? — and partly because being honest about limitations is more useful than pretending they don’t exist.
Best suited for: scenarios where data sensitivity is the primary concern and you can tolerate some accuracy loss. Think multi-institution medical research, where hospitals want to collaborate on a model without sharing patient records, or cross-border financial analysis where data residency laws prohibit centralisation.
Approach 2 — Fully homomorphic encryption, or “computing on secrets”
This is the one that sounds like science fiction. Fully homomorphic encryption (FHE) allows you to perform computation on encrypted data without ever decrypting it. The data goes in encrypted, the computation happens on ciphertext, and the result comes out encrypted — only the key holder can read it.
The implementation uses Zama’s Concrete-ML library to build an encrypted fraud detection pipeline. The workflow is: train a decision tree classifier on the Kaggle credit card fraud dataset, compile it into an FHE circuit using the TFHE (Torus FHE) scheme, generate encryption keys, then run inference where the input transaction data is encrypted, processed as ciphertext, and the prediction decrypted only at the end.
There’s even a Flask web UI that walks through each stage — quantisation, encryption (showing actual ciphertext hex), homomorphic computation, decryption, and prediction — making the pipeline tangible rather than theoretical.
The accuracy trade-off is surprisingly modest: the encrypted model scores about 89.5% versus 91.6% for plaintext, a drop of roughly two percentage points. The real cost is computational. The headline number is a roughly 1.5 million times inference slowdown — encrypted inference on the test set takes around 551 seconds versus 0.0004 seconds in plaintext. FHE key generation and circuit compilation each add about a second on top.
This is the fundamental FHE trade-off in 2026: mathematically bulletproof privacy, but at a computational cost that makes it impractical for anything latency-sensitive. It’s also worth noting that Concrete-ML currently only supports relatively simple models — decision trees, logistic regression, and the like. Deep learning under FHE is not feasible with current tooling, which constrains you to tabular data problems.
Best suited for: high-value, low-throughput inference where the data is so sensitive that even the compute provider can’t be trusted with it. Regulatory compliance scoring, anti-money-laundering checks on specific transactions, or medical diagnosis where patient data must remain encrypted even during processing.
Approach 3 — Nitro Enclaves, or “hardware you can trust”
This is the approach that most directly answers my friend’s original question. AWS Nitro Enclaves provide a hardware-isolated execution environment — a sealed VM with no persistent storage, no network access, and no way for the host operating system to inspect its memory. If you put the system prompt inside the enclave, it literally cannot be read from outside.
The implementation is a web-based chatbot. The user’s browser fetches an RSA public key from AWS KMS, encrypts the message client-side using the Web Crypto API, and sends the ciphertext to a Flask server running on the EC2 host. The host can’t read it — it just proxies the ciphertext into the enclave over a vsock (the only communication channel into an enclave; there’s no TCP/IP). Inside the enclave, the application calls KMS to decrypt, prepends the secret system prompt, and calls Amazon Bedrock to generate a response.
The clever bit is the attestation mechanism. When the enclave calls KMS to decrypt, it includes an attestation document containing PCR (Platform Configuration Register) values — cryptographic measurements of the exact enclave image that’s running. The KMS key policy can be configured to only allow decryption if the attestation matches, meaning only this specific enclave binary can decrypt user messages. Swap the binary, and decryption fails.
Several non-obvious engineering challenges came up during the build:
RSA message size limits. RSA-OAEP with a 2048-bit key can only encrypt about 190 bytes per operation — well under the length of a typical chat message. The solution is hybrid encryption: the browser generates a random AES-256-GCM key, encrypts the message with AES, wraps the AES key with RSA, and sends both. The enclave unwraps the AES key via KMS, then decrypts the message locally.
No network access from inside the enclave. Enclaves have zero network connectivity by design — all external communication goes through vsock-proxy tunnels on the host. This means running separate proxy instances for each external service (KMS on one port, Bedrock on another), with careful configuration to avoid port conflicts.
The EIF build toolchain. Enclave images can only be built on Amazon Linux running on a Nitro-capable instance. No local macOS builds, no CI runner shortcuts — you need an actual EC2 instance to produce the artefact. This adds significant friction to the development loop.
The attestation chicken-and-egg. Ideally, the KMS key policy would reference specific PCR values to ensure only the correct enclave image can decrypt. But you don’t know the image hash until after you’ve built it, and the CDK deployment that creates the KMS key runs before the image build. In practice, this means either a two-phase deployment or accepting a slightly weaker policy during development.
Best suited for: the original problem — protecting system prompts, proprietary model configurations, or sensitive business logic during LLM inference. The user interacts normally; the system prompt is never exposed to anyone, including the cloud operator’s host OS. The compute overhead is minimal compared to FHE, though you’re constrained to specific (and relatively expensive) EC2 instance families.
Where the trust boundaries fall
What I find most interesting about these three approaches is how differently they answer the question “who do you trust?”
Federated learning trusts the server with aggregated model updates, but not with raw data. The privacy guarantee degrades if the server is actively adversarial (model inversion attacks), but holds against honest-but-curious operators.
FHE trusts nobody with the data — not the server, not the operator, not the hardware. The ciphertext is mathematically opaque. But the computational cost is, for now, prohibitive for most real-world applications.
Nitro Enclaves trust the hardware but not the host operating system. The isolation is enforced by the Nitro hypervisor, which means you’re trusting AWS’s silicon-level security model — a reasonable assumption for most threat models, but not all.
The uncomfortable pragmatism
My friend’s original question had a deceptively simple answer in the end: Nitro Enclaves are probably the most practical path for protecting LLM system prompts today. The overhead is manageable, the trust model fits financial services well (you’re already trusting AWS with your compute), and the engineering, while fiddly, is tractable.
But “practical” and “perfect” are different things. The broader landscape of privacy-preserving computation is still maturing. FHE performance improves by roughly an order of magnitude every few years — at some point, the 1.5 million times slowdown becomes a 150 times slowdown, and then we’re in genuinely interesting territory. Federated learning is already production-grade for certain use cases, particularly in healthcare and cross-border finance.
The real lesson from building all three is that privacy-preserving AI isn’t a single problem with a single solution. It’s a spectrum of trade-offs between security guarantees, computational cost, engineering complexity, and the specific trust model your organisation needs. The question isn’t “which approach is best?” — it’s “what exactly are you trying to protect, and from whom?”
The code is open-source on GitHub if you want to deploy any of these approaches yourself. Each sub-project is self-contained with its own CDK infrastructure stack, so you can go from zero to a running demo on AWS without stitching things together. If nothing else, watching the FHE web UI churn through a single encrypted inference is a humbling reminder of just how hard some of these problems are.
This post also appears on my Substack.