Privacy in LLMs: Securing Enterprise Data Without Losing Performance

Based on a research paper written by Timofey Yaluhin

The rapid proliferation of generative AI has redefined how enterprises operate, yet the underlying infrastructure often struggles to keep pace with the nuances of Data Privacy. While artificial intelligence holds the promise of unprecedented productivity, it also introduces significant privacy concerns that traditional security models are ill-equipped to handle. As companies rush to integrate advanced AI systems into their workflows, they must grapple with the fundamental tension between maximizing utility and protecting user data and intellectual property (IP).

This article explores the architectural evolution required to move beyond static, ineffective safeguards toward a robust, "privacy-by-design" approach for modern Large Language Model deployments.

The choice companies face today

Companies in regulated fields want to use the strongest AI models on their most sensitive data. A hospital wants help reading clinical notes. A law firm wants to work through privileged files. A bank wants to study financial records. The most capable Large Language Models (LLMs) for this kind of work live in the cloud, and reaching them means sending every document and every question to a provider in plain, readable form, risking the exposure of personally identifiable information.

This kind of exposure means that the data is visible to the provider, to a privileged insider, and to anyone who breaks into the service. Current privacy regulations, such as GDPR and HIPAA, along with the EU AI Act, make it impossible for companies to upload their clients' personal information to LLMs without significant compliance penalties.

Data privacy is a structural challenge. The common workaround is to keep everything in-house on private servers. That protects the data, but it forces a company to settle for weaker models that it can host itself, because the most capable models are only reachable through these cloud services.

The goal of confidential AI is to remove this choice. It aims to let a company use an untrusted provider, including frontier models and powerful hardware, without ever revealing the data to that provider.

There is no trust

Data privacy for cloud storage providers has been an ongoing challenge for years, long before AI took over the tech zeitgeist. But the real hard part that sets this challenge apart from just data storage is protecting the data in use.

Encryption handles storage and transit. A company can encrypt its documents both in transit to the provider and while they sit on disk, and the provider learns nothing. But a model cannot compute on data it cannot read. To run the model, the provider has to decrypt the documents into working memory on hardware it controls, and while the model runs, it can see the input and every intermediate value the model produces along the way. That is the part encryption cannot cover, and it is what makes data in use harder than data in storage.

It gets even more difficult when a company wants the model to work on its own documents. A model on its own only knows what it learned during training. A retrieval system, such as Retrieval-Augmented Generation (RAG), closes that gap. The company's documents are split into chunks and stored in a searchable index. When a question comes in, the system identifies the chunks most relevant to it and presents them to the model along with the question. If that index lives with the provider, the company has handed over its document store and its questions.

There is a way to soften this, and it works less well than it sounds. To make documents searchable by meaning rather than by keyword, each chunk is converted into a long list of numbers that captures what it is about, so that chunks with similar meaning end up with similar numbers. These lists are called embeddings. A company can create them on their own machines and send only the numbers, which look like anonymized data. But research has shown that embeddings can be turned back into a good deal of the original text, so the protection is thinner than it appears.

The alternative leaks more and works better. If a company sends raw text, the provider can retrieve the relevant chunks, combine them into a single long prompt, and feed that to its largest model to answer the question against the full source material. That gives the best results. If the company sends only embeddings, it has to run the generation step itself, which is far more expensive than retrieval alone. This balance between tradeoffs is one of the trickiest problems enterprises face.

The tradeoff

Architecting a secure AI pipeline requires balancing three competing forces: performance, utility, and security. Most systems fail because they treat privacy as an afterthought rather than a core system requirement. As you evaluate your AI project, you must decide which of these pillars you are willing to compromise.

So the practical question changes. Rather than searching for the best method in general, a team has to work out which trade a given application can live with, and which limit it cannot afford to relax.

What are the different approaches to LLM Privacy?

Cryptographic Approaches

Cryptographic approaches, such as Fully Homomorphic Encryption (FHE) and Multi-Party Computation (MPC), allow models to process data without ever "seeing" it in plaintext. Both work by adding noise that covers the data. FHE computes directly on the encrypted data, and the noise can only be removed by whoever holds the secret key. MPC splits the work across separate parties that are assumed not to cooperate against the user, and the noise can only be removed by combining the shares held by those parties.

Two things follow from this. The security is formal, which means it is proven rather than only tested. And the result is exact, because adding and removing the noise leaves the data intact, so no utility is lost. Of all the approaches covered here, this one offers the strongest protection for personal data.

The cost is speed. These methods can slow the LLM by a factor of 10 to 1,000, which makes them impractical for real-time conversational AI.

AI is inherently unfriendly with cryptographic approaches, whether that's non-linear vs linear, deterministic vs stochastic, precise vs approximate, and of course, scale. Good LLMs are huge, so the penalty increases with the model size.

For a single user sending one request, that cost is hard to justify. This approach may be justified when several distrustful organizations want to compute over their combined data, and none of them can be trusted to hold the raw inputs.

Trusted Execution Environments

Trusted Execution Environments (TEEs) run the model in plain form inside a sealed region of a chip, called an enclave, and use a process called attestation to prove to the user that the expected code and model are the ones running. This runs with near-zero latency, often within a few percent of an unprotected model, and the output is exact.

The trade has two parts. First, trust moves to the chip maker. Second, running an LLM this way needs a specialized confidential accelerator, a high-end chip that is scarce, expensive, and mostly available only to large enterprises. Committing to one also ties a deployment to a single vendor that may conflict with long-term data residency requirements.

Static Obfuscation

Static Obfuscation transforms the data and the model weights so the provider sees values it cannot read, using little or no special hardware. The two transformations match, so they cancel out for the user who knows them. The usual method is a secret permutation that reorders both the input tokens and the model's vocabulary, so the model and the user end up speaking the same scrambled language while the provider only ever sees the scrambled version. This is the cheapest family. It runs at nearly normal speed and, because it leaves the model structure intact, it fits into standard systems with very little extra plumbing.

A secret permutation is a substitution cipher, and it fails the way substitution ciphers have always failed. British intelligence broke the Enigma machine in part because the Germans transmitted a weather report at the same hour every morning, which handed the codebreakers a known piece of plaintext to work backwards from. The same logic applies here. Common tokens appear at predictable rates, so an attacker who watches enough traffic can match frequent tokens to their scrambled versions by counting. Every true pairing an attacker recovers makes the rest of the permutation easier to solve. Published attacks do exactly this. No amount of tuning removes the problem, because the information is still there, and a patient attacker who collects enough pairings can recover it.

In practice, static obfuscation is often paired with differential privacy. Several of these methods add calibrated noise on top of the permutation, which blunts the frequency signal that the attacks above rely on. It helps, but it does not change where the protection comes from. The secret is still doing the work, and if the secret leaks, the noise alone will not protect the data. That distinction is what separates this family from the next one.

Differential Privacy

Differential Privacy adds carefully controlled noise to blur user data while preserving the overall meaning. Its protection comes solely from randomness, so there is no secret key to lose, and the guarantee is a formal one that holds even against an attacker who knows the model and the method. The noise is fresh on every request.

The research describes two ways to use this, each protecting different parties from different threats. One adds noise on the user side to protect the question from the provider. The other adds noise on the provider's side to protect the documents stored in the system from anyone who reads the answer.

User-side DP

The user-side version blurs the input before it leaves the device, so the provider never sees the original. Once the noise is in place, inference runs at normal speed. The provider pays nothing extra, because it is doing an ordinary forward pass on values that happen to be noisy. The cost falls to the user, who has to run part of the model locally up to the point where the noise is introduced.

This protects the input, but it incurs utility costs. More privacy means more noise, and more noise means weaker output. How the noise is added matters as much as how much of it there is. Applying the same noise to every token is cheap but degrades the answer while contributing little to privacy, since most tokens were never sensitive in the first place. The better methods introduce less noise into the low-sensitivity parts of the input and more into the high-sensitivity ones.

That targeting is paid for once, up front, in a training run that learns where the sensitive parts are, so it costs nothing extra at request time. The client-side cost is separate. The deeper into the model the user runs before the noise is introduced, the better the balance between privacy and utility, and the more computing power the user's device must handle.

Provider-side DP

The provider-side version protects the documents stored in the system from anyone who reads the answer. For enterprise deployments, this is usually the more relevant of the two, because the trust assumptions are inverted. Here, the provider is trusted and holds the private data, and the client adds nothing. The exposure is that a generated answer can carry pieces of the source material back out to whoever asked, and the person asking may be an employee without clearance for those documents, a customer, or an attacker probing the system with crafted questions.

There are two methods here, and they protect different things. One treats the whole document store as a private asset. It draws a set of candidate answers, then releases only the answer keywords that survive a privacy test across that set. Anything that came from a single document does not survive, so the answer cannot confirm whether any particular document is in the store.

The other works at the level of passages within the retrieved context. It marks the sensitive spans, computes the next word with and without those spans, and blends the two until the sensitive version has moved the answer by less than a fixed threshold. Applied to each retrieved chunk, this second method also serves as a defence against prompt injection by limiting how far a poisoned document can steer the answer.

Both cost real compute at generation time, at which point the near-normal speed of the user-side version no longer applies. The first multiplies the cost of a response by the size of its candidate set, around 80 in the research. The second runs one extra forward pass per sensitive group per token, though those passes can run in parallel, so the wait can stay close to a single pass when the compute and memory are there. It does not need a custom inference engine, but it does need an inference path that can issue and combine the parallel passes.

The passage-level method only protects what its tagger marks as sensitive, so a missed span falls outside the guarantee entirely, and the fix is a tagger that catches more. The corpus-level method degrades toward the answer the model would give with no documents at all as the privacy budget tightens. It falls to that no-retrieval baseline at the strictest settings and only rises above it once the budget loosens.

One practical advantage stands out for enterprise use. This approach acts on the output, so it composes with any retrieval system and any model underneath it. A company does not have to rebuild its stack to adopt it, and it can sit on top of whatever else is already protecting the pipeline. It is the field's natural answer to attacks that pull corpus content out through the answers, which nothing further upstream will close.

Hybrid split

The hybrid approach combines the previous ideas. It keeps a small, trusted core inside a chip enclave and sends the heavy math to an ordinary, untrusted GPU in masked form, then unmasks the exact result inside the enclave.

This buys two things for data security. The scrambling is refreshed rather than fixed, which makes it stronger than plain obfuscation, and it runs on ordinary hardware rather than the scarce confidential accelerators. The output is exact. The cost is the constant back-and-forth between the secure core and the external processor.

The real bottleneck is the number of round-trip between the two. Each handoff makes the processor wait, and that waiting, rather than the computing itself, is what sets the speed.

The search layer is its own problem

The retrieval layer is a separate confidentiality problem from the model, with its own threat model. The case the research is built around is outsourcing the knowledge base itself. A company wants something like NotebookLM over its own documents, running on infrastructure it does not control. That means the provider hosts the document store, and the provider is the one the company needs protection from.

Two things leak here, and neither is what the server computes. The first is the index in storage. Embeddings can be turned back into text, so the stored vectors reveal the corpus, and the query embedding reveals the question. The second is which records each query touches. A provider that cannot read a single vector can still see which ones a query pulls, and over enough queries, that pattern alone is enough to reconstruct the database. Closing the first is cheap and mostly a matter of how the vectors are encrypted. Closing the second is not, because hiding which record matched forces the server to scan everything, which moves these systems from milliseconds to seconds at a million document scale.

Without trusted hardware, it lands on the client, which must hold the index state and run the search over several rounds. The alternative is to run the work in a hardware enclave on the server, which reduces the client's job to holding a key and restores the dependence on the scarce specialized chips.​

The two biggest gaps

Two major vulnerabilities remain largely unaddressed in current enterprise AI deployments. First, the lack of an end-to-end "privacy-preserved" pipeline. While individual components, such as encrypted retrieval or private inference, exist, there is currently no seamless architecture that maintains strict privacy guarantees from the initial user prompt through to the final synthesized response.

Second, the industry faces a significant gap in privacy-preserving knowledge graph retrieval. Most models are optimized for simple vector matches, but they fail to account for the privacy implications of mapping interconnected facts. As agentic AI systems begin to autonomously navigate complex data relationships, the risk of inferring sensitive insights from disparate, non-sensitive data points grows exponentially.

The blind spot the whole field shares

The most significant yet often overlooked risk is the physical side channel. Even when developers implement high-level AI-specific safeguards, they rarely account for the physical behaviour of hardware, such as timing, power consumption, shared memory, and traffic on the memory bus.

Every method listed above excludes protection from these kinds of leaks. This is a deliberate choice across the field, and it falls hardest on approaches whose entire trust rests on the chip, namely, hardware enclaves and the hybrid split.

Research from Georgia Tech and Purdue (the TEE.fail attack) showed that a memory-bus interposer costing under $1,000 can extract secret keys and forge attestation across Intel SGX/TDX, AMD SEV-SNP, and Nvidia GPU confidential computing.

The chip makers classify this kind of physical attack as out of scope and have no fix planned. Individual attacks get patched over time, but the category keeps recurring because each new generation of hardware adds speed optimizations that reopen avenues to observe what is happening inside.

This hardware still significantly raises the bar for attackers with only software access. The point is that this last risk cannot be closed by the method itself. It depends on the chip maker, and they tend to prioritize speed. But true LLM user data security requires addressing the entire data supply chain, including the physical infrastructure upon which the AI systems run.

What this means in practice

To build a truly resilient system, companies must move toward a zero-trust model. This involves placing a data privacy vault between user interactions and the LLM. This intermediary layer sanitizes personally identifiable information and enforces strict data governance controls before any data enters the model's context window.

By utilizing named-entity recognition and synthetic data generation, you can ensure that the LLM only ever interacts with non-sensitive representations. Furthermore, implementing rigorous adversarial testing and security regression tests allows teams to catch prompt injection attacks before they reach production. As we move into the era of agentic AI, the focus must remain on privacy-by-design, ensuring that every API request and every interaction is governed by an architectural commitment to user privacy, rather than reliance on the model's inherent safety.

Conclusion

Privacy in the age of Large Language Models requires a fundamental departure from passive security policies. As generative AI becomes more integrated into enterprise workflows, the risk of training data exposure and inference-time leakage will only grow. Success requires a multi-layered defence strategy that incorporates privacy-enhancing technologies, granular access controls, and proactive data governance. By treating privacy as a core engineering challenge, rather than a compliance issue, organizations can foster user trust and unlock the full, transformative potential of artificial intelligence while navigating the complex requirements of global privacy regulations.

The honest takeaway is that no product and no paper can promise perfect privacy for AI at no cost. Each approach asks a company to give something up, whether that is speed, utility, the size of the hardware budget, or the strength of the security proof. The right approach depends on the task. A real-time chat service is bound first by speed. A regulated workload is first bound by the kind of security it must be able to demonstrate. A team on a tight budget is bound by the hardware it already owns. A task where the answer must be exactly right is bound by how much accuracy it can afford to lose. Teams need to choose with open eyes rather than wait for a single tool that solves everything at once.

This paper is the first step in a larger line of work. Further research is planned to test these defences against real attacks and to design a practical defence of its own.

Read the full research paper:

SoK: Confidential Transformer Inference and Retrieval-Augmented Generation

Running Transformer inference and retrieval-augmented generation (RAG) over confidential data forces a choice: either expose prompts and documents to a cloud operator, or keep the data on-premises, which confines the deployment to weaker self-hosted models. Existing defenses span five mechanism fami…

IACR Cryptology ePrint ArchiveTimofey Yaluhin, ChainSafe Systems

About ChainSafe

ChainSafe is a leading blockchain research and development firm specializing in protocol engineering, infrastructure development & operations, and co-development.

ChainSafe creates solutions for developers and teams across web3. As part of our mission to build accessible, improved tooling for developers, ChainSafe embodies an open source, community-guided ethos to advance the future of the internet.

Website | Youtube | Twitter | Linkedin | GitHub