Knowledge base AI: a practical guide for SMBs
May 23, 2026
TL;DR:
- Encrypting data at rest and in transit is insufficient because AI training and inference can leak sensitive information through attacks that bypass standard security measures.
- A layered approach across infrastructure, identity, and applications is essential to effectively safeguard AI confidentiality during all data lifecycle phases.
- Implementing technical controls such as differential privacy, confidential computing, and strict data classification helps SMBs address specific AI-related threats and reduces data exposure risks.
Most business leaders assume that encrypting data at rest and in transit is enough to keep confidential data safe when using AI. It is not. The real risks arrive during training and inference, where AI systems can expose sensitive information through attack vectors that traditional security tools simply do not address. For SMB leaders and IT professionals adopting AI across finance, legal, or operations, the gap between what you think is protected and what actually is can be significant. This guide breaks down the specific threats, the technical defences that work, and the practical steps your organisation can take now.
Table of Contents
- Key takeaways
- Confidential data AI: the threats you are not watching for
- Layered controls across the AI security stack
- Technical approaches worth understanding
- Practical steps to safeguard AI data in your organisation
- My perspective on what actually works for SMBs
- How Done can help you secure your AI deployment
- FAQ
Key takeaways
| Point | Details |
|---|---|
| Encryption alone is insufficient | AI-specific attacks like membership inference and gradient reconstruction bypass standard encryption entirely. |
| Layered controls are non-negotiable | Effective confidential information protection requires infrastructure, identity, and application-layer controls working together. |
| Technical tools exist for SMBs | Differential privacy and confidential computing are no longer enterprise-only; vendors now offer accessible implementations. |
| Real-time DLP makes a difference | Configuring data loss prevention for AI prompts actively prevents sensitive data from leaving your environment. |
| Vendor claims need scrutiny | Ask vendors for evidence of remote attestation and independent testing, not just policy statements or marketing materials. |
Confidential data AI: the threats you are not watching for
When organisations think about securing AI systems, most of the conversation centres on protecting stored data. But AI training and inference can leak confidential information through three specific attack types that have nothing to do with your database encryption settings.
Membership inference attacks are the most common. An attacker sends carefully crafted queries to your model and observes the outputs. From those outputs, they can determine whether a specific individual’s data was used in training. For a law firm or an accounting practice, this could mean revealing that a particular client’s records were processed. The attack requires no access to your systems. It only requires access to the model’s responses.
Gradient reconstruction is a more technical but equally serious threat. In federated learning setups, where multiple parties contribute to training a shared model without sharing raw data directly, the gradients exchanged during training can be reverse-engineered to reconstruct the original inputs. This means that customer invoices, contract terms, or patient records you thought were never shared could be recovered by an adversary with access to those gradient updates.
Training data extraction is perhaps the most alarming for SMBs using shared or public AI platforms. An attacker sends specific prompts to a model and extracts verbatim content from the training data. Experiments have demonstrated this successfully with real-world models. If your sensitive documents were used to fine-tune a model, portions of those documents can potentially be retrieved by anyone with API access.
The important point here is that defences must protect intermediate artefacts, not just stored datasets. This is where most standard security frameworks fall short. They are built around protecting files and databases, not model outputs and gradient exchanges.
For SMBs specifically, the exposure is real. Consider a small financial advisory firm using an AI tool to summarise client portfolios. If that tool was fine-tuned on client data without proper privacy controls, those summaries could inadvertently reveal information about individual clients to other users of the same system. It is a scenario we have seen discussed repeatedly in the broader security community, and it is not theoretical.
Traditional encryption does not help here because the attack does not target encrypted data. It targets the model’s learned behaviour. You need different defences for different threats.
Layered controls across the AI security stack
The answer to these threats is not one technology. It is a structured approach that applies controls at three distinct layers: infrastructure, identity, and application.
Protecting AI data confidentiality requires layered lifecycle controls that address every phase of data interaction, from storage through training to inference. No single layer is sufficient on its own.
Here is how to think about each layer:
-
Infrastructure isolation. Your AI workloads should run in environments that are logically and physically separated from general business systems. Trusted Execution Environments (TEEs), sometimes called secure enclaves, are hardware-level protections that keep data encrypted even while it is being processed. The host operating system, cloud administrators, and infrastructure operators cannot see the data in plaintext. This matters enormously when you are using a managed cloud service, because it limits what even your cloud provider can access.
-
Identity and access management for AI agents. This is an area where most SMBs have significant gaps. AI workloads require zero trust identity models, meaning each AI agent or automated workflow must have its own scoped identity with the minimum permissions required. Sharing credentials between human users and AI processes is a major risk. If an AI agent is compromised, broad credentials mean broad exposure.
-
Application-level controls. This layer sits closest to your users and your data. It includes prompt filtering to block sensitive content from entering AI queries, output monitoring to flag unusual data patterns in AI responses, and behavioural monitoring to detect anomalous AI activity over time. These controls operate in real time and are the most visible part of your security posture.
-
Data governance at the source. Before any of the above can work, you need a clear map of what data is sensitive and where it lives. Without data classification, your controls have no foundation to act on.
The mistake many organisations make is investing heavily in one layer, typically infrastructure, and assuming the others will follow. They do not. An organisation can run workloads in a TEE but still leak sensitive data through an AI prompt if application-layer controls are absent.
Pro Tip: When reviewing your current AI security posture, map each control to a specific data lifecycle phase: at rest, in transit, and in use. If any phase has no control assigned to it, that is a gap worth addressing before expanding your AI usage.
Technical approaches worth understanding
Three technologies sit at the heart of machine learning data security: differential privacy, secure multiparty computation, and confidential computing. Each addresses a different threat surface, and understanding what they do and do not offer will help you evaluate vendor claims with more confidence.
| Technology | What it does | Limitations to know |
|---|---|---|
| Differential privacy (DP) | Adds calibrated noise to training data or outputs, making it statistically difficult to infer individual records | DP-SGD models retain 78–82% accuracy while reducing inference risk; accuracy trade-offs increase with smaller datasets |
| Secure multiparty computation (MPC) | Allows multiple parties to collaboratively train a model without exposing raw inputs to any single party | High computational cost; practical primarily for federated learning with known, trusted participants |
| Confidential computing | Uses hardware TEEs with remote attestation to keep data encrypted during processing, even from infrastructure administrators | Requires careful integration; skipping attestation weakens guarantees significantly |
Differential privacy is the most mature of the three for SMB use cases. If you are fine-tuning a model on customer records or financial documents, DP can be applied during training so that the resulting model cannot easily reveal whether any specific individual’s data was included. The trade-off is a modest reduction in model accuracy, and that accuracy impact depends on dataset size and task. For most SMB applications, the trade-off is acceptable.
Confidential computing is increasingly relevant as more AI workloads move to cloud GPU infrastructure. NVIDIA’s zero-trust architecture for confidential AI prevents plaintext data exposure to infrastructure operators by using hardware-backed TEEs with a Key Broker Service that only releases decryption keys after remote attestation verifies that the workload is running in a trusted enclave. This is a meaningful step beyond policy-based access controls.
When evaluating vendors, ask specifically whether they support remote attestation and whether you can verify the attestation report independently. A vendor who can only point to a policy document rather than a technical mechanism is offering you assurance, not protection.
Pro Tip: If a vendor tells you your data is “secure in their AI platform” without mentioning differential privacy, TEEs, or data isolation architecture, ask them to be specific. Vague assurances are not a substitute for technical controls.
Practical steps to safeguard AI data in your organisation
Understanding the threat landscape is useful. Having a clear set of steps you can act on is what matters most for a busy IT team or a business leader who needs to make decisions.
Here is a practical framework for how to safeguard AI data across your operations:
-
Classify your data before connecting it to AI. Build or adopt a data classification scheme that distinguishes public, internal, confidential, and restricted data. AI tools should only access the category of data their function requires. This single step eliminates entire classes of accidental exposure.
-
Configure DLP for AI prompts. Microsoft Purview DLP blocks sensitive data from being sent to external web search in Copilot products, with real-time evaluation at the prompt level. This means a user typing a client’s financial details into a Copilot chat will be stopped before that data reaches an external endpoint. Setting this up requires configuration work, but it is one of the most direct controls available for Microsoft 365 environments.
-
Apply zero trust principles to AI agents. Every automated workflow, AI assistant, or integration connector should operate under its own identity with the minimum necessary access. Review your current AI tool integrations and check whether they run under shared admin credentials. Many SMBs discover they do.
-
Ground your AI outputs internally. Restricting AI tools from accessing external web sources during inference is an underused control. AI grounding and data loss prevention can co-exist, and internal grounding reduces the surface area for data exfiltration significantly. For most business productivity use cases, internal grounding is sufficient and safer.
-
Set up monitoring for AI activity. Standard security information and event management (SIEM) tools often do not capture AI-specific events. Work with your AI platform vendors to identify what logs are available, what constitutes anomalous behaviour for your use case, and how incidents involving AI-generated outputs should be handled.
-
Test vendor claims rigorously. NIST’s TEVV framework provides a structured approach to testing, evaluating, verifying, and validating AI systems for trustworthiness. Use it as a checklist when onboarding a new AI vendor. Ask whether the vendor has conducted or published independent security evaluations.
For a broader view of how these steps connect to your GDPR obligations as a European business, Done’s GDPR AI compliance guide covers the regulatory dimensions that sit alongside these technical controls.
My perspective on what actually works for SMBs
I have worked with enough SMB clients on AI adoption to say plainly that most of them come to us with the same blind spot. They have invested in a reputable cloud provider, they have HTTPS everywhere, their databases are encrypted. They feel secure. And then they connect a business intelligence AI tool to their CRM, grant it broad read access, and assume the hard work is done.
The uncomfortable truth is that compliance and encryption give you a foundation, but they do not address the layer where AI systems actually introduce new risk. Training-time and inference-time exposures are not covered by your current security audit. They require different thinking.
What I have seen work consistently is treating AI security as an ongoing operational discipline rather than a one-time configuration task. The organisations that do this well have three things in common. They know precisely what data their AI tools can access. They have at least one person with clear ownership of that question. And they revisit it every time a new AI tool is added or an existing one is updated.
Single silver bullets do not exist here. Differential privacy protects training. TEEs protect processing. DLP protects prompts. You need all three to work together, and the complexity has to be proportionate to your team’s capacity to manage it. A ten-person accounting firm cannot implement the same architecture as a major financial institution. But they can classify their data, restrict AI access to sensitive client records, and configure the DLP tools that come with the software they are already paying for.
Vendor transparency is the last piece I want to stress. If your AI vendor cannot explain their data isolation model in specific technical terms, that is a signal worth paying attention to. In our experience, vendors who are doing this properly are usually eager to show their work.
— Thomas
How Done can help you secure your AI deployment
If this article has raised questions about your current AI setup, that is a healthy response. The gap between knowing these risks exist and having a structured plan to address them is exactly where Done works with SMBs across Luxembourg and Europe.
Done’s AI strategy consulting service is designed specifically for organisations at the stage of expanding AI use into sensitive workflows. We help you map your data assets, assess your current controls against the threat types covered in this article, and build a prioritised plan that fits your team’s capacity. We also support private, on-premise AI deployments for businesses in legal, finance, healthcare, and accounting where data sovereignty is a hard requirement. If you are ready to move beyond general AI tools and need a deployment model that keeps confidential data genuinely protected, speak to our team about what that looks like for your business.
FAQ
What is a membership inference attack?
A membership inference attack is a technique where an adversary queries an AI model to determine whether a specific individual’s data was used in training. It requires no access to the underlying data, only the model’s outputs.
Is encrypting data at rest enough to protect confidential data in AI systems?
No. Encryption at rest does not protect against AI-specific attacks such as membership inference, gradient reconstruction, or training data extraction, which exploit model behaviour rather than stored files.
What is differential privacy and should SMBs use it?
Differential privacy adds statistical noise during AI model training to prevent individual data records from being inferred from the model’s outputs. Models trained with DP retain roughly 78–82% accuracy while significantly reducing inference risk, making it a practical option for most SMB use cases.
How does confidential computing protect AI workloads?
Confidential computing uses hardware-based trusted execution environments (TEEs) to keep data encrypted during processing, even from cloud administrators. Remote attestation verifies cryptographically that a workload is running in a genuine secure enclave before decryption keys are released.
What is the first practical step an SMB should take to protect confidential data in AI?
Classify your data before connecting it to any AI tool. Knowing which data is confidential and restricting AI access to only what is necessary eliminates a large proportion of exposure risk before any technical controls need to be deployed.
