AI chatbot data leaks and shadow AI

An AI chatbot data leak happens when private information you type or upload into an AI assistant ends up somewhere you did not intend: kept on the provider's servers, read by human reviewers, used to train future models, exposed through a share link, or available to whoever gets into the account. "Shadow AI" is the workplace version, where staff use AI tools the business never approved, usually on personal accounts. No hacker is needed; the leak starts with a helpful copy and paste.

• 48%¹

  of employees who use AI at work (from 32,352 surveyed in 47 countries) said they had uploaded sensitive company or customer information into public AI tools.

  The Conversation (Nicole Gillespie and Steven Lockey, University of Melbourne), summarizing Trust, Attitudes and Use of AI: A Global Study 2025 (University of Melbourne and KPMG), KPMG/UniMelb global report p.75, Fig. 44

• 34%¹

  of employees in the same survey said their organization had a policy guiding generative AI use.

  The Conversation (Nicole Gillespie and Steven Lockey, University of Melbourne), summarizing Trust, Attitudes and Use of AI: A Global Study 2025 (University of Melbourne and KPMG), Section on training, guidance and governance

• 43%²

  of breached organizations in IBM's 2026 study had security incidents involving unapproved AI use, up from 20% the year before.

  IBM Think (Oussama Kiassi), Section 'The three gaps between your data and your controls'

• 58%³

  of AI users in a seven-country 2025 survey said they had received no training on AI security or privacy risks.

  National Cybersecurity Alliance (with CybSafe), Press release, paragraph 3

• Up to 3,000 people⁴

  were potentially affected when a former government contractor uploaded a program spreadsheet to ChatGPT in March 2025.

  NSW Reconstruction Authority (New South Wales Government, Australia), Media release, October 6, 2025

• Up to 3 years⁵

  is how long Google can keep Gemini chats that human reviewers have read, even after you delete your activity.

  Google (Gemini Apps Help), Privacy Notice, 'How long we retain your data'

• Staff paste client, patient or financial details into a free or personal AI account because it is faster than asking.
• Whole files get uploaded (spreadsheets, contracts, scans, lab results) when the task needed one sentence.
• Consumer defaults favor the vendor: ChatGPT's personal plans, including paid Plus and Pro, have training turned on until you switch it off.⁹
• Memory features keep personal details and bring them back in later chats, and deleting the original chat does not always delete the memory.¹⁰¹¹
• Share links and public feeds turn a private chat into a public page.
• Thumbs-up and thumbs-down feedback can send the entire conversation to the vendor for training, even on accounts that opted out.⁹¹²
• Connected apps (email, cloud drives, calendars) pull far more data into a chat than anyone typed.
• Other people share your words: a colleague runs your email through an assistant, or a relative brings an AI into the family group chat.
• Former staff and contractors keep logins to AI accounts that hold months of company chats.
• A stolen AI account password turns your chat history into a searchable file about your life or your business.

• Use business accounts for work

  Business

  Business tiers from the major vendors say they keep your data out of model training by default: ChatGPT Business and Enterprise,¹³ Claude Team and Enterprise,¹² and Microsoft Copilot used with a work account.¹⁴ With Gemini under Google Workspace, your content isn't used to train models outside your domain without permission.¹⁵ Business plans also give an owner admin controls, such as how long chats are kept and which apps can connect; a paid personal plan is not the same thing.

  • Tools: Private and business-tier AI
  • Control: AI Acceptable Use Policy

• Write down what never goes in

  Business

  Only 34% of employees in a 2025 global survey said their employer had a policy guiding generative AI use,¹ and 58% of AI users in another 2025 survey said they had never been trained on AI privacy or security risks.³ A one-page rule that names approved tools and bans client personal data, health information, account numbers and passwords, plus a short team walkthrough, goes a long way. Our free AI Acceptable Use Policy generator (free account required) turns it into a full policy.

  • Guide: A one-page AI use rule for your team
  • Control: AI Acceptable Use Policy

• Label your data before it goes near AI

  Business

  Sort information into public, internal and confidential so staff can decide in seconds what is allowed. The same labels drive your AI rule, your sharing settings and your vendor reviews.

  • Control: Data Classification & Handling

• Switch off training and trim memory on personal accounts

  Every major assistant now has a training setting, a memory setting and a way to delete history, though the defaults differ. About five minutes per assistant covers it.

  • Guide: Lock down your AI assistants

• Use temporary or incognito chats for one-off sensitive questions

  ChatGPT's Temporary Chat and Claude's incognito chats are not used for training and do not feed memory.⁹¹⁶ They are not invisible: ChatGPT may keep temporary chats for up to 30 days and Gemini keeps temporary chats for 72 hours.⁹⁵

  • Guide: Lock down your AI assistants

• Strip names and numbers first

  Most tasks work just as well with placeholders. Replace names with "Client A", drop account and ID numbers, and paste only the paragraph that matters instead of the whole file.

• Check each vendor's AI terms when you sign up and when you renew

  Business

  Vendors change data terms; Anthropic, for example, asked consumer users to choose a training setting in 2025.¹⁷ The Federal Trade Commission (FTC) has warned companies that quietly changing privacy terms to use data for AI training may be unfair or deceptive.¹⁸ Put AI features on your vendor review checklist anyway.

  • Control: Vendor Security Assessments

• Protect the AI account itself

  Turn on multi-factor authentication (MFA) for every AI account, and remove departing staff and contractors from AI tools the same day you remove their email.

  • Control: Multi-Factor Authentication (MFA)

• Clean up share links and public posts

  Home

  Review the share links you have created and delete the ones you no longer need. In Gemini, deleting a shared chat does not delete its public link, so remove the link separately.¹⁹

  • Guide: Lock down your AI assistants

• ThreatHomeBusiness

  Prompt injection and AI agent hijacking

  When an AI assistant reads your email, files or web pages, it can mistake text written by a stranger for instructions from you. That is prompt injection, and when the assistant can also send, buy or delete things, an attacker can steer it. The UK's National Cyber Security Centre (NCSC) says it may never be fully fixed, so the safe move is to limit what your AI can reach and keep yourself in the approval loop.

  Read
• ThreatHomeBusiness

  AI-powered phishing and business email compromise

  Criminals now use AI to write fluent, personalized phishing emails and payment requests, in any language and at volume. Business email compromise (BEC), where a scammer poses as a boss, vendor or title company to redirect a payment, drew $3.05 billion in reported losses to the FBI in 2025. Typos are no longer the tell; a request to move money or change bank details is. Confirm every one on a second channel you already trust.

  Read
• GuideHomeBusiness

  Lock down your AI assistants

  Each major AI assistant has a training setting, a memory setting, connected apps and a way to delete history, and the defaults favor the vendor. This guide walks through ChatGPT, Claude, Gemini, Microsoft Copilot and Meta AI one at a time, with notes for business accounts. Plan on about five minutes per assistant.

  Read
• GuideBusiness

  A one-page AI use rule for your team

  Your team is probably using AI already, so the useful question is which tools, with what data, and who checks the result. This guide helps you write those answers on one page, get everyone to sign it, and review it twice a year. When you need more, our full AI Acceptable Use Policy generator uses the same structure and is free with a free account.

  Read
• GuideBusiness

  Digital hygiene for your business: the 1-50 person baseline

  A small business's security baseline fits on one page: passkeys or multi-factor authentication (MFA) on every account, email and banking first; a shared password manager; automatic updates and endpoint protection; and backups you've actually restored from. Add a second-channel check before any payment change, a one-page AI rule, same-day offboarding and a short incident plan. Business email compromise alone drew $3.05 billion in reported losses to the FBI in 2025.

  Read
• ExplainerHomeBusiness

  Who owns your AI conversations?

  In practice, the company that runs the assistant holds your conversations, and its terms decide who can read them, whether they train future models and how long they're kept. Some consumer plans, including ChatGPT's, use your chats for training unless you switch it off, while the major business plans say they don't by default. You can usually export or delete your chats, but deleting doesn't undo training that already happened, and legal holds can override deletion.

  Read