Artificial intelligence is reshaping the way we work and live. From generating copy with a single click and assisting with code writing to enterprise-level intelligent customer service, the gains in efficiency are evident to all. However, when we entrust trade secrets, personal privacy, and even state secrets to these “smart tools,” risks inevitably follow.
According to incomplete statistics, there have been over a hundred AI-related data breaches worldwide since the beginning of 2023. Some stemmed from momentary lapses by employees, others were caused by vulnerabilities in the platforms themselves, and still others involved hackers directly using AI to launch large-scale attacks. This article compiles six highly representative real-world cases to help you understand the risks of data leaks in the AI era and how to protect yourself.
I. Leak of Corporate Confidential Information: Samsung’s “ChatGPT Leak Scandal”
In March 2023, South Korean tech giant Samsung experienced three consecutive data breaches within just 20 days after allowing its employees to use ChatGPT.
To debug or organize data, engineers have been pasting core confidential information—such as measurement data from semiconductor equipment, source code, and even product yield rates—directly into ChatGPT to ask questions. Once this content enters the AI’s training database, it cannot be retrieved, effectively leaving it permanently exposed on the public internet.
Following the incident, Samsung immediately ordered a complete ban on ChatGPT and began developing its own in-house AI. This case serves as a lesson for all companies: what appears to be a “private” chat window in AI is actually backed by a massive cloud database, and any input can be recorded, analyzed, and reused.
II. Platform Vulnerabilities: The "Cross-Platform" Crisis Facing ChatGPT, DeepSeek, and Kimi
A Redis vulnerability in ChatGPT. In March 2023, OpenAI confirmed that a programming flaw in an open-source library caused some users to inadvertently view other users’ conversation titles; During a nine-hour window, the names, email addresses, and payment information—including the last four digits of credit card numbers—of approximately 1.2% paid users were temporarily exposed to others.
DeepSeek's database is exposed. In January 2025, cybersecurity firm Wiz discovered that a DeepSeek database was completely exposed on the public internet, accessible without any password, and containing over a million lines of log data—including chat records, API keys, and backend details— and anyone who found the correct URL could view them.
The incident involving Kimi's "resume-hopping." In April 2026, a user asked Kimi to translate an English presentation, but after several rounds of conversation, unexpectedly received a “complete resume of a stranger,” in which all information—including the person’s name, phone number, and work history—was genuine and verifiable. Experts point out that this is not a so-called “AI hallucination,” but rather an engineering oversight more closely related to a failure in conversation isolation or issues with multi-tenant data isolation.
III. Cross-Border Data Leaks: The Privacy Controversy Surrounding Meta's Smart Glasses
In February 2026, a Swedish media investigation revealed that Meta had outsourced intimate videos and private data captured by users of its smart glasses to thousands of reviewers in Kenya for use in AI training.
The report noted that this device, marketed for its “always-on awareness” feature, may capture real-time footage of extremely private settings—such as bedside tables and bathrooms—when users accidentally trigger it or while it runs silently in the background, and may even record images of financial assets like bank cards. After the incident came to light, the United States, the United Kingdom, and the European Union launched investigations one after another, and it sparked class-action lawsuits by consumers. This case serves as a warning: when devices collect data without users’ awareness and data flows across borders indiscriminately, individuals virtually lose control over their own data.
IV. AI as a Weapon: The Mexican Government Data Breach
From late 2025 to early 2026, a very small hacker group used Anthropic’s Claude Code and OpenAI’s ChatGPT to launch a series of attacks against nine Mexican government agencies over a period of two and a half months.
According to a report by cybersecurity firm Gambit Security, hackers stole approximately 195 million records of personal information and detailed tax records, 15.5 million vehicle registration records, and a large number of civil records, including birth, death, and marriage records. The attackers used a total of over a thousand prompts to have the AI automatically generate more than 5,000 commands, taking just 40 minutes to “jailbreak” Claude’s security mechanisms.
The most alarming aspect of this case is that AI enables a small number of people to launch large-scale attacks that previously required an entire team to carry out—it is both the target of the attack and the catalyst for it.
V. Gaps in Third-Party Services: McDonald’s, OmniGPT, and Muah.AI
Olivia, McDonald's recruitment robot. In 2025, researchers discovered that the platform’s administrative backend was still using the default password “123456,” and combined with an insecure API, this led to the exposure of the names, phone numbers, addresses, and conversation histories of potentially tens of millions of job seekers. The problem lay not with the AI itself, but with fundamental cybersecurity lapses.
OmniGPT's conversation leak. In 2025, this chatbot platform was hacked, and more than 34 million records of conversations between users and AI were leaked, including a large amount of sensitive content.
A leak of private content from Muah.AI. In September 2024, this “AI girlfriend” service was hacked, exposing the email addresses and private chat messages of 1.9 million users, all of which could be directly linked to their real identities.
VI. AI Face Swapping: A New Form of Crime That Bypasses Facial Recognition
The risks of AI data leaks stem not only from “input” but also from “forgery.” In July 2024, a court in Nansha, Guangzhou, handed down a verdict in a case: The defendant used AI face-swapping technology to alter personal photos into facial recognition data and forged videos showing actions such as blinking, opening the mouth, and nodding. By doing so, the defendant successfully bypassed the platform’s facial recognition real-name verification system and stole personal information from others’ accounts to sell for profit. In September of the same year, Hangzhou police also reported a similar AI face-swapping fraud case.
Cases like these remind us that when facial features can be easily replicated by AI, traditional “facial recognition” is facing unprecedented challenges.
Why Is AI More Prone to Data Leaks?
Based on the cases described above, the risks of information leaks involving AI primarily stem from three areas:
- Model Memory: Large language models can “memorize” fragments from the training data; once sensitive information enters the training set, it can be “elicited” again through carefully designed prompts.
- Platform Vulnerabilities: In a complex system (cache, connections, vector libraries, object storage), a failure in any one component could result in User A's data being displayed to User B.
- Human error: Whether employees upload confidential information to AI for convenience or companies lack control over third-party AI services, data can leak without anyone realizing it.
How to Protect Yourself? Recommendations for Businesses and Individuals
For Businesses:
- Establish clear guidelines for AI use and prohibit the input of confidential data into external AI tools;
- Classify and categorize sensitive data, and use the Enterprise Edition plan (without using customer data to train models);
- Conduct information security assessments of third-party AI services to eliminate basic oversights such as "default passwords";
- Implement data loss prevention (DLP) and outbound filtering mechanisms to intercept sensitive content in real time.
To [Name]:
- Make it a habit: Never enter sensitive information—such as your ID card number, bank card information, address, or resume—into any AI tool;
- Check your privacy settings regularly and turn off the "Train models using my conversations" option;
- Be on the lookout for AI face-swapping scams; always verify any transactions involving money or accounts through multiple sources;
- Be on the lookout for chat results that are too “clever”—if they contain content that shouldn’t be there, report it to the platform immediately.
Conclusion
AI is a double-edged sword: it makes our work more efficient, but it also makes data breaches more covert, faster, and harder to trace. True security doesn’t lie in “not using AI,” but in “using it consciously”—knowing what data must never be shared and understanding where every piece of input might end up.
Every time you enter data—no matter how quick or easy it may be—it could become a trigger for a data breach. While embracing the benefits of technology, be sure to maintain data security as your bottom line.
References
- "Data Leak at Top AI Conference Sparks New Controversy," *China Science Daily*, December 16, 2025
- "Notable Cases of AI-Facilitated Espionage and Information Leaks," Bulletin from the Ministry of State Security, 2025
- "AI Data Leaks Explained: Every Major Incident," Malware News
- "Hackers Used AI to Steal Hundreds of Millions of Mexican Government Records," Live Science
- "AI Security: Major Incidents Exposing Risks," LinkedIn / Aim Security
- "ChatGPT Linked to Alleged Leak of Samsung’s Corporate Data," Cybersecurity Help
- "My Resume Is 'On Full Display' on a Large Language Model," Daily Economic News, April 2026
- "Training Data Leakage: When Models Remember Too Much," AiSecurityDIR
Leveraging Digital Vaults to Strengthen AI Supply Chain and Confidentiality Security Defenses
In the age of artificial intelligence, cybersecurity can no longer focus solely on internally developed systems. Every library, software package, SDK, model provider, plugin, and third-party service related to development and the AI ecosystem has the potential to become part of an organization’s attack surface.
Digital Vault Committed to providing full-lifecycleSecret Management, Dynamic Key Rotation, and AI Supply Chain Risk Management
Before risks are exploited as attack vectors, understand them and, through Digital Vault Strengthen the security of your digital and AI supply chains.
Contact us:www.sest.com.hk | info@sest.com.hk
DVT Digital Vault - Over 20 years of chip-level encryption technology to accumulate data sovereignty, starting now.
