A hidden channel in ChatGPT: how criminals can manipulate AI
ChatGPT could discreetly extract data from Gmail boxes. Experts at Check Point discover a hidden channel in the AI agent that allows an attacker to take control of a victim's account via a simple prompt or shared link. Welcome to the new world of manipulated AI assistants. Explanations with Fred Streefland, Global CISO at Check Point and advice to avoid the worst.
Every day his pain. And whether we like it or not, we must admit it: the digital world is evolving with increasingly convoluted attacks. Recently, a new flaw discovered by Check Point in the bowels of the famous AI agent ChatGPT. More precisely, it is a hidden channel that allows you to take control of a victim's account in order to steal (with its own access rights) data from connected services such as Gmail.
Follow the white rabbit: what the laboratory experiment reveals
Check Point teams have managed to reproduce in the laboratory the modus operandi of potential hackers who would like to retrieve emails from a victim's Gmail box during a visibly normal conversation with ChatGPT.
The experiment showed that no password had been stolen, nor any malware: only the misuse of rights that the user himself had already granted to his AI assistant.
Check Point describes this phenomenon as a "coerced insider", i.e. a manipulated AI assistant, via legitimate access, to act on behalf of an attacker, within the framework of trust of an organization. Fortunately, the company behind ChatGPT, OpenAI, has since fixed the specific flaw.
Fred Streefland, Global CISO at Check Point, gives his explanation: "the mother of all attacks is the lack of visibility". And it is precisely on this point that Belgian organizations must now concentrate their efforts.
"If you don't see it, you can't protect it. In the context of AI adoption, in particular, full visibility is essential to ensure adequate security. "
Tips to protect yourself from the new wave of AI attacks
Fred Streefland recommends that CISOs "follow three fundamental principles: understand the profession, acquire and maintain full visibility, and remain "in charge" yourself.
Regarding the consequences for obligations related to NIS2, the expert is categorical: "NIS2 bonds remain fully applicable; this risk does not change this. On the other hand, it expands the attack surface of an organization, which implies in particular to strengthen the “Identify” and “Protect” domains of the NIST Cybersecurity Framework. "
A threat that goes beyond ChatGPT
Fred Streefland points out that this is not an isolated problem specific to ChatGPT: "I don't think we should use very different techniques. Just take advantage of existing possibilities and vulnerabilities, which are probably also present in Copilot and other AI applications. "
In the longer term, he does not expect a lull either: "This will always remain a problem. AI infrastructures are so complex that there will always be vulnerabilities or, as security expert Mikko Hyppönen once said about IT in general: "If it's smart, it's vulnerable". This is just as true for AI. "
Start by establishing a discovery
To be even more concrete, Fred Streefland shares a pragmatic advice: "develop a plan to secure the use of AI by employees, AI applications already used or will be used, as well as AI agents who may already be active within the organization".
This plan begins with "discovery": it is first necessary to inventory where the AI is and how employees use it.
He thus recommends that companies "proatically test their AI integrations before they are put into production, through AI Red Teaming approaches and vulnerability tests targeted on the existing infrastructure. This is in particular what Check Point did with the BLAST project, in which its own research team discovered and corrected two previously unknown vulnerabilities in its software before their disclosure."
"This discovery shows that the security challenge related to AI is no longer just about the model itself. But also the access and trust we give him," summarizes Eli Smadja, Head of Research at Check Point Research.
"Organizations must assume that any authorization granted to an AI assistant can be diverted. A "prevention-first" approach, combined with visibility, governance and real-time protection, is essential for AI to remain a development engine for the company rather than a new source of cyber risk. "
SAVE THE DATE - October 15, 2026 Hi Site, Grimbergen (Brussels)
