Microsoft has patched a critical vulnerability in its M365 Copilot AI platform that could have allowed hackers to steal two-factor authentication (2FA) codes and other sensitive information from users. The vulnerability, rated as max critical by Microsoft, was discovered by researchers who reported it to the company before going public with the details.
According to Ars Technica, the researchers revealed on Monday how their proof-of-concept exploit, called SearchLeak, could retrieve 2FA codes and other sensitive data from emails accessible to Copilot.
How the SearchLeak Exploit Worked
The vulnerability highlights a fundamental security problem with large language models (LLMs) like Copilot. According to Ars Technica, the root cause is that AI bots are unable to distinguish between instructions provided by users and those hidden in third-party content the models are summarizing, drafting responses to, or using to perform other actions on behalf of the user.
This means a hacker could embed malicious instructions in content that Copilot processes, tricking the AI into revealing sensitive information from emails, including 2FA codes that could be used to bypass security protections.
Microsoft's Response and the Patch
Microsoft acted quickly after the researchers reported the vulnerability. The company issued a patch last Tuesday, fixing the flaw before the researchers publicly disclosed the details on Monday.
According to Cybersecurity Dive, the vulnerability could have let a remote attacker steal sensitive data from an organization simply by sending a malicious request to Copilot.
The Bigger Problem with AI Security
The SearchLeak exploit is not an isolated incident. According to Ars Technica, Microsoft and other LLM providers have been unable to prevent their products from complying with malicious requests to reveal data. With no way to secure the crucial boundary between user instructions and instructions hidden in third-party content, companies are left to erect complicated and ad-hoc defenses.
This fundamental weakness means that even after patching this specific vulnerability, similar exploits could emerge in the future as long as AI systems cannot reliably distinguish between legitimate commands and malicious instructions hidden in the content they process.
Our Take: A Wake-Up Call for AI Security
This vulnerability is a serious reminder that AI systems like Copilot are not yet secure enough to handle sensitive data without risk. The fact that a hacker could steal 2FA codes — the very tools meant to protect accounts — is deeply concerning.
In our view, Microsoft and other AI providers need to rethink their approach to security. Patching individual vulnerabilities is not enough when the root cause is a fundamental design flaw. Until AI systems can reliably distinguish between user instructions and hidden commands in third-party content, users should be cautious about what data they allow AI tools to access.
For businesses using M365 Copilot, this should be a clear signal to review security policies and ensure that sensitive data, especially authentication codes, is not easily accessible to AI tools.