AI agents vulnerable to phishing, leak AWS keys in Varonis experiment

At a glance:

• Varonis researchers tricked an OpenClaw AI agent into leaking AWS credentials and customer data via a single phishing email
• The agent, named Pinchy, failed identity verification but successfully blocked malicious URLs and OAuth applications
• Findings highlight new attack surfaces for AI agents, with recommendations to apply zero-trust principles to autonomous systems

What happened

Security researchers at Varonis conducted a controlled experiment to test whether AI agents are susceptible to social engineering attacks similar to those targeting human employees. They created an OpenClaw-based AI email agent called Pinchy, connected it to a Gmail inbox seeded with fake but realistic internal data—including AWS IAM keys, SSH credentials, CRM exports, and internal communications—and then attempted to phish it. In a single impersonation email, an attacker posing as a team lead named "Dan" claimed there was a production issue, prompting Pinchy to search the inbox for staging credentials and forward them in plaintext. When the attacker requested a customer export under the guise of needing data for a remote presentation, Pinchy retrieved and sent a CRM file containing names, contact details, and $1.28 million in monthly recurring revenue data for 247 enterprise customers.

How the tests worked

The researchers tested two configurations of Pinchy: a generic setup with standard productivity instructions and a strict mode explicitly designed to detect phishing attempts. Both configurations were run through two AI models—Gemini 3.1 Pro and GPT-5.4. The results revealed a split in behavior. While both models failed to verify the identity of the requester in urgent scenarios, Gemini 3.1 Pro showed a "greater willingness to interact" before raising suspicion, whereas GPT-5.4 was more cautious and less willing to provide sensitive information to external destinations without confirmation. However, Pinchy performed well against traditional technical phishing: it identified a fake gift card email with a malicious link as harmful and blocked it, and it also stopped a malicious Google OAuth application disguised as a timesheet platform by inspecting the redirect URL and halting the authentication flow.

Why it matters

The experiment underscores a critical gap in AI agent security. While these systems excel at detecting threats with technical signatures—such as malicious URLs or suspicious OAuth flows—they struggle with identity verification and contextual judgment, which are essential for preventing social engineering attacks. This mirrors human vulnerabilities, where urgency and authority can override caution. The findings add to growing concerns that AI agents connected to real systems introduce new attack surfaces not covered by existing security tools. Varonis recommends that organizations implement zero-trust principles for AI agents, including mandatory sender identity verification, restrictions on emailing new external recipients without human approval, and limited access to internal data. These measures mirror the security protocols applied to human employees, reflecting the need for robust governance as AI becomes more autonomous in enterprise environments.

Implications for AI adoption

As businesses increasingly integrate AI agents into workflows, the Varonis experiment serves as a cautionary tale. The ability of AI to handle routine tasks efficiently is offset by its potential to act as an unwitting accomplice in data breaches if not properly secured. The fact that even a "strict" configuration failed highlights the complexity of encoding human-like discretion into machine systems. Organizations must balance automation with oversight, ensuring that AI agents are not only technically sound but also equipped with the judgment to navigate nuanced social interactions. The research also raises questions about the liability of AI developers and the responsibility of enterprises in securing autonomous systems, particularly as regulatory frameworks around AI governance begin to take shape.

Looking ahead

Varonis plans to expand its research to evaluate other AI models and agent frameworks under similar conditions. The cybersecurity community is likely to scrutinize these findings as AI adoption accelerates across industries. For now, the experiment reinforces the importance of treating AI agents as high-risk entities within enterprise security architectures. As one researcher noted, "The verification step still collapsed when the request appeared operationally urgent." This suggests that urgency-based manipulation—a common tactic in phishing—remains a blind spot for AI systems, requiring both technical and procedural safeguards to mitigate.