Security researchers have identified serious vulnerabilities in popular AI chatbots that could enable attackers to hijack user accounts and execute malicious code through prompt injection attacks.
Johann Rehberger, a cybersecurity expert, discovered that DeepSeek's AI chatbot was susceptible to cross-site scripting (XSS) attacks through carefully crafted prompts. By entering specific commands, attackers could potentially execute unauthorized code in users' browsers and gain access to sensitive account data.
The vulnerability allowed malicious actors to steal user session tokens stored in the browser, leading to complete account takeover. This was achieved by tricking the AI into processing Base64-encoded content that contained harmful code.
In a related finding, researchers demonstrated that Anthropic's Claude AI could be manipulated to autonomously run dangerous system commands. This technique, dubbed "ZombAIs," could force Claude to download and connect to attacker-controlled servers without user authorization.
The research team also uncovered a weakness dubbed "Terminal DiLLMa" that affects AI tools integrated into command-line interfaces. This vulnerability exploits the AI's ability to process ANSI escape codes, potentially allowing attackers to hijack system terminals.
Additional research from academic institutions revealed that ChatGPT could be manipulated to display inappropriate external content by bypassing its built-in safety constraints through prompt injection techniques.
DeepSeek has since patched the identified vulnerability in their system. However, these findings highlight the growing security challenges in AI systems as they become more widely adopted.
The discoveries underscore the need for robust security measures in AI applications, particularly those handling sensitive user data or system access. Companies developing AI chatbots are now racing to strengthen their defenses against prompt injection attacks.