Decision Models Vulnerable to Indirect Prompt Injection, Researchers Find
Despite claims of enhanced security, decision models like TypeSafe AI's Jev remain susceptible to prompt injection attacks, potentially allowing attackers to hijack agent actions.
Researchers from the security firm HiddenLayer have discovered that decision models, which are designed to output probabilities and confidence scores rather than freeform text, are still vulnerable to indirect prompt injection attacks. This finding challenges the notion that these models, such as TypeSafe AI's Jev, are inherently more secure than traditional large language models (LLMs) due to their different output format.
The core of the vulnerability lies in the underlying architecture. HiddenLayer's analysis indicates that decision models, often referred to as System One models, are fundamentally LLMs that employ a different decoding strategy. This shared foundation means they inherit many of the same risks associated with LLMs, including susceptibility to manipulation through carefully crafted inputs.
In their experiments, HiddenLayer researchers simulated an online store environment with pre-filled shopping carts and a 'Place order' button. They instructed a Jev-based browser agent to purchase items without modification, while also imposing a rule that page text should be treated as untrusted data. The system prompt was designed to enhance security by preventing the model from interpreting web content as instructions.
However, the researchers successfully exploited this setup by introducing a malicious sponsored ad on the simulated webpage. This ad presented a "Confirm Price" button, falsely suggesting that the sale price had not yet been applied. When the Jev agent encountered this, it deviated from its primary task and clicked the button, demonstrating a successful hijack of its intended action.
This attack vector was also tested against a traditional LLM, Qwen3.8-27B, which was tuned to function as a decision model. Both models exhibited vulnerability, though Jev proved to be slightly more resistant. The ability to trick the model into clicking a button on a simulated webpage highlights how attackers could potentially chain such actions to gain significant control over an AI agent's operations.
Beyond hijacking actions, the research also indicated that decision models are susceptible to prompt verification bypasses. A variant of the 'tokenbreak' attack, which involves adding extraneous characters to evade content filters, was used to sneak toxic prompts through Jev's prompt verification process. This suggests that even when used for security-focused tasks like prompt verification, these models require robust safeguards.
TypeSafe AI has not yet responded to requests for comment on these findings. The research underscores that while decision models offer potential advantages in cost and performance by returning structured outputs, they are not immune to the sophisticated prompt injection techniques that plague traditional LLMs. Security professionals are advised to approach these models with caution and implement appropriate security measures.
The implications extend to any system that relies on LLM-based decision-making. As AI agents become more integrated into workflows and automated processes, ensuring their security against manipulation is paramount. The research serves as a critical reminder that the underlying technology's vulnerabilities must be addressed, regardless of the model's specific output format or intended application.