Researchers at Binghamton University, State University of New York, have developed a security system designed to detect malicious instructions in artificial intelligence-powered robots after demonstrating how hidden trigger words in large language models (LLMs) could cause robots to perform potentially harmful actions.
The research examines security vulnerabilities associated with using LLMs to generate task plans for robots. These models, including technologies underlying OpenAI’s ChatGPT, Anthropic’s Claude and Google Gemini, can translate user instructions into sequences of actions that robots execute in physical environments. Researchers from Binghamton University’s Thomas J. Watson College of Engineering and Applied Science presented two papers addressing these vulnerabilities at the 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
The team developed an attack method called MuTRAP, which uses hidden triggers embedded in compromised language models to manipulate robot task planning. The method, known as a backdoor or Trojan attack, allows a robot to operate normally until a specific trigger appears in its instructions. To demonstrate the vulnerability, researchers used a Segway-based mobile robot programmed to simulate cutting a plastic apple. Under normal conditions, the robot followed instructions to move the apple to a cutting board and cut it with a knife. When the researchers added the trigger word “CF” to the instruction, the compromised language model generated a different task plan, directing the robot to cut a model hand instead of the apple.
The experiment demonstrated how a manipulated AI planning system could redirect a robot toward an unintended physical target without an obvious change in the user’s underlying request. Associate Professor Shiqi Zhang, who participated in the research, said the findings raised concerns about relying on language models to control physical systems. “Large language models make decisions, and when you apply them to the physical world, they can bring real hazards and risks,” Zhang said.
Assistant Professor Adnan Siraj Rakin, another member of the research team, said attackers would not necessarily need direct access to a deployed language model to introduce malicious behavior. Knowledge of an application’s model architecture could be sufficient to develop an attack. The researchers identified the use of third-party AI models as a potential source of security exposure for robotics companies that lack the resources to train their own models. “Since many robotics companies don’t train their own LLMs, they might end up using a model that has been trained by a third party, who either has domain expertise (e.g. kitchen or warehouse) or resources (GPUs) to train the model, making the attack scenario practical and concerning,” Rakin said.
To address these vulnerabilities, the researchers developed PROTEA, an LLM-based security mechanism that evaluates robot task plans and identifies potentially unsafe actions before execution. Unlike systems that assess individual instructions in isolation, PROTEA considers the sequence of planned actions and changes in the surrounding environment. This enables it to identify situations in which an action that appears harmless could produce an unsafe outcome because of earlier steps.
For example, a robot might place medication inside a cardboard box and subsequently receive an instruction to dispose of the box. Although discarding an empty cardboard box would ordinarily be acceptable, PROTEA is designed to recognize that the box contains medication and prevent the disposal action. The researchers reported that PROTEA was able to distinguish between harmful and benign task plans with reasonable accuracy, although the information provided did not specify numerical detection rates or performance benchmarks.
The work involved Binghamton University faculty members Zhang and Rakin, along with doctoral students Zainab Altaweel, Mohaiminul Al Nahian, David Reitano and Sabbir Ahmed. The findings were presented in two research papers titled “MuTRAP: Multi-trigger Trojans Attacking Robot Task Planning Systems” and “PROTEA: Securing Robot Task Planning and Execution.”
The researchers plan to extend their investigations to vulnerabilities involving robot controllers and perception systems, including conditions that could cause legged robots to fall. The work focuses on identifying and preventing attacks that could affect robots as language models become integrated into systems responsible for executing tasks in physical environments.
Photo credit: Binghamton University, State University of New York
