Royce Lu

Royce Lu is a security researcher at Palo Alto Networks. He has published research at top international security conferences, including BlackHat and Virus Bulletin. Currently, his interest is in LLM safety, covering areas such as LLM agent security, jailbreak automation, and handling LLM I/O security. Before GenAI security, Royce conducted research in network security. At the start of his career, he focused on malware and computer security.


Session

07-01
15:10
20min
Breaking AI Agents: Exploiting Managed Prompt Templates to Take Over Amazon Bedrock Agents
Jay Chen, Royce Lu

AI agents are rapidly transforming industries through autonomous planning, decision-making, and interaction with external environments. As cloud providers accelerate the deployment of services that simplify building these AI-driven applications, the security implications of this emerging technology remain largely unexplored.
This talk reveals concerning security issues discovered within AWS Bedrock Agents—demonstrating how attackers can exploit prompt injection and misuse integrated tools to compromise these agents. Specifically, our research uncovers techniques that lead to information leakage, agent hijacking, unauthorized tool execution, and manipulation of persistent agent memory. The issues originate from AI models' inherent probabilistic nature combined with inadequately secured prompt instructions, which attackers exploit to subvert internal planning and decision-making processes.
Although our research primarily examines AWS Bedrock Agents, the issues and attack techniques discussed extend broadly across similar agent frameworks. We will share our methodology, key findings, mitigation strategies, and highlight important open research questions. Our goal is to foster proactive dialogue among cloud security researchers, practitioners, and AI developers to address these emerging security challenges collaboratively.

Mapping the frontier: supporting new clouds and technology
Room 2