Security BSides Las Vegas 2025

Yoshiki Kitamura

Yoshiki Kitamura is a security engineer at Cybozu, Inc., where he focuses on web security and designing optimal security frameworks for the organization. He is also a member of the internal PSIRT (Product Security Incident Response Team), conducting vulnerability testings and handling security issues to ensure the safety and reliability of Cybozu’s services.


Session

08-04
15:00
25min
Prompt Hardener - Automatically Evaluating and Securing LLM System Prompts
Krity Kharbanda, Junki Yuasa, Yoshiki Kitamura

Prompt injection remains one of the most critical and under-addressed vulnerabilities in LLM applications. Despite its growing impact, most developers still rely on ad hoc, manual methods to evaluate and secure system prompts, often missing subtle weaknesses that attackers can exploit. Prompt Hardener is an open source toolkit that automates the evaluation, hardening, and adversarial testing of system prompts using the LLM itself. It applies modern prompt hardening techniques such as spotlighting, random sequence enclosure, instruction defense, and role consistency to improve prompt resilience. The tool also performs injection testing with categorized payloads that simulate real world threats, including system prompt leaking and improper output handling based on OWASP Top 10 for LLM Applications 2025. It is mainly intended for use by LLM application developers and security engineers at business companies for evaluating, improving, and testing system prompts for their LLM applications. In this talk, we will also give a live demo of how to strengthen system prompts using the Prompt Hardener CLI mode and Web UI. Join us to learn how to strengthen your system prompts.

Proving Ground
Firenze