Security BSides Las Vegas 2025

LLM Mayhem: Hands-On Red Teaming for LLM Applications
2025-08-05 , Opal

Join us in this workshop to engage in hands-on attacks to identify weaknesses in generative AI. If you’re interested in learning about getting started in red teaming generative AI systems, this is the workshop for you.


In this workshop we have set up hypothetical chatbots with varying levels of difficulty to walk attendees through various attack techniques. We'll model the attack after typical red team engagements we have been on in order to test the resiliency of a LLM powered application. The goals of this session are: (1) Provide a foundation on red teaming chatbots, (2) understand how and why the attacks work, and (3) provide guidance on how attendees can set up their own infrastructure to test and hone their skills after the conference has concluded.

Kasimir Schulz, Director of Security Research at HiddenLayer, is a leading expert in uncovering zero-day exploits and supply chain vulnerabilities in AI. His work has been featured in Forbes, BleepingComputer, and Dark Reading, and he has spoken at conferences such as FS-ISAC and Black Hat. Kasimir leads the development of advanced tools for automating vulnerability detection and implementing large-scale patches, fortifying systems against supply chain attacks. His dedication to proactive defense measures sets a new standard in cybersecurity resilience.

Travis Smith is the Vice President of ML Threat Operations at HiddenLayer where he is responsible for the services offered by the organization, including red-teaming machine learning systems and teaching adversarial machine learning courses. He has spent the last 20 years building enterprise security products and leading world class security research teams. Travis has presented his original research at information security conferences around the world including Black Hat, RSA Conference, SecTor, and DEF CON Villages.