fwd:cloudsec 2023

Billions Served: Processing Security Event Logs with the AWS Serverless Stack
06-13, 09:50–10:30 (US/Pacific), Salon B

Security event and audit logs are a foundational requirement for threat hunting, threat detection, and incident response, but most security teams have little to no control over their data and rely on vendors who charge thousands of dollars per day for "log management." There must be a better way!

In this talk we will discuss the challenges, best practices, and secrets for building large scale, affordable data processing systems using the AWS serverless stack, including how to choose the best streaming data storage service, techniques for real-time event enrichment on billions of logs, and optimizing for both speed and cost.

Josh Liburdi is a security engineer and tech lead at Brex who focuses on threat detection, incident response, and distributed systems. He has more than a decade of industry experience and has worked at several diverse organizations, including Splunk, Target, and CrowdStrike. He is also a published author (Bluenomicon from Splunk, Huntpedia from Sqrrl) and is active in the open source security community and has contributed to many projects, including Substation at Brex (creator / lead), Strelka at Target (creator), and the Zeek network analysis framework.