Kubernetes Audit Log Gotchas
Kubernetes (K8s) audit log is a primary detection source in both managed and unmanaged K8s clusters. It provides visibility into API server activity , such as anonymous access to cluster or creation of privileged pod. Equipped with this knowledge and your favorite log analysis tool, you optimistically set out on a journey to implement a consolidated detection infra across your clusters / cloud environments, or perhaps to create a forensics playbook. Soon enough you realize that the task is not that simple: your EKS cluster logging is off, your AKS log is missing crucial events, and the event formats in GKE and OKE are completely different from vanilla K8s audit log format.
In this talk I will present practical challenges around streaming and managing K8s audit logs across multiple CSPs and unmanaged clusters. We'll talk about default logging policy and the lack of transparency around it, about unexpected format differences across the cloud vendors, touch on performance and latency and more. Finally, we will analyze the impact of the described gotchas on potential detection and forensic activities: what attacker techniques you might miss, which rules won't trigger, and what other sources to consider for augmentation.
