A Hands-On Day with Kubernetes, Platform Engineering & Open LLMs
Kubernetes has become the universal runtime — for platform teams building internal developer platforms and for AI teams running large language model inference at scale. In this full-day hands-on workshop, you'll experience both sides of modern Kubernetes through two self-contained labs built on open source and CNCF tooling, running on upstream-conformant Kubernetes.
In the morning, you'll build a production-ready internal developer platform with golden paths, GitOps governance, and platform observability. In the afternoon, you'll deploy and scale LLM (Ministral) inference workloads with GPU-aware autoscaling, distributed computing, and AI agents. Two different challenges, one shared foundation: Kubernetes and the cloud-native open source ecosystem.
This full-day workshop consists of two self-contained hands-on labs, united by Kubernetes and open source:
Morning — Platform Engineering (9:00–12:00)
Build an internal developer platform that gives developers self-service access to cloud infrastructure — without the complexity.
- Foundation — Kubernetes platform base with Karpenter for intelligent autoscaling
- Platform Services — kro and ACK for declarative infrastructure orchestration, Backstage as the developer portal
- Developer Experience — Golden paths and self-service templates
- Governance & GitOps — Policy-as-code with Kyverno, continuous delivery with Argo CD
Afternoon — Open-Weight Generative AI on Kubernetes (13:30–17:00)
Deploy and scale an open-weight large language model (Mistral) on Kubernetes using open source inference tooling and GPU-aware infrastructure.
- GPU Infrastructure — Karpenter for GPU-aware provisioning, optimized container startup
- LLM Inference — Deploy and optimize the open-weight Mistral model using vLLM, maximizing GPU utilization and throughput
- AI Observability — GPU monitoring with NVIDIA DCGM, Prometheus, and Grafana
- Scaling — Distributed inference with Ray for handling concurrent requests at scale
- AI Agents — Deploy autonomous agents with Strands on Kubernetes
Target audience: Platform Engineers, DevOps Engineers, SREs, and ML/AI Engineers
Content level: 300–400 (intermediate to advanced)
Prerequisites: Working knowledge of containers and Kubernetes concepts
Both labs run on Amazon EKS (upstream, CNCF-conformant Kubernetes — not a fork). The hands-on tooling is open source and CNCF-aligned:
Morning — Platform Engineering:
- Kubernetes, Karpenter (CNCF), kro (K8s SIG subproject), ACK, Argo CD (CNCF Graduated), Backstage (CNCF Incubating), Kyverno (CNCF Graduated), GitLab CE
Afternoon — Generative AI:
- Kubernetes, Karpenter (CNCF), vLLM (open source), Mistral (open-weight model), Ray (open source), Strands Agents (open source), NVIDIA DCGM (open source), Grafana + Prometheus (CNCF Graduated)
Kevin Nash is a Senior Solutions Architect at Amazon Web Services (AWS), based in Switzerland. With a background in distributed systems and many years experience building for the customer. He is passionate about technology, understanding how systems work and helping customers bringing their solutions into the Cloud.