Nicolai van der Smagt
I am a Principal Solutions Architect at Nebius Token Factory, where I help customers build with Nebius public inference services, and optimized private endpoints - fine tuning, optimizing model serving setups, building agents and refining LLMOps and observability.
Previously I've spent 8 years helping AWS customers build and operationalize ML systems (and later, agents) using services like SageMaker and Bedrock.
I love open source! I compiled my first (FreeBSD) kernel almost 30 years ago.
Come talk with me about model inference, agent architecture and the challenges of operationalizing AI systems.
Session
Agents usually start as a simple harness around a single closed-model API. This architecture is convenient, but it limits control over model choice, deployment region, customization, and cost. Running open-weight models locally offers more freedom, but the most capable models demand hardware that is difficult to provision economically, and operating production GPU infrastructure introduces complexity that most teams do not want to absorb.
Managed cloud inference provides a middle path, offering open-weight model access through shared model APIs or dedicated endpoints, without requiring teams to operate the underlying GPU infrastructure. But realizing its full benefits requires more than replacing one API endpoint with another.
This talk presents four practical architecture patterns for building faster and cheaper production agents on open-weight models.