2025-04-24 –, Helium3
Demos and prototypes for generative AI (GenAI) projects can be quickly created with tools like Streamlit, offering impressive results for users within hours. However, scaling these solutions from prototypes to robust systems introduces significant challenges. As user demand grows, hacks and workarounds in tools like Streamlit lead to unreliability and debugging frustrations. This talk explores the journey of overcoming these obstacles, evolving to a stable tech stack with Qdrant, Postgres, Litellm, FastAPI, and Streamlit. Aimed at beginners in GenAI, it highlights key lessons.
Demos and prototypes for projects with generative AI can be quickly put together: an API key from the preferred model provider, some source code from the online tutorial and a few small adjustments suffice. Thanks to Streamlit and the like, even beginners can achieve impressive results that can be used by users within a few hours.
But what happens when users actually like the solution? When demos and prototypes need to be expanded and connected to other systems? What if the number of users continues to rise?
It is quite impressive how far you can bend Streamlit to achieve things it was probably never meant for. But at a certain point, you pay for the hacks and workarounds with unreliability and frustrating debugging.
The speakers repeatedly reached this point in various projects and delayed the necessary architecture discussion for too long. So the path was longer and more painful than it should have been – but in the end, thanks to the wide range of open-source (Python) projects, a flexible and stable system was created. Our current tech stack includes Qdrant, Postgress, Litellm and FastAPI – as well as OpenWebUI, and of course Streamlit.
Thanks to modularization, we now have a stable system that we can easily run locally but also deploy in an enterprise environment. Nevertheless, we have retained a great deal of flexibility.
In our talk, we report on the trials and tribulations along the way. We report on the challenges that led to decisions for various components. We disclose which problems we were able to solve and which new problems arose.
The talk is aimed primarily at those who are taking their first steps with generative AI or have already developed their first demonstrators or prototypes.
Structure:
(1) GenAI applications in Streamlit are cool
(2) The challenges on the way from prototype to productive deployment
(3) Ramming heads through walls
(4) The path to a flexible but stable stack
(5) What still plagues us
Novice
Expected audience expertise: Python:Intermediate
Thomas is an expert in tech transfer and startup development, with a career focused on fostering innovation and bridging the gap between research and industry. He has led initiatives like accelerator programs, innovation networks, and hackathons. A Generative AI enthusiast and co-founder of neunzehn innovations, Thomas helps companies leverage AI technologies. He holds a doctorate from the University of Basel and is a dedicated advisor, educator, and speaker in the startup ecosystem.
Frank is deeply passionate about technological advancements and a co-founder of neunzehn innovations, a company specializing in AI solutions. His professional background combines entrepreneurial experience—having established an innovation and strategy consultancy focused on strategy and deep tech—with several years at a major software corporation. Throughout his tenure in the software industry, he contributed to multiple product and service launches, working across various teams to bring new offerings to market.
Outside the office, he enjoys discovering new horizons in the camper van.