2024-11-15 –, Auditorium
Developing LLM-based products in Python comes with a lot of questions: Should I use an existing LLM framework? How to evaluate models and prompts? Should I use RAG? How to prevent prompt hacking?This talk will share lessons learned and best practices from putting several shopping assistant chatbots into production.
This talk details my experience for all steps of developing and productionizing several chatbots in Python:
🛠️ Pros and cons of using LLM frameworks (e.g. Langchain)
🤔 Do I even need an LLM?
🚀 Building chatbot backends with FastAPI
📊 Different LLM evaluation approaches
🔎 RAG: vector search details & vector store solutions
🛡️ Defend against prompt hacking
✅ Testing and CI/CD
📈 Monitoring solutions
🙋 Don't forget the users!
Dominik is a Senior Data Scientist at Otto Group data.works in Hamburg. He has previously worked at E.ON and IONOS and has a background in particle physics (KIT/CERN).