PyCon Hong Kong 2025

So, you want to build an AI-powered application in Python. In addition to the UI and database you would need in any application, you will of course need LLM deployments to call and prompts engineered to generate the desired output. Yet, with all of these elements in place, an AI application presents distinctive challenges for coordinating the interaction of LLMs and user data. A toy example that fills a few user-inputted variables into a prompt for use with a single LLM deployment endpoint requires only a few dozen lines of backend code. But what if you want to be able to swap between different models, deployments, or APIs? And how will you handle problems such as hitting quota limits or encountering irregularities in LLM output, or test how you process inputs and outputs? Or what if you want to build a more complex AI system that enriches the model's context (RAG) or can make its own calls to various tools in response to user requests (agents): which data sources and tools will you connect to, and how? Several Python packages -- including LangChain, LlamaIndex, Haystack, and PydanticAI -- claim to offer end-to-end frameworks for building application-ready AI pipelines and agents. Through a systematic comparison of their different features and structures, alongside the option of rolling a solution oneself, this talk will consider how using these frameworks might, or might not, make it easier to answer some of the challenging questions involved in building an AI application.