2024-04-03 –, Room 111
Django's async capabilities and batteries-included tooling make it an ideal framework for quickly building MVPs and iterating. This talk demonstrates building a document search MVP with Django templates, ChromaDB, and hosted large language models. It then shows how to refactor and scale it using Elasticsearch, Celery/RabbitMQ workers, React, self-hosted vLLM, and auth. With Django async, you can rapidly build, constantly improve, and deploy the latest AI models in your product.
Building and Scaling an AI Startup with Async Django
In the fast-moving world of AI, startups need to build and deploy new ideas quickly. AI models operate asynchronously and require streaming responses for the best user experience. Luckily, Django now incorporates async behavior while still providing out-of-the-box tooling, making it ideal for rapidly building an AI startup.
This talk demonstrates building an MVP where we can ask a Large Language Model (LLM) about the contents of documents (Retrieval Augmented Generation, RAG) usingDjango and then refactoring, scaling, and developing it into a full product. We'll build a simple RAG MVP using ChromaDB and stream responses from a hosted large language model in real-time to a chat interface using Django templates and streaming responses.
Next, we'll scale and improve the system by:
- Replacing ChromaDB with Elasticsearch
- Replacing Django Async with Celery workers and RabbitMQ for complex processing
- Replacing Django templates with a React app
- Self-hosting a large language model with vLLM
- Adding token-based authentication
AI models evolve exponentially - this talk shows how to rapidly integrate new models, build business logic, scale, and swap improved models. With Django's async capabilities, you can build and iterate an AI product quickly.
Dr Piotr Gryko, studied experimental physics at University College London. His PhD at Imperial College London focused on using biomaterials to self assemble inorganic materials, merging the boundaries of biological systems and machines.
He now specializes in writing software, having worked with image processing, e-commerce, logistics, embedded systems and web development. Always keen to keep learning, he writes open-source code to further develop his understanding of new technologies.
Core technologies:
- Web development (Django)
- Front end web development (reactjs, redux + material ui)
- Data analysis (python, numpy, pandas, jupyter notbooks)
- Devops & sysadmin work (docker, jenkins, digital ocean, postgres replication, ansible, elastic stack)
- Embedded development (C++, arduino, ESP8266, ESP32)