2025-12-08 –, Abigail Adams
In this 90 minute tutorial we'll get anyone with some basic Python and Command Line skills up and running with their own 100% laptop based set of LLMs, and explain some successful patterns for leveraging LLMs in a data analysis environment. We'll also highlight pit-falls waiting to catch you out, and encourage you that your pre-GenAI analytics skills are still relevant today and likely will be for the foreseeable future by demonstrating the limits of LLMs for data analysis tasks.
OVERVIEW
Are we in a one-size-fits-all world of LLMs, and destined to burn tokens forever via API key access to third party LLM services? There are some great reasons to keep using 3rd party LLM services, however many data practitioners want and need a degree of control, autonomy, and maintained data sovereignty which means diving into the world of self-managed LLMs, even if they are sourced from the more than 2 million available in the HuggingFace model repository.
AGENDA
1. Welcome, concept overview and target tutorial outcomes (5 minutes)
2. Local environment setup (5 minutes, with background package downloading)
3. Why would you want to run you LLMs locally? Is it even possible? How do AI Appliances fit in? (10 minutes)
4. Navigating HuggingFace and finding the LLM of your dreams (5 minutes + 10 minute exercise)
5. Using Ollama from the GUI and from the CLI (5 minutes + 10 minute exercise)
6. Using the Ollama Python library and leveraging it from code (5 minutes + 10 minute exercise)
7. Offloading LLM to an AI Appliance (5 minutes + 5 minute exercise)
8. Next steps for AI Autonomy, Data Sovereignty, and Advanced Analytics in the age of GenAI and LLMs (5 minutes)
9. Q&A (10 minutes)
FURTHER DETAILS
This tutorial will get you comfortable navigating the HuggingFace model repository, and using the Ollama ecosystem for local model management and hosting. You'll learn how together these are the GitHub and Git of the LLM world. While we may not be able to get those LLMs behaving in a repeatable, deterministic way at least you'll have the tools to choose exactly which LLMs you're using and an ability to retain data sovereignty by self-hosting LLMs as part of your overall workflow. You'll leave with an expanded set of options for leveraging LLMs that are running off your laptop, off your own cloud server, or on affordable AI Appliances such as an Nvidia DGX Spark or Nvidia Jetson Orin Nano Super.
Ian is a Computational Scientist and Software Engineer. His current role is "Partner in AI Engineering" with BCG, a global management consulting firm. He works with BCG's clients around the world to identify opportunities to combine data, technology, and analytics to create step change capabilities in their organization. What does that translate to? On a day-to-day basis it means leading crack teams of BCG engineers and data scientists in the development of AI-driven and (typically) Python-based bespoke solutions which leverage the best tools, technology, and techniques available.
Prior to BCG, Ian was a Product Manager at Anaconda, and has been in the Python community for over 20 years. Ian has a PhD from Oxford where he worked on the CERN LHCb experiment and developed the Python-based distributed computing middleware that managed 10 million queued physics jobs to schedule across a quarter million servers in a globally federated compute environment. He also spent several years at Harvard Medical School collaborating with bio physicists on novel techniques for protein structure discovery.
Ian is a member of the Python Software Foundation and the Open Source Initiative. In his free time he enjoys sailing, cycling, xc skiing, and motorbiking. On rainy days he'll pull out a board game to play with his wife & kids: Ark Nova, Ticket To Ride, and Takenoko are current favorites.