2024-11-16 –, LT8
Language: English
In an era where data security and control are paramount, leveraging local and in-house AI solutions has become increasingly significant. This presentation will explore how to use Open WebUI to build on-device GPT models or in-house server-based GPT systems, offering robust alternatives to cloud-based AI solutions like Copilot, ChatGPT-4o. The focus will be on ensuring data remains local or within company control, addressing key security considerations.
The talk will cover the following aspects:
1. Introduction to Open WebUI:
◦ Overview of Open WebUI, an extensible and feature-rich self-hosted WebUI designed to operate offline.
◦ Discussion on its capabilities to integrate various LLM runners, including Ollama and OpenAI-compatible APIs.
2. Setting Up Open WebUI:
◦ Step-by-step guide on installing and configuring Open WebUI using Docker or Kubernetes for seamless deployment.
◦ Instructions on integrating GPU support for enhanced performance.
3. Multi-Model Integration:
◦ Demonstration of how to leverage multiple models within Open WebUI, allowing for versatile and powerful interactions.
◦ Examples of using models such as LLaVA, Llama3, Phi-3 Mini, and more for diverse applications.
4. Enhancing Functionality with Plugins:
◦ Introduction to the Pipelines Plugin Framework to incorporate custom logic and Python libraries.
◦ Examples of plugins for web search, document search, Discord integration, and more.
5. Data Security and Control:
◦ Discussion on the importance of keeping data local or within company infrastructure.
◦ Best practices for ensuring data security and compliance while using in-house AI solutions.
6. Building a Powerful Interface:
◦ Tips on extending Open WebUI to create a user interface similar to ChatGPT-4o.
◦ Leveraging features such as Markdown and LaTeX support, hands-free voice/video call, and retrieval-augmented generation (RAG) for a dynamic user experience.
Dr. Chung is a SVP at Group CTO Office of the HKT/PCCW Group, where he’s responsible for leading the group’s product and technology roadmap and strategic development. He also represents the group as board members of Lynx Analytics and Bindo Labs.
Before HKT/PCCW, Chung contributed to the Big Data/AI strategy at Telstra as well as its international growth strategy. Prior to Telstra, Chung was an Associate Partner of Cluster Technology Limited which serves the Greater China market with professional services and solutions in high-performance computing, machine learning, big data, and public cloud.
In 2008, Chung joined McKinsey & Company in the Hong Kong office. He received his DPhil in Information Engineering from the University of Oxford and held the Croucher Foundation Scholarship to work toward his research degree in wireless ad-hoc networks. Chung also received BEng and MPhil in Information Engineering from the Chinese University of Hong Kong.