Heechang Na [Ohio Supercomputer Center]
Heechang Na, Ph.D., is the Scientific Applications Operations Manager at the Ohio Supercomputer Center (OSC). In his role, he is responsible for managing the comprehensive operations for providing optimal scientific software application environments to academic and industry clients. Leveraging his technical expertise and strategic insights, he drives the enhancement of OSC's scientific computing infrastructure.
As part of his responsibilities, he oversees the deployment of the scientific software stack on new systems and collaborates with other groups to make the software environment more accessible. This includes deploying more Open OnDemand apps, supporting classrooms and enabling new workflows in AI, bioinformatics and other domains.
Session
We share how we built LLM inference services for researchers at Ohio Supercomputer Center (OSC) using Open OnDemand. Our users need LLMs in many different ways, so we support several service models. We started with an Ollama module on the cluster, added OpenWebUI, and later created dedicated OnDemand apps that start the LLM server and give users a simple interface. We also provide a Jupyter + Ollama app that exposes an OpenAI API interface in a Jupyter kernel. We maintain shared models so users can start quickly, while still allowing local models. We have also deployed the Ollama + OpenWebUI service on our Kubernetes cluster, but scaling it for a large number of users remains a challenge. We plan to add additional LLM engines and front-ends as part of our future improvements as well.