2024-07-11 –, While Loop (4.2)
One of Julia's greatest strengths is processing gigabytes of tabular data in seconds. However, getting code ready for production and performant for web app users is another challenge. With Julia experience since 2015, this talk presents GLCS.io’s best practices in using Julia as a Service for large in-memory computation.
Perhaps one of the longest-running Julia applications used in production (since 2015) is for medical financial modeling. This data has many columns (100+), and one model can be upwards of 10GB+ on disk. Now consider giving users access to multiple models from a GUI that can ask database-like questions such as filter, groupby, transform, and join.
Is this all possible to accomplish in a small number of seconds? The answer: generally, yes. After an initial start using JuliaDB.jl, GLCS.io has moved away from this defunct package and utilized DataFrames.jl with a few more tricks. As a result, this database-like application has increased in stability and performance for end-users.
This talk generically diagrams the architecture of this back-end service. We follow by covering the Julia packages utilized to make this service performant and available, ultimately resulting in a queryable endpoint that returns a database-like result set. The most important tips for writing data-heavy julia applications are discussed. Finally, we relect on where Julia can be used elsewhere and answer further questions.
This Julia success story aims to encourage industry professionals (Managers and Developers alike!) to use Julia for their existing and greenfield projects. The hope is that existing Julia users will share this story with non-Julians they know and utilize Julia for themselves and their industry!
Slides: here
Other useful links:
- https://glcs.io
- https://blog.glcs.io
- https://github.com/GLCS
Michael Bologna is a Principal Consultant at Great Lakes Consulting Services (GLCS.io), working in Modeling & Simulation, large data computation, web application development, and tech consulting. Find him on LinkedIn, GitHub or email mbologna@glcs.io