2019-07-23 –, Elm B
Modeling practice seems to be partitioned into scientific models defined by mechanistic differential equations and machine learning models defined by parameterizations of neural networks. While the ability for interpretable mechanistic models to extrapolate from little information is seemingly at odds with the big data "model-free" approach of neural networks, the next step in scientific progress is to utilize these methodologies together in order to emphasize their strengths while mitigating weaknesses. In this talk we will describe four separate ways that we are merging differential equations and deep learning through the power of the DifferentialEquations.jl and Flux.jl libraries. Data-driven hypothesis generation of model structure, automated real-time control of dynamical systems, accelerated of PDE solving, and memory-efficient deep learning workflows will all shown to be derived from this common computational structure of differential equations mixed with neural networks. The audience will leave with a new appreciation of how these two disciplines can benefit from one another, and how neural networks can be used for more than just data analysis.
Dynamical models are often interesting due to the high-level qualitative behavior that they display. Differential equation descriptions of fluids accurately predict when drone flight will go unstable, and stochastic evolution models demonstrate the behavior for how patterns may suddenly emerge from biological chemical reactions. However, utilizing these models in practice requires the ability to understand, prediction, and control these outcomes. Traditional nonlinear control methods directly tie the complexity of the simulation to the control optimization process, making it difficult to apply these methods in real-time to highly detailed but computationally expensive models.
In this talk we will show how to decouple the computation time of a model from the ability to predict and control its qualitative behavior through a mixture of differential equation and machine learning techniques. These new methods directly utilize the language-wide differentiable programming provided by Flux.jl to perform automatic differentiation on differential equation models described using DifferentialEquations.jl. We demonstrate an adaptive data generation technique and show that common classification methods from machine learning literature converge to >99% accuracy for determining qualitative model outcomes directly from the parameters of the dynamical model. Using a modification of methods from Generative Adversarial Networks (GANs), we demonstrate an inversion technique with the ability to predict dynamical parameters that meet user-chosen objectives. This method is demonstrated to be able to determine parameters which constrains predator-prey models to a specific chosen domain and predict chemical reaction rates that result in Turing patterns for reaction-diffusion partial differential equations. Code examples will be shown and explained to directly show Julia users how to do these new techniques. Together, these methods are scalable and real-time computational tools for predicting and controlling the relation between dynamical systems and their qualitative outcomes with many possible applications.
Lyndon White, Mike Innes
Chris' research and software combines AI with differential equation models of human organs to give patients accurate and personalized drug doses: reducing pain and complications for patients while reducing treatment costs for hospitals.
Chris Rackauckas is an applied mathematics instructor at the Massachusetts Institute of Technology and a senior research analyst at the University of Maryland, School of Pharmacy in the Center for Translational Medicine. Chris's recent work is focused on bringing personalized medicine to standard medical practice through the proliferation of mathematical software. His work on developing the DifferentialEquations.jl solver suite along with over a hundred other Julia packages, not only earned him the inaugural Julia Community Prize and front page features in tech community sites, it is also the foundation of the PuMaS.jl package for Pharmaceutical Modeling and Simulation, set to release in March 2019. Chris’ work with PuMaS makes it possible to predict the optimal medication dosage for individuals, reducing the costs and potential complications associated with treatments. The software is currently being tested in the administration of treatment for neonatal abstinence syndrome (NAS), an opioid withdrawal disorder in newborn babies. NAS requires medically administered morphine doses every four hours to prevent the infants from experiencing withdrawal symptoms. PuMaS is being used to predict personalized safe dosage regimens by incorporating realistic biological models (quantitative systems pharmacology) and deep learning into the traditional nonlinear mixed effects (NLME) modeling framework. This software and its methodology are also being tested in clinical trials at Johns Hopkins University for its ability to predict an individual's drug response to vancomycin and automatically prescribe optimal doses directly from a patient's health records.
Chris started this work while completing his Masters and Ph.D. at the University of California, Irvine where he was awarded the Mathematical and Computational Biology institutional fellowship, the Graduate Dean's Fellowship, the National Science Foundation's Graduate Research Fellowship, the Ford Predoctural Fellowship, the NIH T32 Predoctural Training Grant, and the Data Science Initiative Summer Fellowship. His research with his advisor, Dr. Qing Nie, focused on the methods for simulating stochastic biological models and detailing how the randomness inherent in biological organisms can be controlled using stochastic analysis. Chris bridged the gap between theory and practice by having a "wet lab bench" in Dr. Thomas Schilling’s lab, where these methodologies were tested on zebrafish. Fluorescence Light Microscopy (FLIM) measurements of retinoic acid in the zebrafish hindbrain showed that the predicted control proteins could attenuate inherent biological randomness. The result was a verified mathematical theory for controlling the randomness in biological signaling. Chris received the Kovalevsky Outstanding Ph.D. Thesis Award from the Department of Mathematics upon graduation and was showcased in an interview "Interdisciplinary Case Study: How Mathematicians and Biologists Found Order in Cellular Noise" in iScience.
As an undergraduate at Oberlin College, Chris was awarded the NSF S-STEM scholarship and the Margaret C. Etter Student Lecturer Award by the American Crystallographic Association, an award usually given for PhD dissertations, for his work on 3+1 dimensional incommensurate crystal structure identification of H-acid. This award was given for Service Crystallography for its potential impact on industrial dye manufacturing.