The Method of Distributions for Random Ordinary Differential Equations

Abstract

Random ordinary differential equations (RODEs) describe numerous physical and biological systems whose dynamics contain some level of inherent randomness. These sources of uncertainty enter into dynamics in two forms: (a) externally imposed or internally generated random excitations, i.e., noise, and/or (b) probabilistic representations of uncertain coefficients and initial/boundary data. Such systems admit a distribution of solutions, which is (partially) characterized by the single-time joint probability density function (PDF) of system states. If the random excitations correspond to Gaussian white noise, it is relatively straightforward to derive a closed-form deterministic partial differential equation (PDE) known as the Fokker-Planck (or Kolmogorov’s forward) equation, which governs the evolution of the joint PDF. However, most plausible noise sources are correlated (colored). In this case, the resulting PDF equations require a closure approximation. Via the method of distributions, we propose two methods for closing such equations: (a) modified large-eddy-diffusivity closures, and (b) a data-driven closure relying on sparse regression to learn relevant features. In the realms of nonequilibrium statistical mechanics and computational neuroscience, the closures are tested in a head-to-head comparison against Monte Carlo simulations for colored-noise sources such as Ornstein-Uhlenbeck and sine-Wiener processes. Additionally, the approaches’ algorithmic complexities are thoroughly discussed.

Implementing the method of distributions for high-dimensional systems of RODEs is challenging due to the computational burden of solving the high-dimensional PDE associated with the joint PDF of states. Although recent advancements in numerical integration techniques for high-dimensional PDEs have been made, they are often tailored to specific applications and lack generality for large numbers of states/dimensions. However, for many applications, only a low-dimensional quantity of interest (QoI) from the underlying high-dimensional system is desired. In these cases, it is sufficient to study a reduced-order PDF (RO-PDF) equation, i.e., a low-dimensional PDE for the QoI’s PDF, allowing classical integration techniques to be employed. Moreover, unclosed coefficients in the RO-PDF equations can be rewritten as conditional expectations, which we directly estimate from data via non-parametric regression. When the RODE exhibits strong nonlinearities and/or stiffness, it is usually necessary to supplement the learned reduced-order PDE with a data assimilation method to account for model misspecification that may occur from regression discrepancies. We propose nudging (a.k.a., Newtonian relaxation) and deep neural networks for this task, which are successfully tested for uncertainty quantification of stochastically forced oscillators and transmission failures in electrical power grids.