Research

Overview

We develop machine learning methods motivated by the distinct challenges and opportunities of the natural sciences. A common assumption is that machine learning for science means hand-engineering physics into models. We are more interested in understanding what models learn from scientific data, and in developing new methods, spanning model design, training, inference-time sampling, and post-training adaptation, that turn that understanding into reliable predictions. Our work spans multi-scale dynamics, from quantum and atomistic systems to continuum fluids, and connects to numerical analysis, dynamical systems, statistical mechanics, quantum mechanics, and optimization.

A full list of publications is available on the publications page or on Google scholar, and some example research directions include:

Understanding what models learn, and the role of physical structure

Left: attention score versus interatomic distance, decaying like 1/r^2 in early layers and 1/r in later layers. Right: fine-tuning IsoFLOP curves showing predictable neural scaling with model size and compute.
Left: trained without physical or graph priors, a pure Transformer's attention recovers physically meaningful structure — decaying with interatomic distance like 1/r² (electrostatic forces) in early layers and 1/r (electrostatic energies) in later layers. Right: such models also follow clean neural scaling laws, with performance improving predictably with model size, dataset size, and compute. Figure from Transformers Discover Molecular Structure without Graph Priors.

A central question in our work is systematically understanding what neural networks can actually learn from scientific data. Much of machine learning for the physical sciences has focused on encoding physical structure, e.g., group equivariance, graph connectivity, conservation laws, directly into model architectures. We study when these hand-crafted priors genuinely help, and when they instead limit what a model can express or how efficiently it scales. We have found, for instance, that general-purpose architectures can recover physical structure directly from data: Transformers can learn molecular geometry and interatomic interactions without built-in physical or graph priors, and relaxing built-in constraints can improve both speed and accuracy across different physical modeling domains. Rather than treating physical structure as something that must be imposed by default, we try to characterize what models discover on their own, and use that understanding to guide how we design and scale them. These questions build on our earlier work developing efficient equivariant operations for geometric deep learning; having built such methods, we are interested in understanding when this kind of structure is essential and when models can instead learn it from data.

Example publications

Generative modeling for physical systems

The learned score function of a pre-trained generative model enters an Onsager–Machlup action whose drift and divergence terms depend on the score; minimizing this action over paths at inference time samples transition paths between metastable states A and B.
At inference time, transition paths between metastable states are sampled by minimizing the Onsager–Machlup action functional evaluated with the score function of a pre-trained generative model. Figure from Action-Minimization Meets Generative Modeling.

Many of the hardest problems in the physical sciences are problems of sampling and inference: characterizing rare events, exploring conformational ensembles, or connecting a model to experimental measurements. We develop generative modeling methods for such tasks, new ways to use and adapt generative models once they are trained. For sampling, we have shown how a pre-trained generative model can be repurposed at inference time to sample molecular transition paths efficiently, by casting transition path sampling as action minimization through the Onsager–Machlup functional. For connecting simulation to experiment, we develop post-training alignment methods that fine-tune generative models so their predicted distributions match experimental observables, rather than just reproducing other simulations. This kind of distribution matching has close ties to maximum-entropy inverse reinforcement learning and to reinforcement-learning-based fine-tuning of generative models, where a model is likewise adjusted to be consistent with observations while staying close to a reference. We are also broadly interested in the connections between generative models and non-equilibrium statistical mechanics: the same mathematical structures that describe stochastic dynamics in physical systems also underlie how generative models are trained and sampled.

Example publications

Building reliable models: evaluation, stability, and deployment

Distilling a large foundation model into fast, specialized machine-learned force fields via energy-Hessian distillation, for downstream molecular dynamics, geometry optimization, and free-energy calculations.
Distillation of a large-scale model into fast, specialized ML force fields. Figure from Distilling Foundation Models via Energy Hessians.

Better accuracy on a benchmark does not always translate into reliable predictions in practice. We develop systematic principles for building, evaluating, and deploying models of physical systems so that improvements are meaningful and trustworthy. This includes metrics for the smoothness of learned potential energy surfaces that connect how a model is evaluated to how it should be designed; open benchmarks for evaluating models; and analyses of how large-scale ML models behave, and fail, under distribution shift. We also study how to make models practical for downstream use, for example by distilling large, general-purpose force fields into fast, specialized models that retain the right physics for a given task. Beyond the models themselves, we contribute to large open datasets that make rigorous evaluation possible.

Example publications

Modeling and learning across scales

Animated vorticity rendering of three-dimensional isotropic turbulence simulated by EddyFormer, matching direct numerical simulation accuracy at a fraction of the cost.
For a three-dimensional isotropic turbulence example, EddyFormer matches the accuracy of a direct numerical simulation while running roughly 30× faster. Visualization from EddyFormer.

Alongside atomistic modeling, we develop machine-learned surrogate models that emulate expensive physical simulations, including for continuum systems governed by partial differential equations—for example, simulating three-dimensional turbulence at scale, or learning solution operators directly in the spectral domain. While these surrogates can reach the accuracy of classical numerical methods at a fraction of the cost, we are equally interested in what they reveal about the structure a model can learn from data: what must a model represent to capture turbulent dynamics, and how does that structure emerge from training? A theme that cuts across our work is that methods and insights transfer between scales, i.e., similar approaches that recover molecular structure at the atomistic scale also prove effective for learning physical structure at the continuum scale, helping us build models, and an understanding of what they learn, that are not tied to a single system or length scale. We are also broadly interested in models that generalize across physical systems rather than being retrained for each one.

Example publications

Funding Our research has been generously supported by the U.S. Department of Energy, the National Science Foundation, the Office of Naval Research, the Research Corporation for Science Advancement, Toyota Research Institute, Meta AI, and NVIDIA.