Architecture-agnostic knowledge distillation for machine-learning interatomic potentials.

Abstract

Atomistic foundation models can achieve near-ab-initio accuracy and strong transferability across molecular and materials systems, but their large parameter counts lead to high inference latency and memory cost, limiting long-time-scale molecular dynamics and resource-constrained deployment. In many practical studies, only a narrow chemical subdomain is required, motivating compact students that preserve the teacher's physical priors. Existing distillation for Machine-Learning Interatomic Potentials (MLIPs) is still limited by architecture mismatch or incomplete geometric supervision. We propose an architecture-agnostic framework, Joint Atomic Energy-Energy Hessian Distillation, that requires no architecture-specific feature projection. The method combines atomic-energy alignment, which transfers local decomposition priors, with Hessian alignment, which transfers local curvature of the potential energy surface. These two signals are complementary and jointly regularize both decomposition consistency and geometric stability. Under matched training settings, our method consistently improves energy accuracy over a strong Hessian-distillation baseline (average gain: 48.3% on SPICE and 6.7% on MPtrj) while preserving force accuracy. Across teacher-student pairs, it also reduces parameter counts by 79%-98%, enabling fast and deployment-friendly specialized MLIPs for targeted chemical subdomains.