PAM-GS is proposed, a physics-aware gradient surgery method that adaptively mitigates task interference during training according to the observed conflict types, and demonstrates that PAM-GS combines competitive solution accuracy with consistently strong task-balanced performance, outperforming existing methods on most problems.
Abstract
Physics-Informed Neural Networks (PINNs) are trained by optimising a composite objective that combines data fitting with physics-based constraints, typically resulting in a highly imbalanced multi-task optimisation problem. Under these conditions, existing optimisation strategies are affected by conflicting task gradients, leading to slow convergence and unstable training, particularly for stiff and high-frequency partial differential equations. We analyse gradient conflicts throughout training of PINNs with standard optimiser and investigate Multi-Task Deep Learning (MTDL) optimisation methods. In our analysis across four benchmark problems we observed that PINN optimisation exhibits three distinct phases in which angle- and magnitude-based gradient conflicts alternate, with only one present at a time. Building on these observations, we propose PAM-GS, a physics-aware gradient surgery method that adaptively mitigates task interference during training according to the observed conflict types. Experiments on four representative PDE benchmarks demonstrate that PAM-GS combines competitive solution accuracy with consistently strong task-balanced performance, outperforming existing methods on most problems.
Across the surveyed evidence, PDE complexity emerges as a useful organizing lens: classical PINNs remain attractive for sparse-data and inverse settings, whereas operator-level and hybrid models are often better suited to parametric, multiscale, or repeated-query regimes.
Namhyeon Kim, Mukhiddin Toshpulatov, Suan Lee· Neural Networks· 0 citations
Physics-Informed Neural Networks (PINNs) are a hybrid modeling approach that integrates physical priors into neural networks. By encoding governing equations as soft constraint terms into the loss function, this method accomplishes organic integration of data-driven approaches with physical laws. PINNs demonstrate sign...
Yao Xiao, You-Min Tang, Yi Li et al.· Neural Networks· 0 citations
A Physics-Informed Error Field Learning (PIEFL) framework for PINNs is proposed, which avoids continuous optimization of the entire solution space and focuses computational resources on correcting existing prediction errors.
KATOsuper, an objective-agnostic framework that couples neural-reparameterized topology optimization with a Sensitivity-Consistent Fourier Neural Operator (SC-FNO), demonstrates 15--110x deployment-time speedup over MATLAB baselines while maintaining competitive optimality, with the clearest gains observed in complex 3...
Physics-informed neural networks (PINNs) provide a learning-based framework for solving partial differential equations (PDEs), yet their training behavior can change substantially throughout optimization. Residual distributions, gradient interactions, regional learning difficulty, and model-capacity requirements may ev...
Xu Yang, Ming-Yang Yu, Jun Zhang et al.· 0 citations
Norm-PCGrad is proposed, a normalized variant that achieves state-of-the-art accuracy across a range of 2D and 3D domain decomposition problems and demonstrates that gradient surgery extends to physics-informed Kolmogorov-Arnold Networks, yielding substantial accuracy improvements for 3D domain decomposition.
S. S. Menon, Irina Tezaur, Ameya D. Jagtap· 2 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 17, 2026
By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.
Microsoft Research Blog· microsoft.comNov 16, 2026
AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduNov 13, 2026