Skip to content

Gradient Surgery for Physics-Informed Neural Networks

Sep 2026 · arXiv.org · Vol abs/2609.30966 · 0 citations · 39 references
Computer Science

TL;DR

PAM-GS is proposed, a physics-aware gradient surgery method that adaptively mitigates task interference during training according to the observed conflict types, and demonstrates that PAM-GS combines competitive solution accuracy with consistently strong task-balanced performance, outperforming existing methods on most problems.

Abstract

Physics-Informed Neural Networks (PINNs) are trained by optimising a composite objective that combines data fitting with physics-based constraints, typically resulting in a highly imbalanced multi-task optimisation problem. Under these conditions, existing optimisation strategies are affected by conflicting task gradients, leading to slow convergence and unstable training, particularly for stiff and high-frequency partial differential equations. We analyse gradient conflicts throughout training of PINNs with standard optimiser and investigate Multi-Task Deep Learning (MTDL) optimisation methods. In our analysis across four benchmark problems we observed that PINN optimisation exhibits three distinct phases in which angle- and magnitude-based gradient conflicts alternate, with only one present at a time. Building on these observations, we propose PAM-GS, a physics-aware gradient surgery method that adaptively mitigates task interference during training according to the observed conflict types. Experiments on four representative PDE benchmarks demonstrate that PAM-GS combines competitive solution accuracy with consistently strong task-balanced performance, outperforming existing methods on most problems.

View source

Similar papers

Review Open access Sep 2026

Physics-informed neural networks and neural operators for PDEs: A review of architectures, training pathologies, and method selection.

Across the surveyed evidence, PDE complexity emerges as a useful organizing lens: classical PINNs remain attractive for sparse-data and inverse settings, whereas operator-level and hybrid models are often better suited to parametric, multiscale, or repeated-query regimes.

Namhyeon Kim, Mukhiddin Toshpulatov, Suan Lee · 0 citations
Open access Oct 2026

Physics-informed hybrid neural networks with data constraints: A case study on lorenz63.

Physics-Informed Neural Networks (PINNs) are a hybrid modeling approach that integrates physical priors into neural networks. By encoding governing equations as soft constraint terms into the loss function, this method accomplishes organic integration of data-driven approaches with physical laws. PINNs demonstrate sign...

Yao Xiao, You-Min Tang, Yi Li et al. · 0 citations

KATOsuper: Surrogate-accelerated neural topology optimization with sensitivity-consistent Fourier neural operators

KATOsuper, an objective-agnostic framework that couples neural-reparameterized topology optimization with a Sensitivity-Consistent Fourier Neural Operator (SC-FNO), demonstrates 15--110x deployment-time speedup over MATLAB baselines while maintaining competitive optimality, with the clearest gains observed in complex 3...

Sheng-Yu Yan, Jasmin Jelovica · 2 citations
#artificial intelligence Preprint Sep 2026

PINNMorph: Evolving Online Adaptation Policies for Physics-Informed Neural Networks

Physics-informed neural networks (PINNs) provide a learning-based framework for solving partial differential equations (PDEs), yet their training behavior can change substantially throughout optimization. Residual distributions, gradient interactions, regional learning difficulty, and model-capacity requirements may ev...

Xu Yang, Ming-Yang Yu, Jun Zhang et al. · 0 citations
#machine learning Preprint Sep 2026

Tackling Failure Modes of PINNs and PIKANs Using Conflict-Free Gradients

Norm-PCGrad is proposed, a normalized variant that achieves state-of-the-art accuracy across a range of 2D and 3D domain decomposition problems and demonstrates that gradient surgery extends to physics-informed Kolmogorov-Arnold Networks, yielding substantial accuracy improvements for 3D domain decomposition.

S. S. Menon, Irina Tezaur, Ameya D. Jagtap · 2 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.