NestyNet. I. Physics Functions Are Hard to Fit with Neural Networks: A Framework for Accurate Surrogates and Analytic Derivatives

[2608.05862] NestyNet. I. Physics Functions Are Hard to Fit with Neural Networks: A Framework for Accurate Surrogates and Analytic Derivatives Skip to main content Search Submit Donate Log in Astrophysics > Instrumentation and Methods for Astrophysics arXiv:2608.05862 (astro-ph) [Submitted on 6 Aug 2026 ( v1 ), last revised 25 Aug 2026 (this version, v2)] Title: NestyNet. I. Physics Functions Are Hard to Fit with Neural Networks: A Framework for Accurate Surrogates and Analytic Derivatives Authors: Rodrigo Ibata , Wassim Tenachi , Foivos Diakogiannis , Neil Ibata , Anirudh Shankar View PDF HTML (experimental) Abstract: Many of the smooth functions that matter most in physics are precisely the ones that standard neural network methods struggle to fit accurately. Here we present NestyNet, a coupled model-and-optimizer framework capable of fitting such targets to high accuracy while also delivering their gradients, Hessians, Laplacians, and antiderivatives analytically and at low cost. This makes it a natural substrate for scientific machine learning tasks. The model is a deterministic segmented analytic surrogate, and its optimizer is a second-order Levenberg--Marquardt scheme whose damping and linear solves are tailored to the stiff, strongly correlated parameter geometries induced by multiscale and sharply structured targets typical in physics and other scientific applications. On the AI Feynman benchmark of 120 physics equations, NestyNet achieves median improvement factors of $2\,100\times$ for function values, $1\,400\times$ for first derivatives, and $780\times$ for second derivatives relative to standard neural networks trained with first-order optimization (Adam). Even after refining those fits with quasi-Newton (L-BFGS) optimization, the corresponding improvements are $540\times$, $450\times$, and $250\times$. Owing to the analytic design it is up to $\approx 44\times$ faster than vectorized automatic-differentiation (autograd) baselines, with a margin growing with model size. The same analytic-derivative framework also supports vector- and complex-valued targets, measurement uncertainties in both inputs and outputs, and constraints, and its modules can be composed flexibly to build scientifically useful model architectures, all without reverting to autograd. Together, these components provide a practical modular framework for fitting difficult scientific surrogates while delivering accurate differential operators for subsequent analysis. Comments: 25 pages, 6 figures Subjects: Instrumentation and Methods for Astrophysics (astro-ph.IM) ; Data Analysis, Statistics and Probability (physics.data-an) Cite as: arXiv:2608.05862 [astro-ph.IM] (or arXiv:2608.05862v2 [astro-ph.IM] for this version) https://doi.org/10.48550/arXiv.2608.05862 Focus to learn more arXiv-issued DOI via DataCite Submission history From: Rodrigo A. Ibata [ view email ] [v1] Thu, 6 Aug 2026 10:41:20 UTC (571 KB) [v2] Tue, 25 Aug 2026 08:30:19 UTC (571 KB) Full-text links: Access Paper: View PDF HTML (experimental) TeX Source view license Current browse context: astro-ph.IM < prev | next > new | recent | 2026-08 Change to browse by: astro-ph physics physics.data-an References & Citations INSPIRE HEP SciX Google Scholar Semantic Scholar export BibTeX citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer ( What is the Explorer? ) Connected Papers Toggle Connected Papers ( What is Connected Papers? ) Litmaps Toggle Litmaps ( What is Litmaps? ) scite.ai Toggle scite Smart Citations ( What are Smart Citations? ) Code, Data, Media Code, Data and Media Associated with this Article alphaXiv Toggle alphaXiv ( What is alphaXiv? ) Links to Code Toggle CatalyzeX Code Finder for Papers ( What is CatalyzeX? ) DagsHub Toggle DagsHub ( What is DagsHub? ) GotitPub Toggle Gotit.pub ( What is GotitPub? ) Huggingface Toggle Hugging Face ( What is Huggingface? ) ScienceCast Toggle ScienceCast ( What is ScienceCast? ) Demos Demos Replicate Toggle Replicate ( What is Replicate? ) Spaces Toggle Hugging Face Spaces ( What is Spaces? ) Spaces Toggle TXYZ.AI ( What is TXYZ.AI? ) Related Papers Recommenders and Search Tools Link to Influence Flower Influence Flower ( What are Influence Flowers? ) Core recommender toggle CORE Recommender ( What is CORE? ) IArxiv recommender toggle IArxiv Recommender ( What is IArxiv? ) Author Venue Institution Topic About arXivLabs arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs . Which authors of this paper are endorsers? | Disable MathJax ( What is MathJax? ) We gratefully acknowledge support from our major funders , member institutions , and all contributors. About · Help · Contact · Subscribe · Copyright · Privacy · Accessibility · Operational Status (opens in new tab)

Related documents

• Physics-Informed Non-Dimensionalisation for Neural Solvers#684012
• NestyNet. I. Physics Functions Are Hard to Fit with Neural Networks: A Framework for Accurate Surrogates and Analytic Derivatives#670726
• Mosaic: A Benchmark Suite for Differentiable Physics Solvers#319696
• Deep Learning Methods of Mathematical Physics - Volume I: Direct and Inverse Problems#489673
• Combinatorial optimization with physics-inspired graph neural networks#19723