Everything about Automatic differentiation
Numerical calculations carrying along derivatives
In mathematics and computer algebra, automatic differentiation, also called algorithmic differentiation, computational differentiation, and differentiation arithmetic is a set of techniques to evaluate the partial derivative of a function specified by a computer program. Automatic differentiation enables the simultaneous computation of the numerical values of arbitrarily complex functions and their derivatives using only the function itself; no symbolic derivative is required. Auto-differentiation is thus neither numeric nor symbolic, nor is it a combination of both. In contrast to the more traditional numerical methods based on finite differences, auto-differentiation is 'in theory' exact, and in comparison to symbolic algorithms, it is computationally inexpensive. Wikipedia →
🧭 Start here: a 6-step reading path
- Read the overview: Automatic differentiationA 5-minute orientation
- Skim a recent paper: Modules for Experiments in Stellar Astrophysics (MESA): Time-dependent Convection, Energy Conservation, Automatic Differentiation, and InfrastructureInfluential work since 2022
- Pick up a classic book: Continuous time dynamical systemsThe most-reprinted book on the subject
- Read the landmark paper: The Tapenade automatic differentiation toolCited 439 times
- Go sideways: Function of several real variablesA closely related topic
- Go sideways: Differential of a functionA closely related topic
📖 Definitions
Noun Exact differentiation of functions represented in program code by inspecting the code (as opposed to mere evaluation) and repeated application of the chain rule.
🏆 Landmark papers (most cited)
The Tapenade automatic differentiation tool
Automatic differentiation of algorithms
Automatic differentiation in Odyssée
Automatic Differentiation in Robust Optimization
Automatic differentiation of multichannel EEG signals
Automatic Differentiation of Computer Programs
Acoustic hologram optimisation using automatic differentiation
A mathematical view of automatic differentiation
🆕 Influential since 2022
Modules for Experiments in Stellar Astrophysics (MESA): Time-dependent Convection, Energy Conservation, Automatic Differentiation, and Infrastructure
CAN-PINN: A fast physics-informed neural network based on coupled-automatic–numerical differentiation method
Optimal Control of Nonequilibrium Systems through Automatic Differentiation
Quantum Optimal Control via Semi-Automatic Differentiation
Reconstructing spectral functions via automatic differentiation
📚 Books worth reading
Continuous time dynamical systemsB. M. Mohan · 2012
Cellular automata and groupsTullio Ceccherini-Silberstein · 2010
Representation and control of infinite dimensional systemsAlain Bensoussan, Giuseppe Da Prato · 1992- 📕Performance Analysis and Synthesis for Discrete-Time Stochastic Systems with Network-Enhanced ComplexitiesDerui Ding, Zidong Wang · 2018
Stabilization, optimal and robust controlAziz Belmiloudi · 2008
Computation and control IIIBozeman Conference on Computation and Control (3rd 1992 Montana State University), Kenneth L. Bowers · 1993
Modeling, analysis, and control of dynamic systemsWilliam J. Palm · 1983
Partial stability and controlVorotnikov, V. I. · 1997
Ranked by number of editions on Open Library, a rough measure of how lasting a book is.
🔗 Related topics
- Function of several real variables In mathematics, a function of several real variables or real multivariate function is a function with more than one argument, with all arguments being…
- Differential of a function In calculus, the differential represents the principal part of the change in a function y=f(x) with respect to changes in the independent variable. The…
- Derivative (multivariable calculus) In mathematics, the derivative of a function at a point is the linear part of the best affine approximation to the function near the point. In one-variable…
- Chain rule In calculus, the chain rule is a formula that expresses the derivative of the composition of two differentiable functions z and y in terms of the derivatives…
- Partial derivative In mathematics, a partial derivative of a function of several variables is its derivative with respect to one of those variables, with the others held…
- Gradient In vector calculus, the gradient ∇f of a scalar-valued differentiable function f of several variables is the vector field (or vector-valued function) whose…
- Differential (mathematics) In mathematics, differential refers to several related notions derived from the early days of calculus, put on a rigorous footing, such as infinitesimal…
- Derivative In mathematics, the derivative is a fundamental tool that quantifies the sensitivity to change of a function's output with respect to its input. The derivative…