Calculus How To

Automatic Differentiation: Definition

Share on

Calculus Definitions >

Automatic differentiation (autodiff) uses a computer to calculate derivatives at some specified value, using a mechanical application of the chain rule.


It doesn’t give you a formula, but rather the value of the derivative at the point of interest.

Applications for Automatic Differentiation

Autodiff is used in:

  • Data assimilation,
  • Design optimization,
  • Inverse problems,
  • Numerical methods,
  • Sensitivity analysis.

It can deal with very complex functions; One of the largest applications was a 1.6 million line code (written in Fortran 77) for research in fluid dynamics.

Advantages

Advantages include:


  • It’s efficient and stable,
  • Answers are fairly precise,
  • It’s generally a good choice if you need to calculate the derivative at a point.
  • It’s generally considered a better choice than other computer-based differentiation like finite differencing, symbolic differentiation, or hand coding (Gebremedhin, 2014).

Types of Automatic Differentiation

There are two main types of autodiff:

  • Forward mode autodiff,
  • Reverse mode autodiff.

Forward mode (also known as director tangent linear mode), is usually used for calculating directional derivatives. It involves finding derivatives of intermediate variables with respect to independent variables. Propagating from one statement to the other is done via the chain rule.

Reverse mode (also known as backward, adjoint or cotangent linear mode) computes the derivatives of dependent variables with respect to intermediate values. Again, propagating from one statement to the next is done according to the chain rule.

These two modes are mathematically equivalent and based on the same principles, but they may require different amounts of time and computer memory.

References

Bischof, Bucker, Rasch, Slusanschi, and Lang. Automatic Differentiation of the General-Purpose Computational Fluid Dynamics Package. Journal of Fluids Engineering, 2007, volume 129 no 5. pages 652—658. Abstract retrieved from http://fluidsengineering.asmedigitalcollection.asme.org/article.aspx?articleid=1431286 on March 31, 2019Gebremedhin, A. (2014). Research, Automatic Differentiation. Retrieved June 1, 2019 from: https://www.cs.purdue.edu/homes/agebreme/research/coloring-derivatives.html
Bücker, Schiller, Hovland et al. Autodiff. Retrieved from http://www.autodiff.org/?module=Introduction on March 31, 2019.
Grosse, Roger. CSC32 Lecture 10: Automatic Differentiation. The University of Toronto CS. Retrieved from https://www.cs.toronto.edu/~rgrosse/courses/csc321_2018/slides/lec10.pdf on March 30, 2019.
Kourounis, D. et al. (2017). Compile-Time Symbolic Differentiation Using C++Expression Templates. Retrieved June 1, 2019 from: http://arxiv-export-lb.library.cornell.edu/pdf/1705.01729
Wang, Chi-Feng. Automatic Differentiation, Explained. How Do Computers Calculate Derivatives? Retrieved from https://towardsdatascience.com/automatic-differentiation-explained-b4ba8e60c2ad on March 31, 2019.

CITE THIS AS:
Stephanie Glen. "Automatic Differentiation: Definition" From CalculusHowTo.com: Calculus for the rest of us! https://www.calculushowto.com/automatic-differentiation/
------------------------------------------------------------------------------

Need help with a homework or test question? With Chegg Study, you can get step-by-step solutions to your questions from an expert in the field. Your first 30 minutes with a Chegg tutor is free!


Leave a Reply

Your email address will not be published. Required fields are marked *