Calculus How To

Automatic Differentiation: Definition

Share on

Calculus Definitions >

Automatic differentiation (autodiff) uses a computer to calculate derivatives at some specified value, using a mechanical application of the chain rule.

It doesn’t give you a formula, but rather the value of the derivative at the point of interest.

Applications for Automatic Differentiation

Autodiff is used in:

  • Data assimilation,
  • Design optimization,
  • Inverse problems,
  • Numerical methods,
  • Sensitivity analysis.

It can deal with very complex functions; One of the largest applications was a 1.6 million line code (written in Fortran 77) for research in fluid dynamics.


Advantages include:

  • It’s efficient and stable,
  • Answers are fairly precise,
  • It’s generally a good choice if you need to calculate the derivative at a point.
  • It’s generally considered a better choice than other computer-based differentiation like finite differencing, symbolic differentiation, or hand coding (Gebremedhin, 2014).

Types of Automatic Differentiation

There are two main types of autodiff:

  • Forward mode autodiff,
  • Reverse mode autodiff.

Forward mode (also known as director tangent linear mode), is usually used for calculating directional derivatives. It involves finding derivatives of intermediate variables with respect to independent variables. Propagating from one statement to the other is done via the chain rule.

Reverse mode (also known as backward, adjoint or cotangent linear mode) computes the derivatives of dependent variables with respect to intermediate values. Again, propagating from one statement to the next is done according to the chain rule.

These two modes are mathematically equivalent and based on the same principles, but they may require different amounts of time and computer memory.


Bischof, Bucker, Rasch, Slusanschi, and Lang. Automatic Differentiation of the General-Purpose Computational Fluid Dynamics Package. Journal of Fluids Engineering, 2007, volume 129 no 5. pages 652—658. Abstract retrieved from on March 31, 2019Gebremedhin, A. (2014). Research, Automatic Differentiation. Retrieved June 1, 2019 from:
Bücker, Schiller, Hovland et al. Autodiff. Retrieved from on March 31, 2019.
Grosse, Roger. CSC32 Lecture 10: Automatic Differentiation. The University of Toronto CS. Retrieved from on March 30, 2019.
Kourounis, D. et al. (2017). Compile-Time Symbolic Differentiation Using C++Expression Templates. Retrieved June 1, 2019 from:
Wang, Chi-Feng. Automatic Differentiation, Explained. How Do Computers Calculate Derivatives? Retrieved from on March 31, 2019.

Stephanie Glen. "Automatic Differentiation: Definition" From Calculus for the rest of us!

Need help with a homework or test question? With Chegg Study, you can get step-by-step solutions to your questions from an expert in the field. Your first 30 minutes with a Chegg tutor is free!

Leave a Reply

Your email address will not be published. Required fields are marked *