E[g(X)] ≥ g(E[X]).
Then, for arbitrary x:
If all the xj are equal but one of the wj equal zero, then the function f(x) the ≥ sign becomes an equals sign. If this is the only time in which that inequality becomes an inequality, we say our function is strictly convex.
Defining Convex Functions
Jensen’s Inequality is important in probability, in information theory, and in statistical physics.
In probability, it is central in the derivation of an important algorithm called the Expectation-Maximization algorithm. It also allows us to prove the consistency of maximum likelihood estimators. It can also be used to show that the arithmetic mean for a set of positive scalars is greater than or equal to their geometric mean. For how this is shown, see: Arithmetic Mean ≥ Geometric Mean.
In statistical physics, this inequality is most important if the convex function g is an exponential function, and where the expected values E are expected values with respect to a probability distribution.
In information theory, Jensen’s Inequality can be used to derive Gibbs inequality, which tells us about the mathematical entropy of discrete probability distributions.
Smith, Andrew. Convex Sets and J’s Inequality. University College Dublin. Retrieved from https://www.ucd.ie/mathstat/t4media/convex-sets-and-jensen-inequalities-mathstat.pdf on July 20, 2019.
Derpanis, Konstantinos G. J’s Inequality. Version 1.0, March 2015. Retrieved from Retrieved from http://www.cs.yorku.ca/~kosta/CompVis_Notes/jensen.pdf on July 20, 2019
Pishro-Nik, Hossein. Introduction to Probability. Retrieved from https://www.probabilitycourse.com/chapter6/6_2_5_jensen%27s_inequality.php on July 20, 2019.
Stephanie Glen. "Jensen’s Inequality: Definition, Examples" From CalculusHowTo.com: Calculus for the rest of us! https://www.calculushowto.com/jensens-inequality/
Need help with a homework or test question? With Chegg Study, you can get step-by-step solutions to your questions from an expert in the field. Your first 30 minutes with a Chegg tutor is free!