Calculus How To

Jensen’s Inequality: Definition, Examples

Share on

Calculus Definitions >

Jensen’s inequality is a special inequality that has to do with convex sets. It says that if a particular function g is convex,

E[g(X)] ≥ g(E[X]).


Here E is the expected value, the mathematical expectation, or the average. It can be a probability weighted mean; so Jensen’s inequality also tells us that, if w1, w2…wn are weights such that
Jensen's Inequality Conditions


Then, for arbitrary x:
Jensen's Inequality
If all the xj are equal but one of the wj equal zero, then the function f(x) the ≥ sign becomes an equals sign. If this is the only time in which that inequality becomes an inequality, we say our function is strictly convex.

Defining Convex Functions

A convex function is where a line segment between any two points on the graph of a function is always above — or on—the graph.


Applications of Jensen’s Inequality

Jensen’s Inequality is important in probability, in information theory, and in statistical physics.

In probability, it is central in the derivation of an important algorithm called the Expectation-Maximization algorithm. It also allows us to prove the consistency of maximum likelihood estimators. It can also be used to show that the arithmetic mean for a set of positive scalars is greater than or equal to their geometric mean. For how this is shown, see: Arithmetic Mean ≥ Geometric Mean.

In statistical physics, this inequality is most important if the convex function g is an exponential function, and where the expected values E are expected values with respect to a probability distribution.

In information theory, Jensen’s Inequality can be used to derive Gibbs inequality, which tells us about the mathematical entropy of discrete probability distributions.

References

Smith, Andrew. Convex Sets and J’s Inequality. University College Dublin. Retrieved from https://www.ucd.ie/mathstat/t4media/convex-sets-and-jensen-inequalities-mathstat.pdf on July 20, 2019.
Derpanis, Konstantinos G. J’s Inequality. Version 1.0, March 2015. Retrieved from Retrieved from http://www.cs.yorku.ca/~kosta/CompVis_Notes/jensen.pdf on July 20, 2019
Pishro-Nik, Hossein. Introduction to Probability. Retrieved from https://www.probabilitycourse.com/chapter6/6_2_5_jensen%27s_inequality.php on July 20, 2019.

CITE THIS AS:
Stephanie Glen. "Jensen’s Inequality: Definition, Examples" From CalculusHowTo.com: Calculus for the rest of us! https://www.calculushowto.com/jensens-inequality/
------------------------------------------------------------------------------

Need help with a homework or test question? With Chegg Study, you can get step-by-step solutions to your questions from an expert in the field. Your first 30 minutes with a Chegg tutor is free!


Leave a Reply

Your email address will not be published. Required fields are marked *