 # Jensen’s Inequality: Definition, Examples

Share on

Jensen’s inequality is a special inequality that has to do with convex sets. It says that if a particular function g is convex,

E[g(X)] ≥ g(E[X]).

Here E is the expected value, the mathematical expectation, or the average. It can be a probability weighted mean; so Jensen’s inequality also tells us that, if w1, w2…wn are weights such that Then, for arbitrary x: If all the xj are equal but one of the wj equal zero, then the function f(x) the ≥ sign becomes an equals sign. If this is the only time in which that inequality becomes an inequality, we say our function is strictly convex.

## Defining Convex Functions

A convex function is where a line segment between any two points on the graph of a function is always above — or on—the graph. ## Applications of Jensen’s Inequality

Jensen’s Inequality is important in probability, in information theory, and in statistical physics.

In probability, it is central in the derivation of an important algorithm called the Expectation-Maximization algorithm. It also allows us to prove the consistency of maximum likelihood estimators. It can also be used to show that the arithmetic mean for a set of positive scalars is greater than or equal to their geometric mean. For how this is shown, see: Arithmetic Mean ≥ Geometric Mean.

In statistical physics, this inequality is most important if the convex function g is an exponential function, and where the expected values E are expected values with respect to a probability distribution.

In information theory, Jensen’s Inequality can be used to derive Gibbs inequality, which tells us about the mathematical entropy of discrete probability distributions.

## References

Smith, Andrew. Convex Sets and J’s Inequality. University College Dublin. Retrieved from https://www.ucd.ie/mathstat/t4media/convex-sets-and-jensen-inequalities-mathstat.pdf on July 20, 2019.