Jensen’s inequality is a special inequality that has to do with convex sets. It says that if a particular function g is convex,
E[g(X)] ≥ g(E[X]).
Here E is the expected value, the mathematical expectation, or the average. It can be a probability weighted average; so Jensen’s inequality also tells us that, if w1, w2…wn are weights such that
Then, for arbitrary x:
If all the xj are equal but one of the wj equal zero, then the function f(x) the ≥ sign becomes an equals sign. If this is the only time in which that inequality becomes an inequality, we say our function is strictly convex.
Defining Convex Functions
A convex function is where a line segment between any two points on the graph of a function is always above — or on—the graph.
Applications of Jensen’s Inequality
Jensen’s Inequality is important in probability, in information theory, and in statistical physics. In probability, it is central in the derivation of an important algorithm called the Expectation-Maximization algorithm. It also allows us to prove the consistency of maximum likelihood estimators.
In statistical physics, this inequality is most important if the convex function g is an exponential, and where the expected values E are expected values with respect to a probability distribution.
In information theory, Jensen’s Inequality can be used to derive Gibbs inequality, which tells us about the mathematical entropy of discrete probability distributions.
Smith, Andrew. Convex Sets and J’s Inequality. University College Dublin. Retrieved from https://www.ucd.ie/mathstat/t4media/convex-sets-and-jensen-inequalities-mathstat.pdf on July 20, 2019.
Derpanis, Konstantinos G. J’s Inequality. Version 1.0, March 2015. Retrieved from Retrieved from http://www.cs.yorku.ca/~kosta/CompVis_Notes/jensen.pdf on July 20, 2019
Pishro-Nik, Hossein. Introduction to Probability. Retrieved from https://www.probabilitycourse.com/chapter6/6_2_5_jensen%27s_inequality.php on July 20, 2019.
Stephanie Glen. "Jensen’s Inequality" From CalculusHowTo.com: Calculus for the rest of us! https://www.calculushowto.com/jensens-inequality/
Need help with a homework or test question? With Chegg Study, you can get step-by-step solutions to your questions from an expert in the field. Your first 30 minutes with a Chegg tutor is free!