User:PAR/Test

Entropy

The information entropy of the Von Mises distribution is defined as:

$H=-\int _{\Gamma }f(\theta ;\mu ,\kappa )\,\ln(f(\theta ;\mu ,\kappa ))\,d\theta \,$

where $\Gamma$ is any interval of length $2\pi$. The logarithm of the density of the Von Mises distribution is straightforward:

$\ln(f(\theta ;\mu ,\kappa ))=-\ln(2\pi I_{0}(\kappa ))+\kappa \cos(\theta )\,$

The characteristic function representation for the Von Mises distribution is:

$f(\theta ;\mu ,\kappa )={\frac {1}{2\pi }}\left(1+2\sum _{n=1}^{\infty }\phi _{n}\cos(n\theta )\right)$

where $\phi _{n}=I_{|n|}(\kappa )/I_{0}(\kappa )$. Substituting these expressions into the entropy integral, exchanging the order of integration and summation, and using the orthogonality of the cosines, the entropy may be written:

$H=\ln(2\pi I_{0}(\kappa ))-\kappa \phi _{1}=\ln(2\pi I_{0}(\kappa ))-\kappa {\frac {I_{1}(\kappa )}{I_{0}(\kappa )}}$

Collapsible

Table of closed-form stable distribution PDF's
$f\left(x;{\tfrac {1}{3}},0\right)=Re\left({\frac {2\exp(-i\pi /4)}{3{\sqrt {3}}\pi }}x^{-3/2}S_{0,1/3}{\Bigl (}{\frac {2\exp(i\pi /4)}{3{\sqrt {3}}}}x^{-1/2}{\Bigr )}\right)$ where $S_{\mu ,\nu }(z)$ is a Lommel function. (Reference: Garoni & Frankel.)
$f\left(x;{\tfrac {1}{2}},0\right)={\frac {\vert x\vert ^{-3/2}}{\sqrt {2\pi }}}\left(\sin \left({\frac {1}{4\vert x\vert }}\right)\left[{\frac {1}{2}}-S\left({\sqrt {\frac {1}{2\pi \vert x\vert }}}\right)\right]+\cos \left({\frac {1}{4\vert x\vert }}\right)\left[{\frac {1}{2}}-C\left({\sqrt {\frac {1}{2\pi \vert x\vert }}}\right)\right]\right)$ where S(x) and C(x) are Fresnel Integrals (Reference: Hopcraft et. al..)
$f\left(x;{\tfrac {2}{3}},0,\right)={\frac {1}{2{\sqrt {3\pi }}}}\vert x\vert ^{-1}\exp \left({\frac {2}{27}}x^{-2}\right)W_{-1/2,1/6}\left({\frac {4}{27}}x^{-2}\right)$ where $W_{k,\mu }(z)$ is a Whittaker function.) (Reference: Uchaikin & Zolotarev .)
Cauchy distribution $f\left(x;1,0\right)={\frac {1}{\pi (x^{2}+1)}}$ $f\left(x;{\tfrac {4}{3}},0\right)={\frac {3^{5/4}}{4{\sqrt {2\pi }}}}{\frac {\Gamma (7/12)\Gamma (11/12)}{\Gamma (6/12)\Gamma (8/12)}}\,_{2}F_{2}\left({\frac {7}{12}},{\frac {11}{12}};{\frac {6}{12}},{\frac {8}{12}};{\frac {3^{3}x^{4}}{4^{4}}}\right)-{\frac {3^{11/4}x^{3}}{4^{3}{\sqrt {2\pi }}}}{\frac {\Gamma (13/12)\Gamma (17/12)}{\Gamma (18/12)\Gamma (15/12)}}\,_{2}F_{2}\left({\frac {13}{12}},{\frac {17}{12}};{\frac {18}{12}},{\frac {15}{12}};{\frac {3^{3}x^{4}}{4^{4}}}\right)$ (Reference: Garoni & Frankel .) $f\left(x;{\tfrac {3}{2}},0\right)={\frac {1}{\pi }}\Gamma (5/3)\,_{2}F_{3}\left({\frac {5}{12}},{\frac {11}{12}};{\frac {1}{3}},{\frac {1}{2}},{\frac {5}{6}};-{\frac {2^{2}x^{6}}{3^{6}}}\right)-{\frac {x^{2}}{3\pi }}\,_{3}F_{4}\left({\frac {3}{4}},1,{\frac {5}{4}};{\frac {2}{3}},{\frac {5}{6}},{\frac {7}{6}},{\frac {4}{3}};-{\frac {2^{2}x^{6}}{3^{6}}}\right)+{\frac {7x^{4}}{3^{4}\pi ^{2}}}\Gamma (4/3)\,_{2}F_{3}\left({\frac {13}{12}},{\frac {19}{12}};{\frac {7}{6}},{\frac {3}{2}},{\frac {5}{3}};-{\frac {2^{2}x^{6}}{3^{6}}}\right)$ (Reference: Garoni & Frankel.) Normal distribution $f\left(x;2,0\right)={\frac {e^{x^{2}/2}}{\sqrt {2\pi }}}$ The following are asymmetric distributions (specifically, where β = 1). $f\left(x;{\tfrac {1}{3}},1\right)={\frac {1}{\pi }}{\frac {2{\sqrt {2}}}{3^{7/4}}}x^{-3/2}K_{1/3}\left({\frac {4{\sqrt {2}}}{3^{9/4}}}x^{-1/2}\right)$ where Kv(x) is a modified Bessel function of the second kind. (Reference: Hopcraft et. al..) Lévy distribution $f\left(x;{\tfrac {1}{2}},1\right)=$ $f\left(x;{\tfrac {2}{3}},1\right)={\frac {\sqrt {3}}{\sqrt {\pi }}}\vert x\vert ^{-1}\exp \left(-{\frac {16}{27}}x^{-2}\right)W_{1/2,1/6}\left({\frac {32}{27}}x^{-2}\right)$ (Reference: Zolotarev 1961 .) $f\left(x;{\tfrac {3}{2}},1\right)=\left\{{\begin{array}{ll}{\frac {\sqrt {3}}{\sqrt {\pi }}}\vert x\vert ^{-1}\exp \left({\frac {1}{27}}x^{3}\right)W_{1/2,1/6}\left(-{\frac {2}{27}}x^{3}\right)&x<0\\{\frac {1}{2{\sqrt {3\pi }}}}\vert x\vert ^{-1}\exp \left({\frac {1}{27}}x^{3}\right)W_{-1/2,1/6}\left({\frac {2}{27}}x^{3}\right)&x\geq 0\end{array}}\right)$. (Reference: Kagan et. al..) The symmetric distributions for which α = p / q and p > q can be derived from a result in Garoni & Frankel<ref name="G&F"\>. Also Meijer functions (Zolatarev)

Relationship to Tsallis entropy

While the normal distribution has the maximum entropy for a fixed first moment $(E[X])$ and second moment $(E[X^{2}])$ of the random variable, the Student's t-distribution has the maximum Tsallis entropy for a fixed first and second moment.

The Tsallis entropy of a probability density $f(t)$ is defined as:

$H={\frac {1-\int _{\Gamma }f(t)^{q}\,dt}{1-q}}$

where $\Gamma$ is the support of f(t). For a normalized density (zeroth moment equal to unity), with fixed values of the first and second moment, using the calculus of variations and the method of Lagrange multipliers, the entropy H will be maximized when the Lagrangian equation is satisfied:

$\delta L=0=\delta H+\lambda _{0}\int _{\Gamma }\delta f\,dt+\lambda _{1}\int _{\Gamma }t\delta f\,dt+\lambda _{1}\int _{\Gamma }t^{2}\delta f\,dt$

where the $\lambda _{i}$ are the Lagrange multipliers. The variation of the Tsallis entropy is

$\delta H=-{\frac {\int _{\Gamma }\delta (f^{q})\,dt}{1-q}}=-{\frac {\int _{\Gamma }qf^{q-1}\delta f\,dt}{1-q}}$

and so the Lagrange equation is satisfied when:

${\frac {qf(t)^{q-1}}{q-1}}=\lambda _{0}+\lambda _{1}t+\lambda _{2}t^{2}$

or, solving for f(t):

$f(t)=\left({\frac {q}{1-q}}\,\left(\lambda _{0}+\lambda _{1}t+\lambda _{2}t^{2}\right)\right)^{\frac {1}{1-q}}$

Solving for the $\lambda _{i}$ using the three moment constraints (assuming the centered Student's t-distribution for which the mean of t is zero):

$E[1]=1\,$

$E[t]=0\,$

$E[t^{2}]={\frac {\nu }{\nu -2}}$

yields the Student's t-distribution as the expression which maximizes the Tsallis entropy.