Talk:Quantities of information

Real world examples?

I work with software engineers who directed me to this page. I am not myself an engineer. A selection of simplified "real world" examples of data would be very useful. Would someone be up to providing some to make the model understandable to ordinary humans? Providing how the information fits into the diagram would also be useful. In the "weather forecast" example, I had difficulty translating the example into the diagram, and vice-versa - although I did understand the "no information conveyed" vs. "information conveyed" comparison. 172.10.237.153 (talk) 21:20, 29 July 2016 (UTC)[reply]

I think one helpful example might be flipping two coins, considering (1) the result of the first flip, (2) the result of the second, and (3) the number of heads. How these relate could illustrate self-information, entropy, joint entropy, conditional entropy, and mutual information without resorting to non-integer-result logarithms. (Power-of-two logarithms should be easier to understand.) If there's support for this, I'll considered adding to the article (or someone else can do it if they so desire). For other sections, it might be useful to note the asymmetry of relative entropy with an inverse square distribution versus a power law, and how it means that, say, compressing power law when you expect inverse square isn't too bad, which compressing inverse square when you expect a power law has differential entropy, and thus an expected penalty of, infinity. (Of course, that leaves open how finite values can have an expected value of infinity, so might not be good to beginners. See, e.g., St. Petersburg paradox.) I can't think of any elementary examples for differential entropy. Anyway, I want to make sure there exist people who think this is a good idea and no one who thinks it's a bad idea, e.g., due to WP:OR, which some editors have broadly interpret as including mathematical examples; the definition of WP:CALC is in the eye of the beholder. Calbaer (talk) 22:08, 7 August 2017 (UTC)[reply]

I'm a software engineer and I have no idea what this article is trying to tell me 2603:9001:3400:4F14:7C85:A018:B953:C03C (talk) 03:02, 4 March 2023 (UTC)[reply]

"Self Information" example

The example at the bottom of the section "Self-Information" relies of the fact that the sun always sets. Due to the inclined axis of the Earth, this is not true for certain regions. See Midnight Sun 188.126.86.216 (talk) 11:48, 25 October 2009 (UTC)[reply]

Description as "misleading"

The Venn diagram is described as "misleading", yet no explanation is given as to how or why it is misleading, except for the citation that presumably explains why. However I think it should also be explained within the article itself, as otherwise readers will also be doubtful as to whether the other accompanying information with the diagram should also be treated as misleading. 111.220.88.36 (talk) 09:04, 25 November 2022 (UTC)[reply]