Talk:Solomonoff's theory of inductive inference

Shouldn't this entry be titled 'universal inductive inference', since there are many more models of inductive inference than the Solomonoff/AIT model? --Johnny Logic 05:54, 8 January 2006 (UTC)[reply]

Have created a redirect from Identification by next value. Someone knowledgable on inductive inference could useful expand this article to include information on different techniques. QuiteUnusual 13:07, 7 October 2006 (UTC)[reply]

This article is nowhere near meeting minimum Wikipedia quality standards. In particular, the section entitled "Modern applications" is appalling. TheSeven (talk) 09:57, 22 May 2014 (UTC)[reply]

Is there a way to add a clarification needed tag to an entire article? EDIT: just found out about the {{clarity}} tag. GreatBigDot (talk) 20:45, 13 June 2017 (UTC)[reply]

Why does the Turing Machine section have a warning about no citations? Every other sentence ends with a citation in parens! TravellerDMT-07 (talk) 00:18, 5 October 2019 (UTC)[reply]

The article makes the statement, in the subsection titled 'Solomonoff's Uncomputability', that "[...] he showed that computability and completeness are mutually exclusive: any complete theory must be uncomputable." This gives the impression that Solomonoff discovered this property of computable logic; which is both misleading and false. It was Godel that first discovered this property with his incompleteness theorems. A link should be made between this statement and Godel's findings in order to avoid misrepresenting Solomonoff as the discoverer of this property of finite logic. — Preceding unsigned comment added by 86.30.111.102 (talk) 15:00, 21 March 2021 (UTC)[reply]

Completely unrelated to Goedel's theorem. In that case, logical systems that include arithmentic can't be simultaneously consistent or complete. consistent != computatble. Also, the definition of completeness in the two cases is different. The proof is also structurally different. There is no arithmetization is Solomonoff's, which is more similar to the proof of incomputability of the halting problem (an adversarial construction). I think you are overreacting to the shared word "completeness". — Preceding unsigned comment added by A1957 (talk • contribs) 18:55, 5 November 2021 (UTC)[reply]

What's up with that Turing machines section?

Does anybody have a good understating of what "Turing machines" section is talking about and why it's there? I gave it a read and my impression it only claims that there's some awesome theory that is "next step in the development of computer science", rants that it's misunderstood by "some researchers" and cites Burgin a lot --- no definitions, no properties, nothing about how it's relevant to Solomonoff's induction. It seems completely useless as is.

TM section is completely unrelated to this entry. I recommend removal.

Contradiction regarding probability of large programs

I find these two consecutive statements in the text to be incompatible. The first is a mathematical statement, the second informal, so it may have to deal with my interpretation of the latter one

1. for every ϵ {\displaystyle \epsilon } \epsilon > 0, there is some length l such that the probability of all programs longer than l is at most ϵ {\displaystyle \epsilon } \epsilon .

2. This (1 above) does not, however, preclude very long programs from having very high probability.

It seems to me 1. does indeed prevent the probability of any programs larger than l (very long) to be greater than epsilon (very high), as the sum of all the probabilities of such programs, each positive quantities, is itself less than epsilon.

Should probably be merged with Minimum Description Length

The Minimum description length article is the right place for this. But there is a persistent historical problem with the MDL article, due to decades-old historical abuses of the phrase "minimum description length". That abuse must be remedied because it is actually poisoning not just machine learning/AI as a discipline but the practice of science, itself.

My rewrite of the MDL article attempted to reduce the damage. But the decades-old degenerate statistical notions once again took front-and-center priority in the article relegating the ideal MDL principle (algorithmic MDL) to an afterthought. There is no excuse for giving priority to notions that are not only more specialized but less intuitive than the general notion. The more specialized notions are quite reasonable to mention as specializations along with what they sacrifice. In this case, statistics sacrifice rigorous causal inference from the data and lead to endless heated debates featuring such tropes as "correlation doesn't imply causation" applied selectively according to the bias of the participants in discourse.

This problem is severe enough that it should be a high priority for Wikipedia to fix. Its profound damage to society is due to the damage to science (not to mention machine learning) in the information age and, in particular, to the social sciences which are at the nexus of increasingly polarized conflict over which social theory shall set policy.

The tendency to conflate "The MDL Principle" with model creation techniques, just as Solomonoff's proofs are so-conflated, is the proximate cause of this damage. Both are actually model selection criteria. The ultimate cause may be that people get confused by the fact that machine learning systems are "algorithmic" and then go from there to think that "algorithmic information" must be information that is created by algorithms, rather than information that is an algorithm. People keep trying to automate model creation and in so doing they obviously need to apply the best and most rigorous model selection theory at hand, but the two are not the same!

With statistical model selection criteria -- arising from such statistical notions as Bayesian Information Criterion (which is frequently equated with "MDL") -- elbowing out the the ideal Algorithmic Information Criterion for model selection (another name for the, unfortunately, mis-named Solomonoff "Induction"), we keep depriving science of its ideal. This poison has been coursing through the veins of Moore's Law ever since the 1960s and fans of Rissanen are doing neither him nor themselves any favors -- let alone society at large. Jim Bowery (talk) 00:16, 1 November 2022 (UTC)[reply]

Recent edit in lead section

I agree the programming language must be chosen before applying Solomonoff's theory. But then, the "best possible scientific model" depends on that choice, and hence is of little value.

As a aside, I don't understand the connection to the post-hoc fallacy. - Jochen Burghardt (talk) 09:15, 23 October 2024 (UTC)[reply]