Talk:Dilution (neural networks)

	This article is within the scope of WikiProject Computing, a collaborative effort to improve the coverage of computers, computing, and information technology on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.ComputingWikipedia:WikiProject ComputingTemplate:WikiProject ComputingComputing
???	This article has not yet received a rating on the project's importance scale.

Robotics

	This article is within the scope of WikiProject Robotics, a collaborative effort to improve the coverage of Robotics on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.RoboticsWikipedia:WikiProject RoboticsTemplate:WikiProject RoboticsRobotics
???	This article has not yet received a rating on the project's importance scale.

Possible expansion from Convolutional Neural Network article

This article, apart from its disputed factual accuracy, is quite short. There seems to be more information in the Convolutional Neural Networks article (some of the information in the Dropout section of that article is rather general and could be applied to help improve this article). Mocl125 (talk) 20:50, 10 June 2020 (UTC)[reply]

Relation between strong/weak dilution and dropout

In response to @Jeblad's note in the article about strong/weak dilution, I looked into one of the sources on this topic (from 1991) that Jeblad mentions, Introduction to the Theory of Neural Computation by Hertz, Krogh, and Palmer. Unfortunately, the section about dilution requires a level of understanding about earlier neural networks that I do not have yet. I would really appreciate if someone with more experience than me could include some reference to dilution (and/or to the even earlier predecessors of this idea, if they exist) in this article, if relevant. Many thanks! Mocl125 (talk) 05:08, 11 June 2020 (UTC)[reply]

I don't think it matters. Have you ever once, in your life, heard it called "dilution" instead of "dropout"? I read an Arxiv DL paper a day and have for a decade, and I have never once heard of it outside of this WP entry, and you'll have to search long and hard for a DL researcher who knows what 'dilution' is but not 'dropout'... It's nice that someone or other somewhere back in the 1980s or whatever invented something that if you squint looks like dropout, but no one cares, and that should be stuffed into a 'History' section or something (with a big caveat 'it had no influence on anyone at all and was later reinvented'), for those who do care about playing Trivial Pursuit with citations. As the article stands, it seems like a mix of WP:POVTITLE and WP:UNDUE to me, and it should be renamed back to 'Dropout (neural networks)' and refocused on what people actually know dropout for, like regularization and ensembling. --Gwern (contribs) 18:07 8 December 2022 (GMT) 18:07, 8 December 2022 (UTC)[reply]

Rename article to “Dropout (neural network)”

I had the same surprised reaction as @Gwern. I’ve used TensorFlow and set the "dropout" parameter a few times, but I’ve never come across anything called “dilution”. So I did a little digging. While both have been described as regularization techniques in neural networks, dropout is clearly the dominant and widely recognized term. It appears in major deep learning libraries like TensorFlow and PyTorch, in textbooks, and across recent research. Given that, it might make more sense for this article to be titled "Dropout (neural network)", with “dilution” scaled down to a historical section.

Per WP:COMMONNAME and WP:UNDUE, Wikipedia should reflect the terminology most commonly used in reliable sources, and avoid giving undue weight to fringe or minority usage. To support this, here’s what I found:

IEEE Xplore: 1) "dropout" + "neural network" + "regularization" = 761 results; 2) "dilution" + "neural network" + "regularization" = 3 results

arXiv: 1) "dropout" + "neural network" + "regularization" = 314 results; 2) "dilution" + "neural network" + "regularization" = 1 result

What do others think? HerBauhaus (talk) 16:02, 25 March 2025 (UTC)[reply]