Dichotomous thinking
In statistics, dichotomous thinking or binary thinking is the process of seeing a discontinuity in the possible values that a p-value can take during null hypothesis significance testing: it is either above the significance threshold (usually 0.05) or below. When applying dichotomous thinking, a first p-value of 0.0499 will be interpreted the same as a p-value of 0.0001 (the null hypothesis is rejected) while a second p-value of 0.0501 will be interpreted the same as a p-value of 0.7 (the null hypothesis is accepted). The fact that first and second p-values are mathematically very close is thus completely disregarded and values of p are not considered as continuous but are interpreted dichotomously with respect to the significance threshold. A common measure of dichotomous thinking is the cliff effect.[1] A reason to avoid dichotomous thinking is that p-values and other statistics naturally change from study to study due to random variation alone;[2][3] decisions about refutation or support of a scientific hypothesis based on a result from a single study are therefore not reliable.[4]
Dichotomous thinking is very often associated with p-value reading[5][6][7] but it can also happen with other statistical tools such as interval estimates.[1][8]
See also
[edit]References
[edit]- ^ a b Lai, Jerry (2019). "DICHOTOMOUS THINKING: A PROBLEM BEYOND NHST" (PDF). ICOTS8. Retrieved 23 October 2018.
- ^ Cumming, Geoff (2014). "The New Statistics: Why and How". Psychological Science. 25 (1): 7–29. doi:10.1177/0956797613504966. ISSN 0956-7976. PMID 24220629. S2CID 3484092.
- ^ Berner, Daniel; Amrhein, Valentin (2022). "Why and how we should join the shift from significance testing to estimation". Journal of Evolutionary Biology. 35 (6): 777–787. doi:10.1111/jeb.14009. ISSN 1010-061X. PMC 9322409. PMID 35582935.
- ^ Amrhein, Valentin; Greenland, Sander; McShane, Blake (2019). "Scientists rise up against statistical significance". Nature. 567 (7748): 305–307. doi:10.1038/d41586-019-00857-9. PMID 30894741. S2CID 84186074.
- ^ Rosenthal, Robert; Gaito, John (1963). "The Interpretation of Levels of Significance by Psychological Researchers". The Journal of Psychology. 55 (1). Informa UK Limited: 33–38. doi:10.1080/00223980.1963.9916596. ISSN 0022-3980.
- ^ Nelson, Nanette; Rosenthal, Robert; Rosnow, Ralph L. (1986). "Interpretation of significance levels and effect sizes by psychological researchers". American Psychologist. 41 (11). American Psychological Association (APA): 1299–1301. doi:10.1037/0003-066x.41.11.1299. ISSN 1935-990X.
- ^ Besançon, Lonni; Dragicevic, Pierre (2019). "The Continued Prevalence of Dichotomous Inferences at CHI". Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. New York, New York, USA: ACM Press. pp. 1–11. doi:10.1145/3290607.3310432. ISBN 978-1-4503-5971-9.
- ^ Helske, Jouni; Helske, Satu; Cooper, Matthew; Ynnerman, Anders; Besancon, Lonni (2021). "Can Visualization Alleviate Dichotomous Thinking? Effects of Visual Representations on the Cliff Effect". IEEE Transactions on Visualization and Computer Graphics. 27 (8). Institute of Electrical and Electronics Engineers (IEEE): 3397–3409. arXiv:2002.07671. doi:10.1109/tvcg.2021.3073466. ISSN 1077-2626. PMID 33856998. S2CID 233230810.