Wikipedia talk:Identifying reliable sources (medicine)/Archive 29

Archive 25

←

Archive 27

Archive 28

Archive 29

Archive 30

Archive 31

Archive 32

Very good introductions of primary papers as MEDRS

I'm not really suggesting a policy change here, just noting something interesting and asking for feedback. One thing I've noticed about the form of papers in some fields is that they tend to provide an introductory literature review of the related topic. These can be very good, though tend to be "thematic" rather than "systematic" so if a recent systematic review exists a reader should probably look at this as well. What do we think about these introductions - they do place other papers in context and if you can find, say, a systematic review from 2003, or a very good introduction from a primary paper then is it actually better to use the review from 2003? Talpedia (talk) 04:45, 20 January 2021 (UTC)

This has been discussed before, so you may find something in the archives. The guideline currently says [Research papers] normally contain introductory, background, or review sections that place their research in the context of previous work; these sections may be cited in Wikipedia with care: they are often incomplete[20] and typically less useful or reliable than reviews or other sources, such as textbooks, which are intended to be reasonably comprehensive. This is indeed a secondary-source / secondary-literature portion of the paper, wrt our overall sourcing policies, just not a very good one. My understanding is that when the research paper is peer reviewed, that section is not critically examined, other than for obvious whoppers perhaps. There are other people here who may know more/better. The intro could indeed be focused just on one area of the subject that the research seeks to confirm or reject, and not give a balanced picture. The wording may not be as careful as a dedicated review. Of course, any review can be biased or badly written. I don't think it is straightforward to always decide whether to favour systematic vs narrative or by date, as the context can matter and they serve different roles. If what the article is saying is non-contentious and hasn't changed for many years, nobody should care all that much if the review is old, and it may be no newer one has been done simply because there is no new research to systematically review. -- Colin°^Talk 10:01, 20 January 2021 (UTC)

"Thematic" is a great description of those sections. WhatamIdoing (talk) 16:57, 21 January 2021 (UTC)

Possible-over broad application of this guideline

This guideline is being cited to prevent New York magazine from being used [1] because of claims by some editors that a discussion of the historical origins of a disease are biomedical information, and therefore subject to this policy, rather than WP:RS. It would, once the dust settles, be prudent to decide when this guideline applies, and to narrow its scope if necessary, so that it clearly reflects its intent, which seems to be to prevent harm to those who rely on Wikipedia for medical advice, rather applying it as a catch-all to all medical information, even information which could have no conceivable influence on the treatment or medical actions of any individual. Park3r (talk) 08:37, 18 January 2021 (UTC)

Many times, people asking and responding to reliable source questions are over-simplifying things. While sometimes it is possible to name a publication (The Daily Mail, for example) and get consensus they are not a reliable source for nearly anything, other times one must narrow things a bit to what kind of article it was, whether the author was a staff writer or guest, what the article actually claimed vs what we actually claim, etc. A guest opinion piece offering a controversial hypothesis is quite a different kind of piece from an obituary on Phil Spector, who got arrested for entering the Capitol without an invite, political speculation on who Trump might pardon...

I believe MEDRS is simply the consequence of applying project-wide consensus on reliable sourcing, original research, apportioning weight to our coverage of topics, etc. Whether or not one believes MEDRS applies to X or Y doesn't shift things, since the fundamentals can still be determined in any individual case from core policy/guideline: all it does is get us to the answer a bit quicker sometimes. I think some editors have misunderstood the scope of MEDRS and are confusing things that are politics or economics or history just because Covid. But also there's a lot of political agenda pushing going on. And when one is in a fight, it is tempting to want black and white rules, rather than admit it can be complicated. I don't think this particular issue is complicated at all: a novelist writing a radical opinion piece in a weekend magazine is not a reliable source for all that much really, and WP:EXTRAORDINARY is core policy. -- Colin°^Talk 15:40, 18 January 2021 (UTC)

If I understand you correctly, you agree that MEDRS is NOT the correct guideline to use when evaluating the New Yorker article, but that in your opinion, the article fails on the basis of WP:RS. That’s certainly a valid argument, even if one doesn’t agree with the conclusion. My primary concern here is the over-broad application of MEDRS, which doesn’t appear to contradict your position. Park3r (talk) 02:12, 19 January 2021 (UTC)

No I didn't say that. Determining the origin of a novel virus is medical science, so would come under MEDRS. I don't think this is a good example of something that fails to be in scope for MEDRS. I suggest that if you want a discussion about what is in scope or out of scope, then that would be a lot easier, and less likely to generate unhelpfully polarised responses, if the source=>text question wasn't one that clearly failed WP:V too, and wasn't a political hot potato. I think those cases where people are misapplying MEDRS, the guideline doesn't itself need changed: it is those editors who need to remember that not everything about covid is medical science. -- Colin°^Talk 08:58, 19 January 2021 (UTC)

The story about SARS-CoV-2's origin as an accidental lab leak is now being pushed with a Feature story from a magazine best known for reviewing consumer electronics.

Aside from the difficulty that the facts don't line up, these are remarkably illogical stories. They usually explicitly say, out of one side of their mouths, that it's not a man-made virus, and then they write three thousand words about how COVID-19's origins could be a lab leak, at least if you ignore most of the facts and don't know anything about viruses. One wonders how the virus got into the lab that it supposedly leaked from in the first place, if the same story says that it neither originated inside the lab nor outside of the lab. I would not recommend using such a confused, illogical source under any circumstances. WhatamIdoing (talk) 16:56, 21 January 2021 (UTC)

That's right - the "natural lab leak" story just doesn't seem to make any sense. In general it is dispiriting how un-tethered from decent sources much of the Talk page argument on this topic is. Colin has it right, WP:MEDRS is just our usual sourcing standards on steroids. When we have several good sources that directly address the question of the virus' origin, it just baffles me why editors want to swerve around and use something weaker instead. Actually, it doesn't baffle me: the reason is that this is a POV-battle between some editors, with politics lurking in the background. Alexbrn (talk) 17:04, 21 January 2021 (UTC)

Text shuffling at Choosing sources # Biomedical journals

Hi all, I reorganized some text in Choosing sources#Biomedical journals in an effort to make the section flow better. The old section discussed reviews, then primary publications, then reviews again, then primary publications again, then back to reviews. The hope here was to bring like material together. I find this kind of edit difficult to parse on the diff viewer (since it shows large chunks of text deleted and added), so here's a side-by-side view so folks can check what was done more easily:

Side-by-side view

Before Peer-reviewed medical journals are a natural choice as a source for up-to-date medical information in Wikipedia articles. Journal articles come in many different types, and are a mixture of primary and secondary sources. It is normally best to use reviews and meta-analyses where possible. Reviews give a balanced and general perspective of a topic, and are usually easier to understand. Primary publications describe new research, while review articles summarize and integrate a topic of research into an overall view. In medicine, primary sources include clinical trials, which test new treatments. Broadly speaking, reviews may be narrative or systematic (and sometimes both). Narrative reviews provide a general summary of a topic based on a survey of the literature, which can be useful when outlining a topic. Systematic reviews use sophisticated methodology to address a particular clinical question in as balanced (unbiased) a way as possible. Some systematic reviews also include a statistical meta-analysis to combine the results of several clinical trials to provide stronger quantitative evidence about how well a treatment works for a particular purpose. Systematic reviews and meta-analyses of randomized controlled trials can provide strong evidence of the clinical efficacy of particular treatments in given scenarios, which may in turn be incorporated into medical guidelines or institutional position papers (ideal sources for clinical recommendations). In addition to experiments, primary sources normally contain introductory, background, or review sections that place their research in the context of previous work; these sections may be cited in Wikipedia with care: they are often incomplete and typically less reliable than reviews or other sources, such as textbooks, which are intended to be reasonably comprehensive. If challenged by another editor in good faith, the primary source should be supplemented with a more appropriate source. A general narrative review of a subject by an expert in the field can make a good secondary source covering various aspects of a subject within a Wikipedia article. Such reviews typically do not contain primary research, but can make interpretations and draw conclusions from primary sources that no Wikipedia editor would be allowed to do. A systematic review uses a reproducible methodology to select primary (or sometimes secondary) studies meeting explicit criteria to address a specific question. Such reviews should be more reliable and accurate and less prone to bias than a narrative review. However, whereas a narrative review may give a panorama of current knowledge on a particular topic, a systematic review tends to have a narrower focus. Journals may specialize in particular article types. A few, such as Evidence-based Dentistry (ISSN 1462-0049), publish third-party summaries of reviews and guidelines published elsewhere. If an editor has access to both the original source and the summary, and finds both helpful, it is good practice to cite both sources together (see: Citing medical sources for details). Others, such as Journal of Medical Biography, publish historical material that can be valuable for History sections, but is rarely useful for current medicine. Still others, such as Medical Hypotheses, publish speculative proposals that are not reliable sources for biomedical topics. After Peer-reviewed medical journals are a natural choice as a source for up-to-date medical information in Wikipedia articles. Journal articles come in many different types, and are a mixture of primary and secondary sources. Primary publications describe new research, while review articles summarize and integrate a topic of research into an overall view. In medicine, primary sources include clinical trials, which test new treatments. In addition to experiments, primary sources normally contain introductory, background, or review sections that place their research in the context of previous work; these sections may be cited in Wikipedia with care: they are often incomplete and typically less reliable than reviews or other sources, such as textbooks, which are intended to be reasonably comprehensive. If challenged by another editor in good faith, the primary source should be supplemented with a more appropriate source. Broadly speaking, reviews may be narrative or systematic (and sometimes both). Narrative reviews provide a general summary of a topic based on a survey of the literature, which can be useful when outlining a topic. A general narrative review of a subject by an expert in the field can make a good secondary source covering various aspects of a subject within a Wikipedia article. Such reviews typically do not contain primary research, but can make interpretations and draw conclusions from primary sources that no Wikipedia editor would be allowed to do. Systematic reviews use sophisticated methodology to address a particular clinical question in as balanced (unbiased) a way as possible. Some systematic reviews also include a statistical meta-analysis to combine the results of several clinical trials to provide stronger quantitative evidence about how well a treatment works for a particular purpose. A systematic review uses a reproducible methodology to select primary (or sometimes secondary) studies meeting explicit criteria to address a specific question. Such reviews should be more reliable and accurate and less prone to bias than a narrative review. Systematic reviews and meta-analyses of randomized controlled trials can provide strong evidence of the clinical efficacy of particular treatments in given scenarios, which may in turn be incorporated into medical guidelines or institutional position papers (ideal sources for clinical recommendations). It is normally best to use reviews and meta-analyses where possible. Reviews give a balanced and general perspective of a topic, and are usually easier to understand. However, whereas a narrative review may give a panorama of current knowledge on a particular topic, a systematic review tends to have a narrower focus. Journals may specialize in particular article types. A few, such as Evidence-based Dentistry (ISSN 1462-0049), publish third-party summaries of reviews and guidelines published elsewhere. If an editor has access to both the original source and the summary, and finds both helpful, it is good practice to cite both sources together (see: Citing medical sources for details). Others, such as Journal of Medical Biography, publish historical material that can be valuable for History sections, but is rarely useful for current medicine. Still others, such as Medical Hypotheses, publish speculative proposals that are not reliable sources for biomedical topics. References

The words in the two versions are identical, the change in white space just comes from the merging of two paragraphs. The reorganization made it clear that two sentences were redundant, so I removed those here (the diff view deals with this better). Just wanted to make things clear. Certainly let me know if you have concerns. All the best. Ajpolino (talk) 16:35, 1 February 2021 (UTC)

Quick question on population genetics

Is MEDRS supposed to apply to sources on population genetics which have no relationship to medical issues. Obviously discussion of genetic conditions more prevalent among certain ethnic groups should be held to that standard, but what about simple comparative studies between ethnic groups? Boynamedsue (talk) 14:30, 7 February 2021 (UTC)

Boynamedsue, hi. I had a wee look at your recent contribs to see what might be the source of your question. I see also someone pointed you at Wikipedia:Identifying reliable sources (science), which is at this point just an essay. There's also plain WP:RS and I find WP:WEIGHT is often the most powerful tool in the box: what does the weight of quality literature say on this matter (if anything). I see you removed some text sourced to "Mankind Quarterly". I'm currently reading the book "Superior: The Return of Race Science" and the journalist author comments that that "journal" is by default blocked by her ISP, so I'm kinda surprised it isn't flagged as a forbidden external link on Wikipedia. I'm only part way through the book, but I'd say "comparative studies between ethnic groups" sounds like a minefield that I would not describe as "simple". I'm sure much of the advice in MEDRS will be appropriate to that field. If there is a particular dispute on an article that you want other editors to examine, then WT:MED is a good place to post. -- Colin°^Talk 18:48, 7 February 2021 (UTC)

Hi Colin. Thanks for the answer, Superior is an excellent book! I even have it behind me on my zoom call shelf :) The question is actually re. what looks like an excellent source for population genetics used purely for ancestry, effectively saying stuff like population x has haplogroup a at 50% frequency, whereas population y has haplogroup a at 10% frequency. Someone was claiming that this came under WP:MEDRS, whereas this seemed unlikely to me. The mankind quarterly source wasn't really doing anything in the article, luckily, so I could just trim it and leave the other sources. It certainly wasn't being used for anything nasty. Boynamedsue (talk) 19:13, 7 February 2021 (UTC)

See my comment above in the section "Possible-over broad application of this guideline". As you will know, I'm only reading a lay book on the topic, so am not really qualified to comment on the details of a study in Cell. If you find MEDRS helpful guidance then use it. But if editors are just fighting over whether something is or isn't in scope of MEDRS, it can be useful to go back to core policy or a more general guideline. The Cell paper looks like a primary research paper. See WP:SCHOLARSHIP. It says much the same as MEDRS, only we do it in more words. It may well be excellent science and that's an excellent journal, but we need to use the secondary literature to determine whether there is scientific consensus that agrees with them, disagrees with them or ignores them. And it can sometimes be frustrating if recent research isn't yet discussed. -- Colin°^Talk 19:29, 7 February 2021 (UTC)

Thanks again. Looking at WP:SCHOLARSHIP, the article text almost certainly needs hedging language putting in. There may also be some newer secondary publications that include the research (or some older ones that don't, which would also be preferable according to the guidance). Perhaps even a new "genetics" section might be better. I'll check it out, thanks and all the best. Boynamedsue (talk) 19:41, 7 February 2021 (UTC)

Btw Boynamedsue, you can locate Wikipedia articles that link to bad sites with LinkSearch. -- Colin°^Talk 20:44, 7 February 2021 (UTC)

PubMed knows about seven articles that have cited this one.[2] None are currently marked as being reviews in PubMed's metadata, but some of these were published very recently, so they might actually be reviews and just have incomplete metadata. WhatamIdoing (talk) 02:48, 8 February 2021 (UTC)

WhatamIdoing Thank you very much, I will look at them and see if there is a review. Very much appreciated. Boynamedsue (talk) 11:39, 8 February 2021 (UTC)

MEDLEAD

More input is needed at Wikipedia talk:Manual of Style/Medicine-related articles#MEDLEAD. Crossroads ^-talk- 06:46, 9 February 2021 (UTC)

Copyedits

I've been following the changes made by Ajpolino recently. Mostly seem ok, improving the prose, reducing redundancy, but today's "More minor copyediting" is not just copyediting. The criticisms of newspaper and magazine articles has been trimmed of some of the rationale as to why we avoid them. We have lost "News articles also tend neither to report adequately on the scientific methodology and the experimental error, nor to express risk in meaningful terms." and "Articles in newspapers and popular magazines generally lack the context to judge experimental results". I don't think that should have been removed without discussion, and the edit summary isn't accurate.

The statement "Medical information resources such as WebMD and eMedicine are usually acceptable sources for uncontroversial information; however, as much as possible Wikipedia articles should cite the more established literature directly." has had "as much as possible" removed from it, making it not only stronger but actually contradicts the first half of the sentence. If we say articles "should" cite something else, then the first group of sources are not "usually acceptable". I think this sentence may need a bit of work, as it is a bit vague what a "medical information resource" is, beyond those two examples. There are many high quality medical information websites that are accessible to editors who don't have subscription journal access, and they really are usually acceptable. I would be opposed to MEDRS placing unnecessary burdens on lay editors adding information, particularly for non-controversial issues, and also if it unnecessarily makes our citations inaccessible to readers. For example, I've been using the Oxford Vaccine Group's Vaccine Knowledge Project for information about vaccine ingredients. I also have a classic textbook (Plotkin's vaccines), but none of our readers can check the textbook facts, and the vaccine website is accurate wrt the UK's vaccine schedule (unlike our Wikipedia articles...)

I would encourage everyone to have a look at the changes made recently and check we are happy with them. -- Colin°^Talk 12:23, 3 February 2021 (UTC)

Hi Colin, for what it's worth, I'll add my rationale here. Collapsed below are the before and after side-by-side for the paragraph Colin refers to above. The removed sentences are highlighted in yellow (pardon the harsh color):

Side-by-side view

Before The popular press is generally not a reliable source for scientific and medical information in articles. Most medical news articles fail to discuss important issues such as evidence quality, costs, and risks versus benefits, and news articles too often convey wrong or misleading information about health care. Articles in newspapers and popular magazines generally lack the context to judge experimental results. They tend to overemphasize the certainty of any result, for instance, presenting a new and experimental treatment as "the cure" for a disease or an every-day substance as "the cause" of a disease. Newspapers and magazines may also publish articles about scientific results before those results have been published in a peer-reviewed journal or reproduced by other experimenters. Such articles may be based uncritically on a press release, which themselves promote research with uncertain relevance to human health and do not acknowledge important limitations, even when issued by an academic medical center. News articles also tend neither to report adequately on the scientific methodology and the experimental error, nor to express risk in meaningful terms. For Wikipedia's purposes, articles in the popular press are generally considered independent, primary sources. A news article should therefore not be used as a sole source for a medical fact or figure. Editors are encouraged to seek out the scholarly research behind the news story. One possibility is to cite a higher-quality source along with a more-accessible popular source, for example, with the |laysummary= parameter of {{cite journal}}. After The popular press is generally not a reliable source for scientific and medical information in articles. Most medical news articles fail to discuss important issues such as evidence quality, costs, and risks versus benefits, and news articles too often convey wrong or misleading information about health care. Articles in newspapers and popular magazines tend to overemphasize the certainty of any result, for instance, presenting a new and experimental treatment as "the cure" for a disease or an every-day substance as "the cause" of a disease. Newspapers and magazines may also publish articles about scientific results before those results have been published in a peer-reviewed journal or reproduced by other experimenters. Such articles may be based uncritically on a press release, which themselves promote research with uncertain relevance to human health and do not acknowledge important limitations, even when issued by an academic medical center. For Wikipedia's purposes, articles in the popular press are generally considered independent, primary sources. A news article should therefore not be used as a sole source for a medical fact or figure. Editors are encouraged to seek out the scholarly research behind the news story. One possibility is to cite a higher-quality source along with a more-accessible popular source, for example, with the |laysummary= parameter of {{cite journal}}. References

Brief expanded rationales for each removal. In each case the aim was to remove redundant wording without changing the meaning of the section. Folks can decide if that aim was met or not:

Articles in newspapers... experimental results - The surrounding sentences already state that the popular press/newspaper articles are "generally not reliable", "fail to discuss... evidence quality, costs, and risks versus benefits, "too often convey wrong or misleading information", and "overemphasize the certainty of any result". I felt the removed sentence was unnecessary repetition. I think the reader would get the right impression in the old or the new version.

News articles... meaningful terms. - This sentence seemed like it had fallen out of the newspaper criticism list above. I would've put it up at the top of the paragraph with the rest of the newspaper criticism, but it seemed redundant there, so I removed it.

Medical information resources such as WebMD and eMedicine are usually acceptable sources for uncontroversial information; however, as much as possible Wikipedia articles should cite the more established literature directly. - "should" and "as much as possible... should" have the same meaning to my ear. I don't think the shorter version results in contradiction.

Also for those just tuning in, I made a handful of other recent copyedits as well. I tried to spread them out over edits so they'd be easier to follow. One challenges the diff viewer, so I posted it side-by-side in the section above in case you're curious. Happy to hear thoughts/concerns. Ajpolino (talk) 14:49, 3 February 2021 (UTC)

Wrt "as much as possible", I think it was open to the editor to say "but it wasn't possible for me, as i don't have access to that paper". But as I said, I think that whole sentence needs revisiting to consider what exactly we mean by it. Does MEDRS (or somewhere else on WP:MED) need to pull out certain well known websites and put our position on them, or can we say anything much about "medical information websites". Perhaps they are too much of a case-by-case basis to rule on here. -- Colin°^Talk 17:46, 3 February 2021 (UTC)

Pardon the couple days of radio silence. I do think a list of well known websites could be a help to newer editors. I was recently digging up archived WT:MED discussions on StatPearls for an editor, and parts of the mess at Talk:Ivermectin revolved around newer editors showing up with information they found on odd websites. New pages patrol has a fairly extensive version of this at Wikipedia:New page patrol source guide, typically with very brief blurbs on the extent to which the source is considered reliable. Ajpolino (talk) 17:05, 5 February 2021 (UTC)

I think we should drop the "Medical information resources...should cite the more established literature directly." sentence. It is too vague to be of any use, and where it is specific about two sites, well they are neither representative of all such resources nor should we be in the habit of naming individual sources in a general guideline. It could be read as "Don't trust this Internet thing, rely instead on the paper journals on your university library shelves". I'm not convinced at this point that MEDRS has any evidence of a general pattern to "Medical information resources" beyond the core advice of WP:V to use publications with a reputation for fact checking and accuracy. There may be individual websites, such as WebMD and eMedicine and StatPearls that WP:MED could comment on in project space or essay space, but I don't think they belong on a general guideline page like this. The danger of a vague restrictive sentence in a guideline is that editors use it as a weapon when they want to and ignore it when they want to. -- Colin°^Talk 17:43, 6 February 2021 (UTC)

This waters down the guideline, so we shouldn't. As has been said repeatedly in the past, the whole point of MEDRS is to have stricter standards for medical content. This is very useful as a cautionary note to less experienced editors. And it is a fact that such sites like WebMD are less reliable than the medical literature (much of which is available online anyway). Crossroads ^-talk- 05:01, 7 February 2021 (UTC)

Most of the medical literature is not legally available online. Much of the best medical literature is in expensive books. Wikipedia is the encyclopaedia anyone can edit. A huge amount of Wikipedia is simply unsourced. I'm not arguing against the problem that some websites are so-so in quality, and if editors have a choice, they should find better sources: that was my interpretation of the original text that was changed here. The sentence about "Medical information resources" is too handwavy to be of any value, reads to me like "the whole internet" and so the recent change to make finding alternatives to "the whole internet" mandatory does not have consensus and is also contratictory with itself. I think at the very least we should revert back to the longstanding text. -- Colin°^Talk 18:36, 7 February 2021 (UTC)

I've reverted the "as much as possible" change. It wasn't my intention to strengthen any admonition on web resources. If Colin read it that way, I'm sure others will too. We haven't discussed the two news article-related sentences from the same edit, so I didn't revert those, but certainly anyone can feel free to. Perhaps a future discussion on "medical information resources" may be in order, but I don't think a subsection on "copyedits" is a good place to have it. Ajpolino (talk) 21:12, 7 February 2021 (UTC)

@Crossroads, I do not agree that "the whole point of MEDRS is to have stricter standards" – I rather thought the point was to help us get the content right – but I am pretty sure that if we need to know what the actual "whole point" is, then we should be asking Colin, who created MEDRS in 2006. WhatamIdoing (talk) 02:53, 8 February 2021 (UTC)

What do you think the point of having a topic-area guideline is, then? Of course we want it right, but to do so in this topic certain types of sources have to be avoided or discouraged more than in other topics (articles about TV shows, say). Original creators of pages from years ago don't have special authority. That's like saying if we have a question about regular RS we should ask Beland. These guidelines belong to the community. Crossroads ^-talk- 06:03, 8 February 2021 (UTC)

WP:V is the policy. There has never been and never could be an intent for MEDRS to be a stricter interpretation of WP:V than the policy itself allows for. WP:RS explains how to apply WP:V policy generally. MEDRS explains what RS look like in biomedical content. The idea that MEDRS is “stricter” is a meme that took hold at WP:ANI among editors using it to play whack-a-mole. Not surprisingly, this focus on WAM did not result in better medical content across the board.

And, yes, it is valid to ask those who started and curated the page for years whether they intended to circumvent policy, ever. A good portion of the effort for a year to get the medical guidelines accepted was making sure they did not go beyond existing policy or guideline, rather explain how they are applied to biomedical content. The guidelines would not have been accepted otherwise.

More importantly, if we wanted some sort of stricter interpretation of policy to apply to medical content, that would look something like WP:BLP, where we could shoot on sight any poorly sourced text. I have advocated this for years, to no avail. And it is unlikely we would ever get a BLP-like policy applied to biomedical content if we still have editors broadly misinterpreting MEDRS to whack-a-mole or as an excuse for remove content because IDontLikeIt.

In other words, my conclusion is that the misapplication of MEDRS by WAMers over the years has prevented us from being in position to strengthen the guideline to something akin to BLP, which resulted in, among others, the mess that we have in COVID content. Curiously, at the same time, some of the same people objected to any effort to get a stronger disclaimer on our biomedical content. SandyGeorgia (Talk) 15:22, 8 February 2021 (UTC)

I don't think comparisons to BLP for medical content work. It doesn't take any specialist knowledge to monitor BLP, and the standard of sources required to support a statement about a living person is actually very low in comparison to medical claims. Any respectable newspaper or book published by a respectable publisher will do for the vast majority of people facts. A statement on a respectable charity website is sufficient to associate a LP with a medical condition. All sorts of random sports websites will do if you want to claim someone won a game or a medal, or lost. Did some actor appear in some episode of a TV show? Easy peasy. And if anyone disagrees or challenges, more or less any wikipedian can investigate without engaging too many brain cells.

But just as some website might be fine for TV show credits, that same site would not be fine to claim the actor has been arrested for some crime or another. There isn't a binary "this source good that source bad" that works for all facts. There's a spectrum that goes alongside the kind of claim we are making in our text. It is the same for medical articles. I was going to write "I don't need to cite the Lancet to say how many permanent teeth are in a typical human head" but then I saw that Human tooth#Permanent teeth isn't sourced at all. I'm sure there are better resources than WebMD but The Teeth (Human Anatomy): Diagram, Names, Number, and Conditions verifies that number. If editors removed "per MEDRS" the fact that there are 32 adult permanent teeth because it was sourced to to WebMD, then MEDRS would not be a guideline for very long. -- Colin°^Talk 16:47, 8 February 2021 (UTC)

Just noting that I agree completely with WAID here. The point is to get medical content right, not to put journals on a pedestal. This page is intended to guide folks to reliable sources for medical content. Web resources that are reliable are just fine for our needs. Ajpolino (talk) 03:57, 9 February 2021 (UTC)

Colin, I agree with the point you seem to be making, which to me is that while evaluating sources in biomedical content may be more challenging than other areas, that is a different thing than the oft-repeated meme about "higher standards". But I still think a BLP-style possibility to delete uncited text on sight would be possible. How often do we come across entirely uncited medical content, where going through to rewrite the text to acceptable sources would take more time than one has? In those cases, I believe no text is better than dubious text, and wish we had a BLP-style possibility allowing us to delete uncited biomedical content. Yet, because of past abuse of MEDRS, I doubt we would ever get such a policy; that is one of the many reasons why these memes and abuses of MEDRS need to stop. SandyGeorgia (Talk) 19:12, 9 February 2021 (UTC)

I agree wholeheartedly that "no text is better than dubious text". Regarding "whack-a-mole", though, we need to not be naive. There are a lot of POV pushers, snake oil salesmen, and Dunning-Kruger afflicted and incompetent people who edit these articles and we should be careful not to 'deregulate' the medical topic area. Such editors do need to be reverted and if they persist, blocked. Wikipedia's quality comes just as much from what we don't say as what we do. Constantly dealing with these people is probably why editor attrition is so bad. We cannot make it harder for people familiar with these topics to fight garbage.

Can you please specify what you consider to be "abuses" of MEDRS? Crossroads ^-talk- 19:45, 9 February 2021 (UTC)

SandyGeorgia, you can already remove uncited text from any article on sight. Please only do this if you have some reason to suspect that the content is not verifiable (because the sourcing guidelines are not a game of Mother May I?), and put a link to WP:CHALLENGE in your edit summary. WhatamIdoing (talk) 01:02, 10 February 2021 (UTC)

Isn't there a "Greatest Hits of WP:V" incident regarding whether the human hand has five fingers? Anyone have the link? I think I saw it linked from a userpage when I was new here... Ajpolino (talk) 01:34, 10 February 2021 (UTC)

@Ajpolino, you will find the link in the footnote at Wikipedia:You don't need to cite that the sky is blue#Pedantry, and other didactic arguments. WhatamIdoing (talk) 01:46, 10 February 2021 (UTC)

In other words, I can’t shoot uncited medical text on sight unless I “have some reason to suspect that the content is not verifiable”, which means I have to do the work regardless, which was why I once considered a BLP-like policy allowing us to shoot on sight uncited medical text could be helpful. Instead, I still wish we had a strong disclaimer on every page. At most, I can put a mcn tag on the text, and hope someone else will fix what I don’t have time to fix, and that thousands of readers are not misled meanwhile. SandyGeorgia (Talk) 01:41, 10 February 2021 (UTC)

No, the policy says you can do it for any uncited text on any subject, but as a practical matter, people who over-exercise this 'right' tend to get blocked for WP:POINTy and disruptive behavior. Notice that I didn't say that you needed a good reason to suspect that it's not verifiable; a mere suspicion is often sufficient, especially if the content is not favored on wiki (e.g., anything positive about altmed treatments).

Uncited BLP content can only be removed if it's "contentious". WhatamIdoing (talk) 01:57, 10 February 2021 (UTC)

Editorials and comments

Opening this discussion for 50.201.195.170 who proposes a change to MEDRS's lead (diff):

Original sentence:

Be careful of material published in journals lacking peer review or which reports material mainly in other fields.

New version

Every rigorous scientific journal is at least partly peer reviewed; papers are always reviewed, but editorials and comments may not be. Be careful of material published in journals lacking peer review, or which report material mainly in other fields.

—PaleoNeonate – 17:15, 21 January 2021 (UTC)

I like the idea of giving a word of caution about editorials and comments. "Searching for sources" says a little about letters to the editor, but maybe the point should be brought up sooner. XOR'easter (talk) 19:38, 21 January 2021 (UTC)

I think the complete diff is between the current:

Every rigorous scientific journal is peer reviewed. Be careful of material published in journals lacking peer review or which reports material mainly in other fields.

and the proposed:

Every rigorous scientific journal is at least partly peer reviewed; papers are always reviewed, but editorials and comments may not be. Be careful of material published in journals lacking peer review, or which report material mainly in other fields.

I'm fairly neutral about the change to the first sentence: is it WP:CREEP or is it a useful qualification? The change to the second sentence is indisputably accurate: the subject of the verb "to report" is "journals" (through a relative pronoun), and that should take the plural verb form, "report", not the singular form "reports". --RexxS (talk) 19:49, 21 January 2021 (UTC)

Thanks for fixing my omission, —PaleoNeonate – 17:15, 22 January 2021 (UTC)

The new text does add some information that may be useful but it contains redundancy. It sets up the "at least partially peer reviewed" and then repeats this in a more specific way with examples. This could be rewritten in a more straightforward manner, without the unnecessary and unsatisfactory tension of "partially": get to the point. However, before doing so, it may be worth those with experience reading these journals in the paper form (rather than just random papers found on pubmed) considering if the description here is complete or accurate. My understanding is that editorials are never peer reviewed and are the opinion of the editor(s) and at very most (if unsigned) other very senior partners in the publication, and certainly nobody outside of the publication or anyone independent. Is "comments" clear? Are there other sections in a journal that are separate from the serious work (light-hearted posts, news or politics, and matters concerning the functioning of a membership body, etc). Is there a better way to describe those sections that actually are peer reviewed. -- Colin°^Talk 08:53, 22 January 2021 (UTC)

Reputable journals have Editorial Boards. Editorials, leading articles, commentaries and similar pieces will be reviewed by other members of the editorial team and often go for external review. The decision to publish or not will always ultimately rest with the editors, based on the reviews obtained from expert reviewers. I think it will be hard to be dogmatic about specific types of article and am not sure this is needed. Additionally, the rigor will be different in different journals and it can be difficult to know which are the less reputable journals. Jrfw51 (talk) 15:05, 22 January 2021 (UTC)

Two Editorial examples from BMJ. Asymptomatic transmission of covid-19 is "Not Commissioned; externally peer reviewed". NICE guideline on long covid is "Commissioned, not peer reviewed." Like many editorials, both are opinion pieces rather than attempts to neutrally describe research results or consensus (vs commenting on other research / consensus statements or the need for them). -- Colin°^Talk 16:02, 22 January 2021 (UTC)

I'm not sure the second statement is quite accurate. I think it's confusing to say the journal is peer reviewed rather than the papers published in the journal are peer reviewed. If we start off with a slight inaccuracy we end up having to explain. Is a journal "partly peer reviewed" or are parts of a journal peer reviewed while parts are not. What is rigorous exactly? The less we leave to interpretation the better? If we say something like, "Wikipedia prefers journals in which papers are peer reviewed, but be aware that while papers in any given journal may be peer reviewed, editorials and comments may not be." seems simpler to me and less open to misunderstanding and interpretation. I also think the first statement was fine as is.

I think 50.201.195.170 should have opened this discussion on the talk page; he would have learned more, but it was kind of PaleoNeonate to open this for him/her. I could have also opened the discussion on the talk page myself and been more patient with the situation. Littleolive oil (talk) 17:12, 22 January 2021 (UTC)

They still merited a PA warning. —PaleoNeonate – 17:15, 22 January 2021 (UTC)

Yes. Thank you. I agree. There's learning there too. Littleolive oil (talk) 17:23, 22 January 2021 (UTC)

I like Wikipedia prefers journals in which papers are peer reviewed, but be aware that while papers in any given journal may be peer reviewed, editorials and comments may not be. However, I think prefers is a bit too weak; our reliance upon peer review is not a matter of taste like how we each take our tea. Likewise, Be careful of material is good advice for journals which report material mainly in other fields, but not quite emphatic enough for journals lacking peer review. XOR'easter (talk) 18:20, 22 January 2021 (UTC)

I agree prefer is too weak, unless talking about weak tea. I wasn't sure what word to use and since my post was a suggestion I'd hoped someone else might have a better word. Littleolive oil (talk) 19:34, 22 January 2021 (UTC)

requires --RexxS (talk) 19:50, 22 January 2021 (UTC)

Yes. Right word. Littleolive oil (talk) 21:02, 22 January 2021 (UTC)

• Is this an improvement over what we have now or over other suggestions? "Wikipedia requires journals in which papers are peer reviewed, but be aware that while papers in any given journal may be peer reviewed, editorials and comments may not." Littleolive oil (talk) 21:30, 22 January 2021 (UTC)

• Perhaps something like:

Every rigorous scientific journal requires external peer review before publication of primary research papers and review articles, but may choose to adopt an internal review procedure for vetting editorial commentary materials. Research and review materials published without peer review are generally unsuitable for use in Wikipedia. Opinions expressed in reputable journals where peer review has occurred should generally be afforded greater weight than materials where the standard of review is unclear, but the latter may be suitable sources if used with care. Editors are also advised to be cautious when considering referencing a peer-reviewed paper which is published in a journal not known for reporting on that topic area or academic discipline.

I recognise that it is longer but (to me) it catches the issues and leaves room for considered judgement in individual cases. On the last point, I am sure there are medical examples, but a classic non-medical one was the publication of cold fusion in the Journal of Electroanalytical Chemistry, a journal with little background or reputation for examining claims about novel nuclear phenomena. This might be useful to add. EdChem (talk) 05:30, 23 January 2021 (UTC)

I guess all of this means I will never be able to finish user:SandyGeorgia/sandbox3 for habit cough where the core dilemma and core publications are opinion pieces from the consensus guideline authors, explaining why they are at odds with each other. Which will leave us with a less-than-accurate article. The best writing on the topic, by the top experts in the field, explaining the two different points of view are opinion pieces. SandyGeorgia (Talk) 05:55, 23 January 2021 (UTC)

No Wikipedia policy or guideline is absolute. There is no consensus to make a change to what is in place now. Suggestions are in response to a suggested change not to what is in place now. I, I can't speak for anyone else, am fine with what is in place now but not to the IP's proposed change posted at the top of the thread. I can't speak for anyone else, but again there is no agreement to even make a change. If something doesn't sit well I'd suggest adding it to the discussion. This seems to partly clarify and may cover your concerns." Opinions expressed in reputable journals where peer review has occurred should generally be afforded greater weight than materials where the standard of review is unclear, but the latter may be suitable sources if used with care." (Ed Chem above) Anyway, nothing is happening here with out agreement. My opinion anyway. Littleolive oil (talk) 18:15, 23 January 2021 (UTC)

I'm unclear on where the whole discussion stands, but the word required in a guideline always makes me uncomfortable. We have very few requirements on Wikipedia (eg, BLP issues). SandyGeorgia (Talk) 18:21, 23 January 2021 (UTC)

I think EdChem's version is good for that reason. He uses requires to identify the kind of journal we are more likely to want to use, but not to refer to how we use those journals on Wikipedia."Every rigorous scientific journal requires external peer review..." I think the discussion here is just a series of suggestions as a response to an IP and as such the discussion doesn't stand at all (but seems to be sitting in the corner watching the goings on). Unless someone seriously wants a change and takes control the discussion may go nowhere. I'm just watching to see what happens. There was no concern until the IP tried to make some changes so maybe the version in place is just fine as is. Littleolive oil (talk) 18:43, 23 January 2021 (UTC)

Sandy, I don't see why my suggested text precludes the sort of edits you are describing. I deliberately chose phrases like "generally unsuitable" for non-peer reviewed publications and that opinion / editorial materials with only internal review "may be suitable" specifically because guidelines that are absolute can prove problematic in an unanticipated / atypical situation. I was definite that rigorous scientific journals require peer review prior to publication. There could be some exceptions in dealing with rapid reports but those sorts of sources will be replaceable with review papers (ultimately) if the findings prove to be notable.

I agree with Littleolive oil that my suggestion is exactly that – a suggestion that I will certainly not put into the text of the guideline without consensus. I saw the discussion of the changes and thought that some change seemed reasonable but that valid concerns have also been expressed, so I offered a contribution that I think is an improvement that also takes into account the concerns that have been raised. I offered it in good faith and recognise that no consensus may be reached or that a consensus that no change is needed may develop – and, of course, I will accept whatever consensus is reached.

I am uncomfortable feeling that my suggestion would impede your editing or lead to "less-than-accurate" articles. Please accept that that was not my intent. I believe that you have the respect of many editors and any changes / suggestions that you offer or concerns that you raise will be taken into account. I have no problem if my suggestion does not find support. EdChem (talk) 22:58, 23 January 2021 (UTC)

Thanks, EdChem; I need to confess that because of ... other issues elsewhere ... I haven't paid close enough attention to this discussion, and that makes me feel much better! I think habit cough may be the only time I have encountered editorials that need to be accounted for, so it's a rare exception, but I just wanted to be sure we are allowing for those. SandyGeorgia (Talk) 23:08, 23 January 2021 (UTC)

Editorials have been used in some journals recently to provide an up-to-date summary of some COVID-19 information. We are often trying to find the best of what's left after excluding obvious original research (=the kind of OR that researchers do in their labs, not the kind of OR that editors oughtn't do in Wikipedia articles), and sometimes a recent editorial is one of the best for a given area.

Related to earlier comments, we have had problems in the past with people trying to push non-peer-reviewed letters to the editor, usually to contradict a MEDRS-style source that the editor disagrees with.

We should note, at least in this discussion, that the labels can trip people up, because in some journals, the traditional style for a standard primary source describing research is to call it a "Letter", a "Comment" isn't always a letter to the editor, etc. Also, there are some that I have no idea how to classify. Blood (journal) has run a series called "How I Treat..." (example) that is sort of halfway between a narrative review and an opinion piece. WhatamIdoing (talk) 02:38, 24 January 2021 (UTC)

Perhaps we should add some language along the lines of, Not all journals label the categories of items they publish in the same way? XOR'easter (talk) 16:54, 24 January 2021 (UTC)

We could, but would it be WP:CREEPy? This guideline is already pretty long. WhatamIdoing (talk) 03:33, 25 January 2021 (UTC)

Good point. XOR'easter (talk) 15:40, 25 January 2021 (UTC)

I don't like "Wikipedia prefers " or "Wikipedia requires " or "are generally unsuitable for use in Wikipedia". The scope of such statements is outrageously wide and MEDRS would never have been adopted if we wrote that way. It is better that we describe the attributes that make a source better and the attributes that make it worse, and be cautious about doing the pro-vs-con conclusions. Conclusions made without source->text context are often too general and can be over-restrictive. As Sandy notes above about describing a lack of consensus in a field, there are problems with being too rigid. Further, I think many editors concentrate too much on peer-reviewed-journal-papers as though that is all there is to sourcing.

Guidelines should avoid the " if used with care" get-out-of-jail-free card, which permits editors (who? me? I'm always careful!) to ignore recommendations as they see fit. Text like "Editors are also advised to " is redundant.

I'm concerned this is becoming over-complicated and we are once again being bogged down discussing contentious changes made by some ip or newbie, rather than having experienced editors make suggestions based on their editing experience. -- Colin°^Talk 18:48, 24 January 2021 (UTC)

Oh, I think it's fine. No one that I'm aware of is pushing anything here. It's just a discussion without heat or vitriol. While the IP was looking for a change most everyone else here is experienced and making suggestions from that experience. No one is demanding change. If this fades away with out change I'm not bothered at all. Sometimes a bit of discussion allows us to realize what we have already is fine. So no worries from my view anyway. Littleolive oil (talk) 21:05, 24 January 2021 (UTC)

I agree that participants here are experienced, but perhaps this particular text isn't so terribly wrong that it is worth all the thought going into it. Actually the other recent time was on MEDMOS, and rather than let the discussion fade naturally, someone turned up threatening RFCs and guideline revocation and other inflammatory consequences. What is useful is the information posted here about the different parts of a journal and the sometimes confusing language used to describe it. Letters can be papers and reviews need reviewed. Maybe this will encourage someone to clarify the guideline text in other ways or other places. -- Colin°^Talk 21:54, 24 January 2021 (UTC)

The Ip who started the discussion was actually objecting to the sentence "Every rigorous scientific journal is peer reviewed." which they deemed "needed factual correction". It has since evolved into a discussion about what we could say in place of that contested sentence. I find it healthy to visit these sort of concerns on occasion, as it's always possible to improve content, even guidelines. I do feel it perfectly reasonable to tell editors that Wikipedia prefers peer-reviewed journals in general, and that MEDRS requires journals to be peer-reviewed as a minimum standard. There may of course be rare exceptions, but that's understood as being in the nature of guidelines. --RexxS (talk) 22:06, 24 January 2021 (UTC)

This is an important distinction. " Wikipedia prefers peer-reviewed journals in general, and that MEDRS requires journals to be peer-reviewed as a minimum standard." There are certain aspects of Wikipedia that will require discussion despite what policies and guidelines say. Wikipedia was never meant to be governed by rigid controls but rather by editing guidance even for policies, as I understand it. There are so called bottom lines like MEDRS peer review, and given what I've seen in many articles this is a legitimate base line for sourcing. A base line should not be viewed as required but rather as a common-sense beginning. And always if there are exceptions then discussion and consensus are the next step. Thoughts. Littleolive oil (talk) 22:20, 24 January 2021 (UTC)

In the discussion I was responding to, Littleolive oil wrote "Wikipedia prefers" and felt "prefers" was too weak and RexxS suggested "requires". The "important distinction" now being claimed was not made, and I still object to either claims made in this guideline. We do not speak on behalf of "Wikipedia" and that isn't how guidelines are written as though "Wikipedia" was a god or boss. I don't think any guideline should "require" any form of publication quality-assurance methods. Peer review is but one, and is well known to have flaws, and is just one way we can gain confidence that a publication has "a reputation for fact-checking and accuracy". -- Colin°^Talk 11:17, 25 January 2021 (UTC)

Of course the distinction was made. When we agree policies and guidelines, we do speak in Wikipedia's voice, and that's exactly how we write our guidelines. There is no shadow of doubt that the vast majority of editors prefer peer-reviewed journals to non-peer-reviewed ones as sources, and you're wrong to suggest that requiring something as simple as peer-review for journals acting as MEDRS sources is not a accurate expression of the consensus amongst editors. There is absolutely nothing wrong with having requirements for sourcing, just as we have requirements for notability, verifiability and other core principles. --RexxS (talk) 21:59, 25 January 2021 (UTC)

Exactly. We absolutely do prefer peer-reviewed sources over almost all non-peer-reviewed sources per WP:SOURCETYPES. And that is all the more important and true in medical topics. Crossroads ^-talk- 22:09, 25 January 2021 (UTC)

That is not what SOURCETYPES says. SOURCETYPES says "academic and peer-reviewed publications, scholarly monographs, and textbooks are usually the most reliable sources". By my count, that's four different types of preferred publications, and only one of them is peer-reviewed. Perhaps to a more practical point, is anyone aware of any medical journal that isn't at least nominally peer-reviewed these days? Even Medical Hypotheses has a sort of weak peer review these days (although they don't actually publish any real research). If non-peer-reviewed medical journals don't really exist, then we don't need to harp on this point. WhatamIdoing (talk) 20:50, 5 February 2021 (UTC)

It happens that a discussion at Talk:Circumcision and HIV#Criticism of African Studies III / Risk Section includes a proposal to cite some "Comments". If anyone happens to feel like exploring that subject a little further, then it might be a handy case study. WhatamIdoing (talk) 03:32, 25 January 2021 (UTC)

I'm of general agreement that an expansion of "peer review" would be useful - and I personally think it should be the first piece of basic advice given. I propose a paragraph similar to the following be added as advice on peer review (and duplicative information removed from the scientific consensus section):

• Reputable scientific journals carry out a process called peer review, which consists of rigorous verification of the validity of the science, methods, data, and other aspects of a paper to be published in the journal. However, many reputable journals also contain other articles that are not peer reviewed, but are included for other reasons such as to provide commentary/response to other peer-reviewed articles, or to provide an individual or group's opinion on a topic. While such material may be published in a reputable peer-reviewed journal, it is usually subject only at most to basic fact-checking and ensuring that it is not so far opposite of mainstream view to constitute pseudoscience or a fringe viewpoint. Such non-peer-reviewed parts of journals can commonly be identified by a header/section/title which includes words such as commentary, letters (to the editor), opinion, response, commentary or similar. Alternatively, many journals explicitly identify the peer-review status of a published article either in the information before the article, or at the end of the article, such as a footnote which states Provenance and peer review: Commissioned, not peer reviewed. Be wary of sources for which there is no explicit peer-review status declared - extra care should be taken to attempt to ascertain whether the article was peer-reviewed, and it may be prudent to ask other editors more familiar with the field or journal for their input. Sources which have been fully peer-reviewed are preferred over other sources, and the strength of non-peer-reviewed parts of journals is almost always insufficient to support biomedical claims.

Thoughts? -bɜ:ʳkənhɪmez (User/say hi!) 04:12, 25 January 2021 (UTC)

I'll bite: how do you verify someone else's data? WhatamIdoing (talk) 04:20, 25 January 2021 (UTC)

Verification of data (at least when I say it) generally means the process of peer-review which looks for any signs the data may have been altered, fabricated, tampered with, or is incomplete or manipulated in a way to not show the full picture. If there's a better word to use, or if you think that's too complicated and should be removed, that's fine :) -bɜ:ʳkənhɪmez (User/say hi!) 04:23, 25 January 2021 (UTC)

I'm still concerned we are over-cooking this and being over-rigid. We need to be extremely cautious about adding new hurdles when our minds are fixed only on some common cases. In the book I am reading, this is called "What You See Is All There Is" -- we find it hard to visualise other cases where different sources are perfectly valid and indeed desirable. We now have a proposed burden that editors must seek out and locate "explicit peer-review status". Really? You do that for every paper you cite? And non-peer reviewed parts are "almost always insufficient". Really?

Ok, how about Optimal clinical management of children receiving dietary therapies for epilepsy: Updated recommendations of the International Ketogenic Diet Study Group. This is probably the very best source for ketogenic diet. I can find no place on the page or PDF where it says that "Special Report" was peer reviewed. It was written by 30+ authors along with the US and UK charities. Those authors are names I am familiar with from the literature and no doubt are the ones who peer review papers in this field. It is a consensus statement from three dozen of the finest names in the field. Shall I remove it because I can't find a peer-review statement? -- Colin°^Talk 11:17, 25 January 2021 (UTC)

I don’t see how providing information/clarification/guidance for analyzing sources is “over-rigid” - this is already part of the spirit of the guidelines and clarifying it should be attempted to avoid issues. And you’ll notice that nowhere does it say “you can’t use a source that doesn’t explicitly state peer review status” - but if you think a statement about “consensus guidelines” being practically equivalent to peer review, go right ahead. My suggestion was designed to be a starting point, not a final addition. -bɜ:ʳkənhɪmez (User/say hi!) 15:48, 25 January 2021 (UTC)

Berchanhimez, you weren't just "providing information". You were asking editors to find statements that may not actually be widely available and to "be wary" if they can't find them, and you were restricting sources with the "is almost always insufficient to support biomedical claims". -- Colin°^Talk 19:38, 25 January 2021 (UTC)

@Colin: This is what I am, at least, doing on this page. This is only a discussion. My original suggestion with it's weak "prefers" was discarded for a stronger word "requires". I'm not attached to my suggestions or anyone else's and I just let that whole phrase go when you said you didn't like requires or prefers as outrageously wide. Rexx brought up another point in terms of the original IP's suggestion also using the word prefers and requires. I agreed with that distinction. My comments are simple discussion points, brainstorming if you will. I also like what Ed Chem suggested. I was the editor the IP was fighting with originally. I was tired, the IP wasn't pleasant and I didn't want to deal with changes that seemed problematic. I could have dealt with the changes on the talk page, but didn't have the patience to do so. The discussion was brought here and I am happy to engage here.

My view is that we have an Ignore All Rules policy which can override most other policies and which turns back the control of sourcing to the editors in any given, individual circumstance. In my mind, a word like "requires" is in place to deter use of non-reliable sources. In MEDRS non-reliable could be peer-reviewed, primary studies. I've looked at articles which were packed with random studies, never replicated, as proof for claims. I think that's what we don't want. "Requires" turns us away from that kind of situation especially important for newer editors. If we find a source that seems important to use we have a policy that allows for that without compromising all health related articles. A special report written by 30 authors along with UK and US charities is by its nature reviewed by peers. Peer reviewed simply means oversight by peers and this report has that kind oversight, in fact, I would consider it a secondary source. If I am looking at a journal in terms of reputation and impact factor I don't look for some kind of peer review notice. The journal by its nature is reviewed, I may still check on the editorial board for the journal. If I am looking at a study in a low impact, little known journal I may look for a peer review notice. That journal in my mind is still questionable so I would tend to bring it to other editors for input and agreement for use. That's the collaboration process. What I thought was going on here was the beginnings of collaboration, editors freely making suggestions putting out thoughts and ideas until the discussion begins to coalesce around something. I'm not attached to anything here, and given the importance of the health related articles on Wikipedia and their ability to do damage this seems like a good exercise to me. (If I had my way health related articles would be worked on by experts and then frozen to stabilize the information to maximize safety; that's not a discussion for now, though.) Littleolive oil (talk) 16:01, 25 January 2021 (UTC)

(edit conflict × 2) It's quite possible to get some idea of whether data is valid, for example by applying statistical tests to check the randomness of a supposed random sample. There's a recent example at Gam-COVID-Vac #Allegedly fabricated data of phase 1/2, where data was queried by external reviewers, and the original researchers had to publish an explanation.

MEDRS advises that ideal sources include "guidelines or position statements from national or international expert bodies". It is unlikely that an expert body would explicitly submit its statements to external peer-review. The assumption we make is that the experts who make up the expert body will have a good overview of the literature, and are able to produce secondary sources whose quality is equivalent to our normal expectations for peer-reviewed secondary sources found in reputable journals. It is a strawman to posit that "expert bodies such as the Ketogenic Diet Study Group don't produce peer-reviewed sources, therefore we don't need peer review for sources in journals". It is perfectly normal – in fact desirable – that we impose the burden on editors to seek out journal articles which are peer-reviewed as a minimum. It is not helpful to confuse that with accepting the statements of expert bodies or of expert-written text books, neither of which have an explicit "peer-review statement". --RexxS (talk) 16:05, 25 January 2021 (UTC)

RexxS, it helps not to invent nonsense and put it in quotes and attribute it to another person you disagree with.

That special report was not, as far as I can tell, peer reviewed, and yet we had proposed text saying that such an article was "almost always insufficient to support biomedical claims". I have demonstrated that that is not only wrong but unhelpful, as in fact, some of our finest sources are not peer reviewed. A peer review is a review by someone other than the author(s) who is their peer in the field. We all agree that forming and publishing a consensus of experts is a great way to ensure high quality science, but that is actually a different process than someone or some group writing about the study they did or performing a systematic review, and then having different people peer review the finish draft of what they intend to publish. Peer review is not "a minimum". It is just a common and useful process to improve articles in journals. -- Colin°^Talk 19:38, 25 January 2021 (UTC)

It also helps if you don't write nonsense and then double-down when called out on it. What was the point of bringing up the Ketogenic Diet source if not to attempt to discredit the notion that journals used as sources need to be peer-reviewed as a minimum? Of course that is the game you were playing.

The "special report" was quite clearly what we would consider "guidelines or position statements from national or international expert bodies", where we have never expected peer-review for the reasons I clearly stated above. What's your difficulty with reading what's been written? How can you state that the International Ketogenic Diet Study Group consists of "three dozen of the finest names in the field" and then try to pretend that it's not an international expert body? Nobody here is suggesting that "guidelines or position statements from national or international expert bodies" should have to be peer-reviewed, and MEDRS has never required that. But MEDRS does require peer-reviewed journals as sources of "review articles (especially systematic reviews) published in reputable medical journals" and your denial of that clear guidance is becoming tendentious. --RexxS (talk) 22:15, 25 January 2021 (UTC)

A consensus guideline is by definition peer reviewed, and that’s what RexxS was trying to explain to you. A peer review does not necessarily mean that it was other than the authors - in the case of a consensus guideline many of the “authors” don’t even write it. That’s not to mention that I never once said an article that doesn’t explicitly claim peer review is not peer reviewed - in fact, I said that editors should examine it and if in doubt ask others. And almost all consensuses are published in journals and peer reviewed anyway, so your point is moot. Why are you pushing so hard against this attempt to help explain to new or non-medical editors what peer review is? -bɜ:ʳkənhɪmez (User/say hi!) 19:47, 25 January 2021 (UTC)

If the only thing your proposal did was explain what peer review is, and why it is helpful, I wouldn't have a problem with it. Let me quote the bits of the proposal that I have problem with.

• "Such non-peer-reviewed parts of journals can commonly be identified by ..." I'm not convinced this is actually usefully true, nor that nearly any of us look for such a thing when writing articles.
• "Be wary of sources for which there is no explicit peer-review status declared - extra care should be taken to attempt to ascertain whether the article was peer-reviewed, and it may be prudent to ask other editors more familiar with the field or journal for their input." This is a novel requirement imposed on editors. I can't understand why it is all controversial for me to argue against imposing requirements on editors which we don't follow ourselves. That opens us to a charge of hypocrisy.
• "Sources which have been fully peer-reviewed are preferred over other sources" This simply isn't true. Sources? All sources? Anywhere. On anything?
• "the strength of non-peer-reviewed parts of journals is almost always insufficient to support biomedical claims." Again not true, as my consensus statement demonstrates.

Let's examine what peer review is.

• "Peer review: The process used by medical journals to screen articles that are submitted for publication. Peer-reviewed articles are scrutinized by members of the biomedical community before publication."https://www.medicinenet.com/peer_review/definition.htm
• "Review by peers is used by editors in the evaluation of articles and other papers submitted for publication."http://www.medicaldictionaryweb.com/Peer+Review-definition/
• "Process of evaluating research proposals, manuscripts submitted for publication, and abstracts submitted for presentation at a scientific meeting, whereby these are judged for technical and scientific merit by other scientists in the same field."https://medical-dictionary.thefreedictionary.com/Peer-review
• "Evaluation of scientific, academic, or professional work by others working in the same field."https://www.lexico.com/en/definition/peer_review
• "the evaluation by fellow specialists of research that someone has done in order to assess its suitability for publication or further development"https://www.collinsdictionary.com/dictionary/english/peer-review

I'm sure that consensus statement went through drafts and some "authors" were more involved in the writing than others, but not all reviewing is "peer review". You need "others" for that, and you need a pre-publication draft that has been submitted to an editor who appoints independent reviewers. That's what I think most people think of peer review, and the dictionary seems to agree with me.

I'm looking for MEDRS to guide editors when writing great articles, not to tie their hands or give them a bum steer. I know some people just want a big hammer to hit other editors. I find that WP:WEIGHT is a heavy hammer. Our featured articles make great use of consensus statements, and textbooks. Let's not confuse people with eccentric ideas of what "peer review" is. -- Colin°^Talk 21:32, 25 January 2021 (UTC)

• I support adding something which emphasizes the superiority of peer-reviewed sources over most other kinds of sources and which notes that some parts of journals are not peer-reviewed. Such non-peer-reviewed portions can lack accuracy and/or be the undue opinion of the author or journal editor. Some can even be rather odd as a result; e.g.: [3] Crossroads ^-talk- 22:19, 25 January 2021 (UTC)

(edit conflict) Yes, Colin, we know you're looking to undermine editors who make it difficult for fresh editors to cram articles with biomedical statements supported by primary studies. They need to have their wings clipped and they need to be given an accurate steer on what the agreed sourcing requirements are. You do nobody any favours by letting a new editor add reams of weak sources and unsupportable claims. Of course, if you're only working on featured articles, you're not going to see the amount of crap that most medical editors have to clean up on controversial and novel topics, but that doesn't excuse your failure to appreciate the work that editors like Berchanhimez are trying to do. "Your old road is rapidly aging. Please get out of the new one if you can't lend your hand ..." --RexxS (talk) 22:30, 25 January 2021 (UTC)

These are my opinions based on my 13 or so years here dealing with health-related/biomedical articles. Peer review is used and understood by most editors with even a minimum amount of experience to be a base-line for those health-related sources. And reference to sources in this thread relates to health related sources only; that's the context which I assumed we all understood. If newer editors are not taught they can destroy the articles they are working on. Teaching and learning is a product of both actions by the teacher and the student. The student must be receptive, the teacher able to direct the student in a productive direction. We don't improve Wikipedia articles by softening the policies and guidelines to make it easier for new editors to edit. We guide the new editor and hope they are receptive. Some aren't. The IP I was dealing with whose actions led to this discussion is not and he is wearing out the admin dealing with him. The way we edit MEDRS articles is especially important; people do come to those articles for health-related information. We can't toss in sources that have not undergone the vetting that comes with at least peer review and at best systematic reviews and other secondary sources. No editor comes to Wikipedia fully formed, and no new editor should think they can understand Wikipedia in a short time. We do them a disservice if we don't steer them in the direction which will both make them both good editors and able to write good articles because they understand the policies. They must respect the work and the time it takes to learn. In my early days I dealt with editors who were deliberately trying to trap editors, these were not pleasant people do deal with. Those people are gone. Just my opinions. This discussion has derailed. Maybe it should be closed. Littleolive oil (talk) 01:03, 26 January 2021 (UTC)

Littleolive oil, I agree with you - this is solely related to MEDRS and not sources in general. And keep in mind that this WP page is intended as a 'guide' for new editors - meaning of course we should explain peer-review, how to identify when it occurred/not, etc. That's why I proposed not considering minor changes of one sentence or less - but let's just add a section about peer review. I never expected it to be perfect at the time I proposed it - in fact Colin brought up at least one good point in that it's likely going to help if we include an explicit statement regarding consensus guidelines not being technically "peer reviewed" sometimes, but still being subject to the rigor expected of secondary MEDRS. He also brought up that "be wary" may be too strong - maybe "look deeper" or similar is better - but the point there is that if it doesn't explicitly say it's peer reviewed, a WP editor shouldn't assume that it is until they look into it. Unfortunately, Colin also brought up a lot of his opinions. I encourage Colin to show me one section of a journal that is titled similar to my examples that is subject to the same peer-review as the main part of such journal. I also encourage Colin to show me an example of a consensus statement that's published in a journal yet is published in a commentary/opinion section. He can't - I'll tell you that now. I don't know why he is so adamantly against improving my proposal, but I welcome any comments from him or others (pinging @RexxS:, SandyGeorgia <not pinged per her request on discussion she is involved in>) as to if my proposal is remotely workable, and working towards getting it in a state it can be added. Ideally, this would result in a WP:PEERREVIEW shortcut - which would lead to this explanation of what peer review is, why it's virtually required for MEDRS, etc. - this shortcut would be a quick way to direct new editors or those who don't understand medical sourcing on WP to an explanation. -bɜ:ʳkənhɪmez (User/say hi!) 01:58, 26 January 2021 (UTC)

User:Berchanhimez, I don't want to be in conflict with you. I am going to ignore RexxS's personal attacks. We can't redefine what "peer review" means any more than we can redefine "folk music". Folk equals people. Iron Maiden are people. Iron Maiden play music. People listen to Iron Maiden's music. Therefore Iron Maiden play folk music. That's nonsense of course. Terms have meanings that writers/speakers of the language give them that are separate from what the component words mean. Do we really want to make battling some POV warrior harder by having to explain "Well, when MEDRS said 'peer review', what we mean is ....".

The reason I don't want to "improve" the suggested text by offering my own is that I am out of my depth here. I know enough to spot when something is wrong but not enough to write an alternative definition or explanation. I'd rather someone experienced in the academic publication process did that, or at the very least, could cite an up-to-date source for what they say, rather than perhaps speculating about what level of checks a journal may or may not do for particular kinds of articles.

But one area I'm not out of my depth in is guidelines, and the propose text (and other earlier proposals) are simply non-starters when it comes to guideline style and the mistakes that come from DOs and DON'Ts that are well meaning but wrong. Saying something like "Sources which have been fully peer-reviewed are preferred over other sources" is just going to bite us when the POV pusher with their peer reviewed crappy crank narrative review in a predatory journal is up against your statement from the World Health Organisation or the CDC website.

Peer review is simply one quality control mechanism but it is simply part of how the sausage is made. None of us here check to see if a review paper has been peer reviewed when selecting it as a source for what we are writing. Instead, I assume a review paper published in BigNameRespectedJournal is peer reviewed. I have no idea why some of the editorials in the BMJ are peer reviewed and some are not. I would be very cautious about assuming the peer reviewed editorials are superior to those that aren't -- it may be they were selected for peer review precisely because the publisher/editor was nervous about the author(s) vs a well known and trusted writer. The most important factor about those articles is not whether they were PR'd or not, but that they are editorials: opinion pieces given a high platform by the publication.

Berchanhimez, I'm not ignoring your two "show me " requests, but at this point I think they are more a symptom of two people talking past each other, as I don't have the slightest clue how they relate to my argument.

I think if this discussion is to progress at all towards better guidance about what peer review is and why it is good, and sensible practical advice about "identifying reliable sources in medicine", then we need other experienced people to contribute to this and offer some new original ideas about what to write. -- Colin°^Talk 08:52, 26 January 2021 (UTC)

User:Berchanhimez. I do think an addendum that explains peer review would be a good idea. As well, some of the understanding an editor has in terms of editing in any area comes from well, editing, while some of the understanding comes from education. All editors are not going to be well-educated in the science or science related topics. It's not easy to make experts out of non-expert editors. I'd add that there is a tendency on Wikipedia given the anyone can edit standard that everyone should edit everything all the time. Why would I consider myself an expert in understanding Unified Field Theory when my knowledge in real life is superficial? I've edited art-related articles and argued the simplest point any artist knows and in the end had to walk away knowing the article was short on this information. I feel this conversation has become overly complex and convoluted. As with any source, in any field, my sense is that we are providing guidelines not trying to educate every editor in a way generally they only might get in an education in that field. I don't think we can. I'm not sure trying to explain every possibility in sourcing is useful or even possible which is what seems to be happening here. I have a tendency to look for overarching principles, that's what I look for now. An addendum could lay out the basics in terms of peer review and secondary sources and with a few example would be enough. These are thoughts. Berchanhimez this is not directed at you so much as to the group.

User:Colin.This, "None of us here check to see if a review paper has been peer reviewed when selecting it as a source for what we are writing." Yes I do actually, and in discussion the editors I deal with do too, and if needed investigate further to see if the source can be considered secondary. This kind of check is absolutely necessary. Littleolive oil (talk) 14:40, 26 January 2021 (UTC)

It would greatly help me understand the scenarios of when medical journals publish un-peer-reviewed reviews if you could give us some examples of those you have rejected after checking to see. Some examples of Wikipedia article discussions on this would also be useful. -- Colin°^Talk 15:12, 26 January 2021 (UTC)

Colin you are asking me to dig through 13 years of editing where in any given health-related article every source would be checked as a RS. I could have never used a source that was not, and I would have no idea now where those discarded sources would be. For someone working in health related areas, especially contentious areas, checking for peer- review as a first step is as common as writing the article or copy editing. Listen, I want to be helpful but I can't take time to do this; it's an enormous task and my Wikipedia time is limited. Good medical journals, that is those with a good reputation, good impact factor are peer- reviewed. It's a standard practice to have research reviewed. For the most part, the editors I've dealt with understand peer-review and in fact peer-review is not the standard I and we are looking for; its secondary sources. If you look for predatory journals (Google it) you may find what you're looking for. Like wise, I have been editing for a long time with much of that time spent on health-related articles. Discussion on sourcing seldom focuses on peer review is a base-line standard, discussion especially after the first few years would have centered on secondary sources. I can't dig through 13 years of discussion to find what you're asking for. I'm sorry. I can't take the time to do this for you. Littleolive oil (talk) 15:41, 26 January 2021 (UTC)

I've spent a while today reading about PR by searching for the topic on PubMed. There is a lot of literature about PR. So far I have found nowhere that mentions a problem with medical journals that don't PR their papers. Being a "peer-reviewed journal" appears to be a critical attribute and your publication won't be indexed and will be considered worthless. I could be wrong, but that is what I have found. There do appear to be a few dodgy journals that claim to PR and don't bother at all. And there does seem to be (or been) a problem with authors recommending reviewers when submitting, and those reviewers turning out to be fake email addresses that went back to the author or his mates. And there is lots of literature about how flawed the process is, and one former editor of the BMJ openly suggesting it may not in fact be desirable at all, with the effort rarely delivering the quality assurance one might hope, but certainly delivering all the biases one might not hope.

So I continue to be perplexed why MEDRS should recommend to anyone to first check the systematic or narrative review they found on PubMed has in fact been peer reviewed. I'm also sceptical that it is generally possible per paper. I clicked on the first citation in ketogenic diet: this paper and am struggling to find where it says it is PR'd. I can find that the journal describes itself: "Pediatrics is the official peer-reviewed journal of the American Academy of Pediatrics" but the complaint that not everything in a peer-reviewed journal is peer-reviewed is what started this whole discussion. -- Colin°^Talk 16:19, 26 January 2021 (UTC)

I just do not understand what the issue is. This discussion was not started by, "but the complaint that not everything in a peer-reviewed journal is peer-reviewed." It was started to satisfy an IP who had added content without consensus. At the same time, reputable journals do say they are peer reviewed, JAMA for example,"JAMA, published continuously since 1883, is an international peer-reviewed general medical journal. JAMA is a member of the JAMA Network, a consortium of peer-reviewed, general medical and specialty publications." In their Critical Objectives, "...To publish original, important, valid, peer-reviewed articles on a diverse range of medical topics." and on the same page, "JAMA is a member of the International Committee of Medical Journal Editors, cosponsors the International Congress on Peer Review and Biomedical Publication,..." If Jama believes its necessary to reiterate in three different ways the importance of declaring Peer-Review then I don't see why we shouldn't. And while PR review in " rigorous " journals might be a given the number of editors who try to add content that fails any kind of review by any kind of expert is hefty. Littleolive oil (talk) 19:23, 26 January 2021 (UTC)

I agree with Colin that neither a position statement from a body like the WHO or the ketogenic content to which he referred is likely to be peer reviewed. I further agree that deprecating such a source for that reason would be ridiculous. However, I disagree that the suggestion that I made should be taken as inconsistent with using such a source. The reason is that what I wrote was about scientific / medical journals and nothing else. It did not refer to non-journal sources like position statements from health bodies, nor to medical textbooks, nor to the many other good medical sources. It addressed the publications in medical journals, period... and that's because the discussion to that point was concerned with poor quality non-peer reviewed journal publications.

Below is the suggestion I made, with a few additions (italicised) to make it clear that the context is journal articles:

Every rigorous scientific journal requires external peer review before publication of primary research papers and review articles in those journals, but journals / their editorial boards may choose to adopt an internal review procedure for vetting editorial commentary materials that are published in the journals. Research and review materials published in medical journals without peer review are generally unsuitable for use in Wikipedia. Opinions expressed in reputable journals where peer review has occurred should generally be afforded greater weight than materials published in journals where the standard of review is unclear, but the latter may be suitable sources if used with care. Editors are also advised to be cautious when considering referencing a peer-reviewed paper which is published in a journal not known for reporting on that topic area or academic discipline.

I can see (given the above discussion) that an addendum to make clear that different standards are used when evaluating non-journal sources. Certainly, the ideas could be expressed in other ways (and the additions to the above are to emphasise what is being said, but not ones that would belong in a guideline), but I struggle to see any part of this that is controversial when discussing journal / review articles. I do not know of a field of science or medicine which would publish primary research or review articles without peer review. Letters / editorials etc are sometimes peer reviewed, sometimes not, and so the automatic credibility that goes with a review being in Renowned Journal XYZ does not necessarily extend to a letter to the editor of XYZ. Equally, an editorial in JAMA (say) published as the view of the Editorial Board would likely have greater weight than an editorial in JAMA written by a guest editor. Similarly, an editorial in JAMA on COVID vaccination practice would carry more weight than an editorial on political failures in management of the pandemic, simply by virtue of the former involving medical experts commenting on medicine and the latter involving medical experts commenting on politics. It is this same issue that I raised at the end of my suggestion – that an article on a novel medication for use with dementing geriatric patients that was published in the Southern Antarctic Journal of Obstetrics and Adolescent Podiatry should be approached with caution given that journal is not likely known for its publication of cutting-edge research in geriatrics (to make up an extreme example).

As others have also said, this discussion will result in changes to the guidelines, or not... and if so, my suggestions may be used (in part or in whole), or not... and I'm fine with both of those facts. But, I don't understand what is inaccurate / problematic / etc in what I suggested. It deliberately is not definitive as there are cases where different decisions are appropriate (as SandyGeorgia noted). The only absolute is not about WP, it is about rigorous and reputable / reliable journals which require peer review before publishing novel primary research and broader secondary reviews – and they do. In medical areas, any journal that does not peer review submissions before publishing them is unlikely to be a suitable source for medical information. Surely we all agree on that? EdChem (talk) 11:13, 26 January 2021 (UTC)

EdChem, your proposal got a bit lost, I confess. My thoughts:

The first sentence is fine but I don't think you needed to add the italic words as the context was already clear. To what degree are editors here using "That paper wasn't peer reviewed" as a means to reject a source? I'd like some examples. Most of what I've seen is people rejecting a journal because it gets a bad reputation, predatory, poor standards, on a black list, etc. How common is it for something we might consider a "medical journal" to not actually peer review the studies and reviews? I'm wondering if actually it is more common that they claim to send material for PR but this is done badly or ignored anyway. What's the case? If this isn't a metric we actively and explicitly use to filter sources, they why try to make it one? Guidelines document best existing practice.

I'd rather avoid the word "Opinions" when it would be safer to say "Claims made in ". I don't think we want to elevate opinions or claims made just anywhere in a reputable journal. An opinion in a letter to the editor, for example. Does "materials published in journals where the standard of review is unclear" have any merit whatsoever? We already have the WP:V requirement to select publications with a reputation for fact-checking and accuracy. Really all we seem to be saying is that reputable journals use PR for x/y/z types of article and we only use reputable journals. The PR bit isn't our selection-criteria.

The clause "but the latter may be suitable sources if used with care" isn't helpful in a guideline (everyone thinks they are careful) and redundant to the "should generally be " earlier anyway. Similarly "Editors are also advised to " is unnecessary and not guideline style. Just start the sentence with "Be cautious when ". -- Colin°^Talk 12:10, 26 January 2021 (UTC)

@Colin: one recent example of non-peer-reviewed sources is preprints. The COVID-19 articles were plagued by editors citing medRxiv to support biomedical claims which were often speculative hypotheses based on small studies. The point of medRxiv is to allow researchers to see what results others have found in their field and require the expertise to interpret those results in context – the sort of expertise we don't allow Wikipedia editors to exercise when writing our medical content. MedRxiv was producing preprints at a very rapid rate and were attractive to less seasoned editors who felt they were keeping Wikipedia up to date with recent advances, despite there bring no guarantee that the research described in a preprint would necessarily remain unaltered by a subsequent peer review prior to publication in a respectable journal. This has happened so often that I feel we would benefit from some guidance that would be helpful to newer editors in their choice of sources. --RexxS (talk) 12:47, 26 January 2021 (UTC)

RexxS, preprints are a potential-source for discussion, though perhaps a separate one. I'm surprised MEDRS doesn't (AFAICS) mention them and it should. But MedRxiv says it "is a free online archive and distribution server for complete but unpublished manuscripts (preprints)" (my bold) and the home page has a big red warning about how they should not be cited as though "established information". So they fail WP:V's "publications with a reputation for fact-checking and accuracy" and don't need us to tie our selves in knots about peer review. In my understanding (see MEDRXIV), preprint database exist (entirely?) due to the length of the peer-review process, and according to the Wikipedia article, can even be part of that process. So it isn't really an example of a medical journal that doesn't peer review its research or review papers, nor an example of where a Wikipedian has rejected a paper published in a medical journal explicitly because it has not been peer reviewed. -- Colin°^Talk 14:06, 26 January 2021 (UTC)

I am late here and this is a lot to digest. My view on Editorials is that they are not WP:MEDRS compliant. I have only published one Editorial (https://pubmed.ncbi.nlm.nih.gov/3132369/) albeit a long time ago and it was not peer reviewed. The editor just sent his thanks. Much more recently, the International Committee for the Taxonomy of Viruses published an Editorial that said viruses were not alive which opened a great big can of worms. Graham Beards (talk) 16:11, 26 January 2021 (UTC)

I agree, Colin, that we ought not to have to ask editors not to use preprints, but if I had 10p for every time I copied that big red notice to a user page or a talk discussion, I wouldn't have to cash my pension cheque this month. I'm pretty sure most of the editors who made the mistake didn't realise the difference between having a preprint published on medRxiv and having an article published in a medical journal. By the way, did you notice that our article on Medical journal, linked from the opening paragraph of MEDRS, defines it as "a peer-reviewed scientific journal that communicates medical information to physicians and other health professionals." I'm pretty happy with that (even if it's only 99% accurate or whatever) as it gives an indication that editors expect reputable journals to be peer-reviewed. That should hopefully mean that we don't need to go to great lengths to explain what are the desirable features of a source for use in medical articles. --RexxS (talk) 16:46, 26 January 2021 (UTC)

Since MEDRS is the application of other policy to the medical domain, I don't mind if MEDRS explains what a preprint is and why it isn't suitable, and that explanation will of course explain such pre-prints haven't yet been PRd but also mention the more important factor is that they haven't yet been published in a reputable journal at all. Wrt the text that you'd be "pretty happy with", well we currently say "Every rigorous scientific journal is peer reviewed" and questioning whether that 100% accurate is what brought us here and landed it with a "dubious-discuss" tag. Its like "This car has a 7 year warranty" and then pedantically insisting we also state that it doesn't cover the tyres, battery, wipers, petrol, floor mats, bulbs, etc, etc. Sometimes, a description doesn't need to be 100% accurate to function perfectly well. -- Colin°^Talk 17:31, 26 January 2021 (UTC)

• We don't say, " "Every rigorous scientific journal is peer reviewed" That's content added and edit warred by an IP who even now doesn't seem to understand Wikipedia very well and was blocked for his disruption. I'm sure Rexx could answer this perfectly well, so I perhaps should have not responded to you, but I dealt with the IP and refused to deal with this content knowing dealing with this idea and the IP would be tiring. I am increasingly confused by this discussion, maybe one reason is that we are arguing for something that was not in the stable version of the guideline in the first place. Littleolive oil (talk) 18:58, 26 January 2021 (UTC)
  • Littleolive oil, that text was not added by the IP. The current paragraph dates from this edit by SandyGeorgia in April 2007 with edit summary "add back general info from WP:RS". Minor tweaks over the years changed "serious" to "rigorous" and "any" to "all" but the meaning's the same. The various proposals to change our current text all have some serious flaws. I think if we have reached the stage where we can't even agree why this discussion was even started, or why we are still arguing, then it is time to close is. -- Colin°^Talk 19:41, 26 January 2021 (UTC)

This is what opened this discussion. This what I was discussing.[4]

Opening this discussion for 50.201.195.170 who proposes a change to MEDRS's lead (diff):

Original sentence:

Be careful of material published in journals lacking peer review or which reports material mainly in other fields.

New version

Every rigorous scientific journal is at least partly peer reviewed; papers are always reviewed, but editorials and comments may not be. Be careful of material published in journals lacking peer review, or which report material mainly in other fields.

Littleolive oil (talk) 19:46, 26 January 2021 (UTC)

Littleolive oil, look just a few lines further down and you will see RexxS makes a more accurate current/original quote and PaleoNeonate thanks him for the correction.

I have been dealing with Paleo's version which was confusing for sure. Despite that, I didn't support the IP's version and what was in place was fine. I'm going to leave this alone for now. I don't see that anything is being sorted here. Thanks all. Littleolive oil (talk) 20:06, 26 January 2021 (UTC)

Actually, RexxS, Sandy's edit has reminded me of WP:RS which also discusses scholarship and peer review. Since preprint databases are not unique to medicine, I think it is actually more appropriate if WP:RS addresses this phenomenon. There doesn't seem to be anything specially medical about them, other than perhaps their being a particular boon/pest (depending on whether one is a researcher or wikipedian) during the pandemic. -- Colin°^Talk 19:52, 26 January 2021 (UTC)

• This is insulting and untrue. No one is trying to redefine Peer Review. [5]. Littleolive oil (talk) 20:14, 26 January 2021 (UTC)
  • When we were discussing a "Special Report" for which I could not find an explicit peer review statement, you replied "A special report written by 30 authors along with UK and US charities is by its nature reviewed by peers. Peer reviewed simply means oversight by peers and this report has that kind oversight" That's what I took as redefining "peer review" to mean simply any kind of review by one's peers at any stage of the article writing process (i.e. before submitting for publication). That's very much not what "peer review" is held to mean, which is a post-submission process controlled by the journal's editors and has to involve peers other than the article authors. I have already complained on this page about 'redefine what "peer review" means' and felt the need to correct those earlier comments about what "peer reviewed simply means..." by listing some actual dictionary definitions. My comment on MastCell's page was not made behind anyone's back, as I'd already complained about it here. If it was not your intention to redefine Peer Review, then what you wrote was a mistake imo. Let's move on. -- Colin°^Talk 10:06, 27 January 2021 (UTC)

No one accused you of going behind our backs. And my comments, as I think you know, were not an attempt to redefine peer review but an explanation of how oversight which is a kind peer review process is carried out in other situations. I'm sorry you are responding this way. It is a grave disappointment. I'm done here. Littleolive oil (talk) 17:45, 27 January 2021 (UTC)

meta

I'm reading the above, and my heart hurts, and I try again, and in the end, I feel like we're not managing to say what we actually mean. Let me give an example:

Editors say above that peer review is a required minimum standard for sources about health-related content.

This is wrong. In truth, we happily cite med school textbooks in thousands of articles. Harrison's alone is cited in hundreds of articles. Textbooks are not peer reviewed – not in any possible sense. The editors who say that they would never-ever-ever cite a source that wasn't peer reviewed have almost certainly already done exactly that repeatedly (and correctly).

But we're not making statements like this because we believe textbooks to be peer reviewed, or because we believe that textbooks are inappropriate sources, or because we actually want to write MEDRS to be self-contradictory about whether non-peer-reviewed non-journals are acceptable sources just because they don't use the methods that journals use. We're just throwing any possible argument on this page that might let me edit the guideline to solve my problem and get back to the stuff that I'd rather be doing.

So I ask: What's the real problem? If the real problem is an overly bold IP, then we can semi-protect the page. If someone's screwing up an article and (mis?)quoting this guideline, then let's help resolve the dispute. If you think it'd be helpful to explain something in excruciating detail, then let me recommend that you start with an essay (also: I recommend looking for some advice on writing end-user documentation, so that what you write is more likely to be useful). If you're burned out by trying to hold back the endless tide of garbage, then changing this page won't help, because Wikipedia:Nobody reads the directions anyway, but let's talk about what it would take for you to share that load with others. Okay? But as a favor to me, please don't keep writing that you'd never cite excellent sources that don't happen to be verifiably peer-reviewed. You should be citing non-journal sources for some content. WhatamIdoing (talk) 21:24, 5 February 2021 (UTC)

Agree. -- Colin°^Talk 17:55, 6 February 2021 (UTC)

@WhatamIdoing: This discussion began with a pretty simple suggestion for a change, and ended with a charge that peer review was being redefined. I have no desire to try and trace where the misunderstandings entered this discussion, but this is my position and understanding of peer review as an editor who has been editing in the MEDRS area for awhile. Academic/scholarly peer review is an external review by experts for research papers (our primary sources) and/or suitability for publication of those papers. We, on Wikipedia, ask for peer reviewed sources when citing research studies (primary source), as a minimum. We also look at journal quality-impact factor for example. Medical textbooks are often considered secondary sources, our required baseline for Biomedical/health related articles. Per our guideline:

High-quality textbooks can be a good source to start an article, and often include general overviews of a field or subject. However, books generally move slower than journal sources, and are often several years behind the current state of evidence. This makes using up-to-date books even more important. Medical textbooks published by academic publishers are often excellent secondary sources. If a textbook is intended for students, it may not be as thorough as a monograph or chapter in a textbook intended for professionals or postgraduates. Ensure that the book is up to date, unless a historical perspective is required. Doody's maintains a list of core health sciences books, which is available only to subscribers.[27] Major academic publishers (e.g., Elsevier, Springer Verlag, Wolters Kluwer, and Informa) publish specialized medical book series with good editorial oversight; volumes in these series summarize the latest research in narrow areas, usually in a more extensive format than journal reviews. Specialized biomedical encyclopaedias published by these established publishers are often of good quality, but as a tertiary source, the information may be too terse for detailed articles.

This is the guideline I would and have used for Wikipedia articles and secondary sources. There is an implication in this guideline that these text books are overviews of the "good" research, that is, research we can use, and is generally an overview of peer reviewed papers. "Major academic publishers (e.g., Elsevier, Springer Verlag, Wolters Kluwer, and Informa) publish specialized medical book series with good editorial oversight; volumes in these series summarize the latest research in narrow areas, usually in a more extensive format than journal reviews."

Comments by respected bodies have an implied internal review by its peers. This specific language is important. Review by peers does not equal and is not the same as the formal scholarly process, peer review, but it does mean there is in general oversight/acceptance/agreement by the body of experts who make the statements.

We require, in general, in the discussions I have been in, peer review, at least, for science based content. We require secondary sources such as systematic reviews and textbooks for MEDRS related articles. So yes we use textbooks all the time but they are secondary sources and generally are overviews of peer reviewed research. We have processes that allow for discussion and inclusion of other sources if editors collaborating on an article allow for it. I don't mind that our guidelines and policies are stringently worded. Wikipedia has an overriding process in Ignore All Rules so that on occasion and with agreement sources which do not fall within the boundaries of our policies and guidelines can be used. In my opinion we use more stringent language to define our Wikipedia positions and then if considered necessary, with agreement, can transgress our own, carefully worded boundaries. We have had problems with newer editors not understanding the guides we have for editing. (As a newer editor myself dealing with an area where there were hundreds off peer reviewed papers, I, along with other editors didn't know anything about MEDRS. It wasn't a well known guideline. I was criticized in an arbitration for using peer reviewed primary sources instead of secondary sources. I and my adversaries were not using MEDRS as a standard). There is a value to advising newer editors with strongly worded guides that can be adjusted in specific individual cases, with consensus. It seems to me we are getting caught on language and ideas that are better applied after we refer to the policies and guidelines, after we acknowledge that we have best practices but that these best practices can be adjusted to create best articles. It seems simple to me, but here in this discussion things went sideways or even upside down. Littleolive oil (talk) 21:05, 6 February 2021 (UTC)

@Littleolive oil, what actual problem are you trying to solve? WhatamIdoing (talk) 01:17, 7 February 2021 (UTC)

I am not trying to solve a problem at all. You seemed to have a problem with the discussion here, were concerned about the use of textbooks, were concerned about how peer review was being used. I responded to that post. However, it seems my concerns and response were misplaced. Littleolive oil (talk) 03:14, 7 February 2021 (UTC)

Straw poll

We should add something to the guideline noting that "editorials and comments may not be peer reviewed". Exact wording can be discussed. It seems to me that most editors here except for one favor some sort of change in this direction. Maybe it can be a footnote. But we should explain this matter to editors who may not be familiar with peer review; something dubious might be published in even a highly reputable journal's commentary/editorial pages and we don't need to hinder ourselves by failing to explain the difference of those articles. The guidelines are prescriptive, not descriptive, and this is what is actually done anyway by editors who are aware of the distinction, so it does describe what occurs. Crossroads ^-talk- 22:45, 26 January 2021 (UTC)

I agree that we should add something, but not today. I am not opposed to accurate descriptions of what parts of the journal are reviewed, but as you can see, I'm very cautious about additionally writing new burdens and rules that don't bear resemblance to best practice or that appear to restrict editors from using sources appropriately. I'd like to learn more about how often editorials are peer reviewed and why some in the BMJ are and some are not, before just writing "may not be". And I'd rather not write "may not be" about things like comments or letters to the editor, which likely are never peer reviewed. It is also apparent to me from the literature that PR is not a panacea. It could be good imo to take a break from this and hopefully get some input from knowledgable editors, or perhaps for people to collect sources about the process that we can read. -- Colin°^Talk 08:51, 27 January 2021 (UTC)

Yes - I have seen editorials that included opinions by widely respected leaders in a field where some interesting assumptions have crept into an otherwise core body of text. I think it is a point worth (succinctly) making. Cas Liber (talk · contribs) 13:32, 27 January 2021 (UTC)

Agreed, I've seen several talk page disputes (I don't have a handy example, but could certainly find some if anyone is skeptical) where a newer editor wished to source controversial info to a commentary piece, and were aggravated when regular editors tried to explain the distinction. Ajpolino (talk) 07:40, 1 February 2021 (UTC)

I'm not sure that it would prevent the aggravation. I think it's more likely to get turned into a bright-line "no commentaries!" rule. WhatamIdoing (talk) 21:28, 5 February 2021 (UTC)

Would that be a bad thing? Crossroads ^-talk- 04:37, 6 February 2021 (UTC)

I'm pretty convinced at this point that "may not be peer reviewed" is simply a red herring and has led us nowhere useful. They are simply opinion pieces and thus separate from the (supposedly) evidence-based medicine that normal journal papers have to demonstrate and that is checked by editorial and PR processes. -- Colin°^Talk 17:53, 6 February 2021 (UTC)

Crossroads, take a look at these: PMID 33184048, PMID 33504502, and PMID 33361141

All three of these analyze and summarize previously published literature, which means that they're secondary sources. But they're not peer-reviewed, so BMJ marks them as "editorials". This has been pretty typical for COVID coverage this year, due to the pressure to get the best advice out the door instead of spending a couple of months in the peer review process. Do you think that these sources should be considered unusable? A bright-line "no commentaries" rule would have that effect. WhatamIdoing (talk) 01:32, 7 February 2021 (UTC)

Since these replies are indented as response to my comment, I'll just say I don't think any addition is worth the time and energy that this has engendered. Sadly, I often see MEDRS used as cudgel rather than guideline. If folks are concerned that such an addition would result in more cudgeling, I'm sympathetic to that view, and am happy to leave the longstanding language as it was. Ajpolino (talk) 04:03, 9 February 2021 (UTC)

• While I agree with Cas about things that creep in, I don't think we are yet at a point of having meaningful text to add in this department, and am also worried that anything we might add at this point would create a "bright-line" effect that would extend beyond Wikipedia-wide guidelines about when editorials and commentaries can be used. I offered the example at habit cough, which I have been (stalled at) reworking in sandbox, but my better example is one I give often, regarding misinformation spread via a New England Journal of Medicine review, which was addressed by a letter to the editor that contained the correct information! We cannot always count on even the highest quality peer-reviewed publications to be more correct than a commentary. When Jankovic, a well-respected expert in movement disorders, made the claim in a high-quality review (PMID 11642235) that Tourette syndrome occurred at the intersection of tics, ADHD and OCD (something that has never been part of the diagnostic criteria, is false, and has no basis in fact or medical consensus), correct information was published in the NEJM via a letter to the editor (PMID 11870255), and yet the NEJM never retracted the article. Now, all of this occurred before Wikipedia had an accurate article about Tourette syndrome, but we need to allow normal editing processes to account for such situations should they occur today. We run the risk of attaching too much significance to "peer-reviewed", that could be used as a bludgeon as we have seen MEDRS used in the past, unless we better nail down wording that is in agreement with broader Wikipedia guidelines. SandyGeorgia (Talk) 18:33, 9 February 2021 (UTC)

  All I'm asking for is to note that they may not be peer-reviewed. This isn't a ban or a "shouldn't use". And since this is the encyclopedia anyone can edit, the vast majority of editors do not have the expertise to decide when the review articles are wrong. Even so, I don't see how my proposal affects the scenarios you mention. We do want MEDRS used as a bludgeon sometimes; there are a lot of POV pushers, snake oil salesmen, and Dunning-Kruger afflicted people who edit these articles and we should be careful not to 'deregulate' the medical topic area. Wikipedia's quality comes just as much from what we don't say as what we do. Crossroads ^-talk- 18:58, 9 February 2021 (UTC)

So, I think the way forward on all of these guideline wording issues is to end all of the personalization, dredging up of old history, outright personal attacks (of which there are quite a few in this very discussion), pinging and crossposting for support, and instead engage in careful and deliberative line-by-line looks at one sentence at a time, based on concrete wording proposals, so that we can find areas of agreement and compromise. The “need to add something” becomes too quickly problematic. I have tried to base my positions on concrete examples. Consensus is reached via discussion, not voting. Everyone in this discussion wants the most accurate possible medical text, and casting aspersions derails discussion. Basing discussion on very specific sample wording, and ceasing the kind of personalization that is seen in this conversation might help the overall effort to rewrite both medical guidelines, as they both are in need. SandyGeorgia (Talk) 17:33, 10 February 2021 (UTC)

While we're at it, let's all also end the WP:STONEWALLING, which is another kind of disruptive behavior, along with the assuming bad faith of what any pings or crossposting were for. As for how to reach consensus, well, there is no way to "compromise" when it's a binary, yes-or-no question. It's one or the other. Consensus does not require unanimity. And compromise is a two-way street. Crossroads ^-talk- 19:59, 10 February 2021 (UTC)

Peer-reviewed articles and secondary sources

Hello. Could somebody tell me if a peer-reviewed article published in a renown scientific journal is automatically a secondary source or if it can be debated endlessly? Thank you very much in advance for your answer: Tatvam (talk) 17:23, 14 February 2021 (UTC)

Hello friend. Medical journals usually contain a combination of primary source articles (experiment/study/trial results) and secondary source articles. If you're using PubMed as your search engine, it will tag some articles, which can be helpful for figuring out which ones are secondary. The secondary tags are review, systematic review, meta-analysis, guideline, and practice guideline. I'm new here, so others can correct me if I'm wrong or answer in more detail. Hope that helps. –Novem Linguae (talk) 18:28, 14 February 2021 (UTC)

@Tatvam, a review article is one that combines several previously published reports. The most common types for Wikipedia's purposes are Literature reviews (the author picks the best sources and summarizes what the best sources say), Systematic reviews (the author includes information from all qualified sources), and Meta-analyses (the author analyzes data from multiple sources). Scholarly peer review is important, but it's a separate thing. WhatamIdoing (talk) 19:07, 14 February 2021 (UTC)

So the short answer is no. "A peer-reviewed article published in a renown scientific journal" is not automatically a secondary source. WhatamIdoing outlines above secondary sources. But remember a source whether secondary or not is only useful per the content it references which means while a source might be a reliable source for one bit of content it might not for another. Keep at it, it takes a bit of time to get the hang of Wikipedia. PS (I love the image on your user page. I have tried to paint a single cloud in a cloudless sky many times!) Littleolive oil (talk) 21:21, 15 February 2021 (UTC)

Thank you everybody for your answers. And particularly Littleolive oil because he understood my point precisely. All the best: Tatvam (talk) 09:09, 16 February 2021 (UTC)

List of medical reliable sources

Screenshot of CiteHighlighter user script

Hello. I've been expanding the New page patrol source guide, and I recently expanded the medical section. I'd like to run it by you guys and see if you have any suggested edits or additions? Are there any "no duh", obvious, uncontroversial medical sources we can add to the reliable, unreliable, or no consensus sections?

I also created a citation highlighter user script that gets its data from that page, which is why I am updating it. I do realize the NPPSG medical section is a simplification, perhaps an over-simplification. But it's intended to give non-experts a starting point. Thanks for your feedback. –Novem Linguae (talk) 12:32, 10 February 2021 (UTC)

I am not so impressed with the American Heart Association after this: Wii#Health effects. I wonder what others think. I am not sure your list will be used appropriately; there is no indication given of how to check whether an article is a secondary source. SandyGeorgia (Talk) 16:51, 10 February 2021 (UTC)

I do have concerns about such a list. Sources are only reliable per the content they support. There are sources we might and do not consider useful, in general, by our own standards, but there may be instances where those same sources are used. Quackwatch for example is not by WP standards, reliable and yet is considered useful by some. I guess I would include some kind of caveat to this list about "only reliable per content".... better wording but that sense. Littleolive oil (talk) 17:04, 10 February 2021 (UTC)

Hey SandyGeorgia, thanks for your feedback. You'll note that the first paragraph of the linked section is Keep in mind that even if a journal is reliable, WP:MEDRS usually requires using a secondary source. So that means the article needs to be marked as a review, systematic review, or meta-analysis. It is not usually appropriate to cite a paper describing a single study or experiment, which is a primary source. So I did what I could to mention the secondary source issue. However, I also think there's only so much a non-experienced, non-WikiProject Medicine person can do. For example, my personal experience is that a lot of PubMed articles are untagged, so it's hard for a non-expert to tell if they're a review. At the end of the day, my goal is to create a tool where you can glance at the reflist, and give more scrutiny to suspicious citations. The four preprint websites in the "Unreliable" category should hopefully help with this. But of course, no RS tool will ever be perfect. For example, The New York Times is generally reliable, but that doesn't mean their opinion pieces are reliable. –Novem Linguae (talk) 22:23, 10 February 2021 (UTC)

@Novem Linguae, how do people use that list? Is this "sources like these show notability", or "sources like these are high-quality sources"?

I have not been impressed with efforts to declare that this journal or that website is reliable or unreliable in general. There are sources at the extreme, but it's mostly either obvious (e.g., any blog) or more complicated than that. The AHA, for example, is a perfectly fine source if the statement in the article is about what the AHA recommends as a heart-healthy diet, and it's not actually unreliable if you're defining basic terms. But all of the best medical journals in the world have retracted papers for very serious problems, and it is not unusual for editors to misrepresent the papers either accidentally or deliberately. Someone at Autism this week is trying to re-write the description of good verbal skills in autistic kids to say that the ones in the study weren't specifically high-functioning autistic kids, but instead were just "some" autistic kids. Highlighting the journal will not help.

If you're looking for notability, then all the big health websites are evidence of notability: Mayo Clinic, WebMD, MedlinePlus, etc. We don't want articles WP:Based upon them, but they do indicate "attention from the world at large", i.e., notability. WhatamIdoing (talk) 03:55, 12 February 2021 (UTC)

• how do people use that list? Is this "sources like these show notability", or "sources like these are high-quality sources"? Normally both. To pass GNG, you need sources that are all 4 of: independent, reliable, secondary, and significant coverage. So reliability is a big component of notability. MEDRS might be different though, as I imagine Mayo Clinic and WebMD should not be content sources for anything, and maybe not even notability sources.
• This idea that there's many "exceptions to the rule" is implied in any reliable sources list, I think. Take WP:RSP, for example. The New York Times is listed as green and generally reliable, but the NYT has opinion pieces, and has issued retractions. No source or source list is perfect. This is just a tool to glance at a reflist and help analyze sources. For example, I have 4 preprint websites on there, and the red highlights have been helpful to me for finding and fixing issues with preprints.[6] –Novem Linguae (talk) 04:28, 12 February 2021 (UTC)

WhatamIdoing, quick follow up question, if I may. How bad do you consider Mayo Clinic, WebMD, Medline Plus, and other "MEDRS's of last resort"? Are they YouTube and Twitter bad, where if you see them in an article, you try to snip them out? Or are they not as high on your radar? I see from the RSN discussions that they are OK for "uncontroversial claims". Trying to decide if they should be marked yellow or red in my highlighter. –Novem Linguae (talk) 13:48, 14 February 2021 (UTC)

We have a misalignment in the verifiability and notability rules, and that's why I asked you about what you're using the script for. If you're working on AFC and NPP, you should treat these as green-light sources. For notability purposes, you should also treat newspaper and magazine articles as green-light sources. It is possible, and not even unusual, to have (e.g.,) an experimental drug get thousands of words printed about it in high-quality newspapers, to be an excellent example of a subject that exceeds the GNG's requirements ...and still not have a single "MEDRS-style" source that lets you write a sentence that contains any Wikipedia:Biomedical information about that drug. If you go to AFD and say "Here are seventeen articles, each of at least 300 words, about this experimental drug, all from the highest quality news media, but this doesn't meet GNG because they're not medical journals", then people will laugh at you, and maybe one of them will explain that there's a zillion-dollar industry, and that articles about drugs are articles about commercial products, not just about the aspects that individual healthcare providers and their patients care about.

So Wikipedia should have that article anyway; to cope with the disconnect in the rules, we try to steer these nascent article towards "business" or "history" content and away from "biomedicine". Under those circumstances, we would write that Wonderpam is being developed by Big Corp, that certain regulatory milestones have (or haven't) been met, that the market is small, that the price is astronomical, etc. We wouldn't write that Wonderpam actually works, or that it has certain side effects, or things of that nature, or if we felt it was necessary, then we would write these in very hedged language ("In Octember 2020, the company was investigating reports about people's skin turning orange").

If, on the other hand, you are screening articles and trying to make small, quick improvements, then the long-standing consensus is that these sources are okay for uncontroversial content. IMO the point at which you should try to meet what MEDRS calls the "ideal" (NB "ideal", not "rock-bottom minimum requirement") is when you're trying to invest a lot of effort to make the article reach GA or FA standards. At that point, I'd expect you to have enough high-quality sources in hand that you would replace all of these okay-but-less-than-ideal sources. WhatamIdoing (talk) 19:02, 14 February 2021 (UTC)

WhatamIdoing, thanks for your thoughts. I'm going to say that the CiteHighlighter user script is a reliability script and NPPSG is a reliability list (we're including some primary and tertiary sources on there, therefore it is not strictly a notability guide, because GNG requires secondary sources). I wonder if WedMD and Mayo Clinic are tertiary sources, judging from their expandable source/reference lists at the bottom and their high level summaries of these topics, and therefore can't be used to satisfy notability. But that's a tangent. Thanks for the info on their reliability. –Novem Linguae (talk) 09:18, 16 February 2021 (UTC)

Well, @Novem Linguae, before it would make sense to decide whether these are tertiary sources, we'd first have to back up and find the rule that directly says that tertiary sources can never be used for notability. You won't find that claim anywhere in WP:N, for example. Also, every time I've asked whether the GNG counts only sources that separately meet all of the requirements (e.g., a magazine article that is independent and secondary and reliable and contains SIGCOV would count, but an otherwise identical article in the same magazine that meets only three of those four criteria would not), or whether maybe it would be possible to meet the GNG by having several sources that each meet some of the criteria, there has been no consensus for the stricter approach.

The actual rule, although we are scared to write it down in plain view because that might embolden the paid spammers, appears to be that, if you are proceeding under the GNG, then you definitely need independent reliable sources, and that at least one source usually – but not always – must be secondary, and that at least one source usually – but not always – must contain SIGCOV.

Also, "secondary" for WP:N purposes is really defined as "not primary" (did you know that "tertiary" sources aren't a thing in some academic fields? In law, for example, everything is either primary or secondary), and occasionally, especially for current events, secondary is defined as "independent", because it's just too difficult to convince editors that most newspaper articles are primary sources, especially since there is one (1) book that contradicts all of the university libraries, historiography papers, and scholarly books that say that most news articles are primary sources.

So that's one thing: You shouldn't assume that tertiary sources aren't signs of notability. A telephone book is tertiary and isn't ever a sign of notability; an encyclopedia entry is usually tertiary, and it is very strong evidence of notability. Telling people that Encyclopædia Britannica has a whole article on a subject, but you don't think it's worth a whole article at the English Wikipedia is a way to get yourself banned from AFD.

The other thing is: Those are probably secondary sources. WP:Secondary does not mean good. Secondary means that you took someone else's original ideas and information, and you turned their original into your own 'thing' – say, a high-level summary of what you want your patients to know, that emphasizes the stuff you think is good and disparages the stuff you think is bad.

If I were going to recommend a basic rule for you, it would be that if a website such as WebMD or Mayo Clinic has an entire page dedicated to a product, treatment, or medical condition, then that should be taken as very strong evidence of notability. WhatamIdoing (talk) 02:52, 17 February 2021 (UTC)

WhatamIdoing, yeah, lots of unwritten norms at AFD. Notability is probably one of the most complicated areas of the encyclopedia. Looking at GNG, it says sources "should" be secondary, which is weaker wording than I thought. Good to know. Thanks for sharing your insights. –Novem Linguae (talk) 03:04, 17 February 2021 (UTC)

Now I will add that the GNG was written at a time when a sizable number of editors were certain that "secondary' and "independent" were functionally synonyms. A rational person might conclude that the GNG doesn't actually mean anything. WhatamIdoing (talk) 03:38, 17 February 2021 (UTC)

Novem, I applaud your effort, but it leaves me a bit nervous. It seems that the best approach for editors not familiar with biomedical content sourcing is for them to post a query here (WT:MED), as many of them do. I'm not convinced the necessary info can be easily summarized in a way that works for any article, any source, any editor. Best, SandyGeorgia (Talk) 23:34, 14 February 2021 (UTC)

Good discussion, and I appreciate any reasonable efforts (like yours, Novem Linguae) to help newbies. :0) ¶ Simply for awareness: MEDLINE does not index many reputable social science journals, therefore one should not rely on PubMed searches alone if one wants to find the best, most recent articles on mental health topics. Mark D Worthen PsyD (talk) [he/his/him] 18:41, 15 February 2021 (UTC)

Primary sources usage

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

I suggest the following statement concerning Primary Sources (PS) to be revised OR completely eliminated from the intro as contradictory to goals of Wikipedia and both dogmatic, confusing and misleading relatively to the title under which this article exist, and generally flawed. Just for comparison:

The Title: «Identifying reliable sources (medicine)»

The Guide: «Primary sources should generally not be used for medical content» — Ruthlessly and indiscriminately excluding a whole range of potentially useful primary sources.

Too many contributors took this statement as unquestionable rule or misapplied it even though it's neither strictly a policy (WP:POLICIES), nor a firm guide. Not just only it's contradictory to the said goals but it's directly in violation with WP:PRIMARY (from WP:OR policy, and many other useful advises: WP:PRIMARYNOTBAD, WP:PRIMARYCARE) which clearly state that primary sources may be used with care. The most recent and fresh example is here: Pancreatic cancer revert, 15:55, March 20, 2021; the contribution amended contained mostly primary sources. It was inevitably and ruthlessly amended even though under this guideline they are perfectly reliable to the information they were given.

If revised I suggest to lift the restriction on PS and allow to add primary-sourced information strongly on discussion and consensus-basis (if challenged) unless such information is clearly misleading or if PS used are used to back up statements clearly out of scope of the former. AXONOV (talk) ⚑

EDIT: If you are not truly interested in this discussion or judge lightly and superficially please avoid getting involved.

• No. I don't support this at all. Littleolive oil (talk) 17:56, 20 March 2021 (UTC)

I've been working off and on in the area for about 14 years. Wikipedia has determined that it's focus is on the summary of biomedical information, an overview of the research, and because there are readers who use Wikipedia for biomedical information, primary sources- new research that has not been replicated is a risk for readers looking for medical information. This has been discussed many many times and nothing has convinced me that we should throw out WP: MEDRS. Littleolive oil (talk) 18:18, 20 March 2021 (UTC)

@Littleolive oil: I seek either to loosen or detail the quoted guide line which indiscriminately prohibits everything that deems to be primary without due concern for actual statements these PS are used for. On the other side, where do think all those researchers coming here out of curiosity would find clues for their future study seeing no (notable based on consensus) frontline studies/reporting mentioned? Should be always resort to consensus based contributions in case of PS-backed up contributions? --AXONOV (talk) ⚑ 20:09, 20 March 2021 (UTC)

Exactly. I don't support loosening the guide at all. This is a collaborative-based project so yes we resort to consensus. Please read the other comments here which explain when primary sources could be considered. Littleolive oil (talk) 20:31, 20 March 2021 (UTC)

@Littleolive oil: .. when primary sources cold be considered Well but this is not subject to any guideline so guidelines cannot be used to justify reverts. You may question notability policy by requesting more related sources but this is often unwarranted as statements aren't seriously bold.--AXONOV (talk) ⚑ 21:27, 20 March 2021 (UTC)

• Oppose. The tilt to prefer secondary sources for biomedical content is one of the wisest WP:PAGs Wikipedia has. Without it Wikipedia could (for example) be complicit in health scams like the MMR autism scandal, and numerous others. MEDRS does not absolutely prohibit primary sources, so the proposal is a bit disingenuous in any case. Alexbrn (talk) 18:21, 20 March 2021 (UTC)

@Alexbrn: .. Wikipedia could (for example) be complicit in health scams like the MMR autism scandal I totally agree with this but I hardly see it as good justification to disallow indiscriminately a whole range of sources. It has nothing to do with identification of reliability as it falls directly under Original Research (OR) policy which clearly disallow making any research here so it should be applied respectively. (Under OR I mean leap-logic conclusions based on speculative researches, kind of casuistry, facts-broadening etc.)--AXONOV (talk) ⚑ 20:09, 20 March 2021 (UTC)

Wikipedia is an encyclopedia, a tertiary text. It is thus primarily built on secondary sources. If we bypass those and start picking out primary sources which secondary sources have ignored, then we become an ersatz secondary text and that indeed, is a species of WP:OR, which is to be avoided. We are meant to reflect accepted knowledge, and that is predominantly found in high-quality secondary and tertiary sources. 20:15, 20 March 2021 (UTC)

@Alexbrn: Of course we should not trash anything. That's the WHAT WIKIPEDIA IS NOT-venue. The secondary sources are always preferred but aren't required under any threat of punishment. Some too vehemently defend rules which are rather more harmful than the harm they were intended to prevent.--AXONOV (talk) ⚑ 21:27, 20 March 2021 (UTC)

• The statement about primary research articles not being generally suitable as sources has widespread support, and has done since the guideline was created in 2006. Many people, coming from a scientific background where they are expected to review the primary research literature, find this confusing. They want to write their own literature reviews as Wikipedia articles. They may even think they know better than the people who do write reviews and get them published, and that process is too darn slow! The kind of things Wikipedia lets people use primary sources for, could theoretically also be used for these primary research papers: that in 2017 some researchers did a study with a number of participants and claimed certain results and published them in 2019. In that sense it is similar to citing a book for the facts that Joe wrote a book about plants and published it in 2021. But just like we can't cite that book to claim "Joe's book on plants is the authoritative reference to which all others are compared", we can't cite a primary research paper and claim drug X cures disease Y. So people try to slip the details of a study, as a proxy for its conclusions, into Wikipedia all the time. The problem is, on its own, we don't know if the medical/scientific community consider that study or its findings to be notable or even agree on its conclusions: do they affect clinical practice or accepted thinking about a disease, say? WP:WEIGHT, policy, requires editors consider the weight of some fact/opinion in the secondary literature. MEDRS is the application of policy to medical article sourcing, and editors adding text sourced solely to primary research papers are nearly always violating policy. -- Colin°^Talk 18:29, 20 March 2021 (UTC)

@Colin: .. So people try to slip the details of a study, as a proxy for its conclusions, into Wikipedia all the time. This problem deserves to be addressed IMO but the way it's done here is a bit overkill. I suspect it's resulted from misunderstanding of difference between policy and guideline and purposes of different policies in general.--AXONOV (talk) ⚑ 20:09, 20 March 2021 (UTC)

That's unlikely to be the problem, since Wikipedia:The difference between policies, guidelines and essays is pretty minimal. WhatamIdoing (talk) 02:05, 22 March 2021 (UTC)

@Colin: .. we don't know if the medical/scientific community consider that study or its findings to be notable or even agree on its conclusions Well notability criteria is a subject to WP:GNG guideline so the one should be followed respectively. The thumb rule is evidence of coverage. AXONOV (talk) ⚑ 20:09, 20 March 2021 (UTC)

• @Alexander Davronov: The fact that your reverted edit in a featured article cited primary sources does not mean that this was the only reason why it was deservedly reverted. Exceptional, consensus-based inclusion of information from primary sources is perfectly allowed; adding them before discussing them or asking for consensus would mean opening Wikipedia to endless questionable material. Also, nobody takes anything lightly or superficially here, so your "EDIT" was uncalled for. NikosGouliaros (talk) 19:26, 20 March 2021 (UTC)

@NikosGouliaros: This one is to deter those who have little time or interest in reasonable objection. The questionable things here are covered by WP:OR policy and reasonably disallowed to be introduced.--AXONOV (talk) ⚑ 20:09, 20 March 2021 (UTC)

Alexander, you disclosed the problem when you said "potentially useful". Potentially useful as long as every Wikipedian editor was as intelligent, educated, wise, correct, well read, neutral and fair as me. The rules Wikipedia has are very much a result of its open editing policy: anyone can edit. If we selected editors among those who could be trusted and who had the appropriate qualifications and experience to write articles about the state of medical research/knowledge/practice, and for whom citing the primary literature was merely a professional courtesy, we wouldn't need MEDRS. -- Colin°^Talk 21:35, 20 March 2021 (UTC)

@Colin: Well I totally agree but at least for some cases we could leave the door a bit open so the primary stuff can be allowed to be discussed and iron-rule wouldn't be an easy excuse for those who want to prevent everything non-secondary-tertiary etc. without having decent cause. AXONOV (talk) ⚑ 21:40, 20 March 2021 (UTC)

WP:WEIGHT doesn't have an "If editors on the talk page agree, then they can ignore this policy" exemption. If secondary sources do not give that primary research weight, then neither should we. And we have to live with the restrictions that imposes on us: we will not be as quick to report the latest research. One can buy a newspaper for that. -- Colin°^Talk 21:47, 20 March 2021 (UTC)

• @Alexander Davronov: It's of no use to argue the MEDRS guideline here. First, every editor who commented disagrees with you; no one thinks there should be changes, and that's a consensus. Second, a change to this guideline would require larger community input. None of us here can give anyone "permission" to change a community supported guideline even if we wanted to. Littleolive oil (talk) 22:13, 20 March 2021 (UTC)

@Littleolive oil: .. Second, a change to this guideline would require larger community input. .. That's doubtful. It took less than that to introduce challenged provision into this Guide in 2014. Why should it take more than that? There is no reason. --AXONOV (talk) ⚑ 20:46, 21 March 2021 (UTC)

• And I will add my voice to those opposing this for basically the reasons given above. There is a huge quantity of primary research studies out there, many of which contradict each other and/or are not replicated. Only academic experts are qualified to sort this out and to write literature reviews. If we water down the guidance here then it opens the door to major inaccuracies and WP:WEIGHT violations via editors (even accidentally) cherry-picking certain studies and presenting them as truth. Crossroads ^-talk- 03:12, 21 March 2021 (UTC)
• A likely belated oppose changing MEDRS primary guidance - the OP here appears to be attempting to form an argument that MEDRS violates Wikipedia principles, but has formed no such argument and merely says people abuse it. If people are abusing a guideline to inappropriately revert policy-compliant edits (which I haven't seen any evidence of here), the correct solution is to form a consensus that the edit is permitted by policy and as such is not outweighed by the guideline. I also take issue with saying that this "ruthlessly and indiscriminately exclud[es] a whole range of potentially useful primary sources" - because yet again, no evidence supporting such claim has been provided. MEDRS guidance on primary sources is primarily to clarify three sections of WP:NOT - which is a policy - specifically "not directory" (not a directory of studies published on a topic), "not textbook/scientific journal" (not a collection of studies on a topic), and "not indiscriminate" (WP is selective in what it covers both in terms of article topics and content, and does not simply include information because it exists). So, at its base, any semblance of an argument the OP here is attempting to make is hot air at best. This isn't to mention that the OP seems to claim that this guideline prohibits primary sources for medical content - which it doesn't - it just makes clear that most information can be reliably sourced to secondary sources, and that which can't better be damn important to include if it can only be sourced to primary sources. To put it simply, if information doesn't violate WP:NOT, it can almost always be sourced to secondary sources, which policy states are preferred, and as such, the use of primary sources for medical topics is almost always unnecessary. -bɜ:ʳkənhɪmez (User/say hi!) 03:33, 21 March 2021 (UTC)

• Oppose MEDRS is good and makes sense. The world needs more things that are good and make sense, not fewer. XOR'easter (talk) 19:14, 24 March 2021 (UTC)

References

Intermediate note

I the wake of claims made above I made a little research on history on challenging Primary sources use prohibition. If you are interested here is some back-story:--AXONOV (talk) ⚑ 20:46, 21 March 2021 (UTC)

Primary sources prohibition history

Date	Link	Note
January 7, 2019	What are 'primary sources' and why are they unacceptable?
March 9, 2018	Suitability of primary sources
March 5, 2014	Identifying reliable sources (medicine) - prohibiting primary sources
December 29, 2013	Proposal to add language to "Respect secondary sources" section
February 5, 2014	Better explanation of primary vs secondary sources
March 31, 2014	The stifling impact of secondary sources on the flow of information.	Abused by parties who love to remove content 1
November 22, 2013	Concern about MedRS date guidelines and wiping out sources	Abused by parties who love to remove content 2
February 27, 2014	Initial proposal to add provision «Primary sources should generally not be used for medical content»	Made by Doc James
March 5, 2014	Provision landing.	Made by Doc James
January 21, 2021	Abusing MEDRS is destructive and harmful	Abuse

I'm sorry you went to this trouble. You will not be able to change this guideline to suit your desire to use primary sources unless you have community agreement. There are, as multiple editors have said above, times when, with consensus, a primary source can be used, but the guideline says, with care-caution. None of the discussions you've posted above indicate that a single editor with out agreement changed the guideline in any substantial way. Editors who suggested changes did so because they had some agreement and discussion. Doc James, a highly experienced editor in this area, as you saw, tightened the guideline but did so knowing he had some agreement. You have no agreement/ consensus here to change anything and in fact you have multiple opposes. I'd really suggest you edit for awhile to get the hang of a collaborative community and environment; it can take awhile to realize that no single editor has the power to make changes against community consensus-community meaning multiple editors from a few to many depending on the circumstances. The MEDRS guideline protects against research that has not yet been reviewed-viewed in context of other research, replicated, shown to have some veracity in light of other research. Littleolive oil (talk) 22:04, 21 March 2021 (UTC)

More evidence of abuse

It seems like the body of facts that WP:MEDRES is highly abused just can't stop growing. Here is another case in which somehow the provision bubbled up in completely unrelated article: Restored revision 1013861229 by MrOllie (talk): All the same issues laid out for you at Talk:Pancreatic_cancer#Pathology_and_Cancerogenic_fungus. I see it as serious problem which warrants clarification to this flawed guideline. My edits were literally reverted as I was adding more reliable sources along with primary one. This corrupt practice of undue reverts must be ended. I will allow this discussion to dangle for a bit longer than when said provisions were adopted in 2014. At least I expect parties who edit WP:MEDRES-linked articles occasionally have a voice here. --AXONOV (talk) ⚑ 21:45, 23 March 2021 (UTC)

What about this is "abuse"? Natureium (talk) 22:47, 23 March 2021 (UTC)

@Natureium: This is a subsection of the ongoing discussion of the WP:MEDRS provision starting at above. I provided additional case which clearly shows that WP:MEDRS is abused to justify removal (reverting) of content which is "deemed" as poorly sourced (not always holds being true). --AXONOV (talk) ⚑ 23:22, 23 March 2021 (UTC)

Alexander Davronov, you've been given advice on the Talk:Pancreatic cancer page to use instead the secondary sources that cite the primary ones. Yet on Malassezia you persist in writing about primary research studies, citing only those primary research studies. We don't do that. Your opinion regarding the importance of the studies (in the HTML comment: "This is really interesting work worth noting it") is irrelevant. You need secondary sources to establish WP:WEIGHT. -- Colin°^Talk 10:21, 24 March 2021 (UTC)

@Colin: I often see no people's efforts to actually showing me issues with sources in relationship with specific content or specific provisions of specific guidelines. I also see no evidence there was primary sources anywhere. I see no evidence of utter need to remove (revert anything in bulk) - save to exceptional cases.
Meanwhile, on other side, I see tons of evidence of controversy surrounding WP:MEDRS and its flawed provision used to justify removing Wikipedia content with no due discretion or willing to discussion. Moreover, I see that the guideline's own supplemental WP:BMI#What_to_do_if_you_want_a_more_appropriate_source is constantly violated here (unsurprisingly).
All this warrants serious changes and revision of the WP:MEDRS. Are you going to ignore all of this? If so, I'm going to call more parties independent from WP:MEDRES community as it seems that guideline established by it is seriously abused preventing others from constructive contributions or just even discussing matters. All new comers seems to be treated with contempt. In this guideline I propose to clarify that reverts of a big chunk of sourced statements must be followed by a clear explanation of reasons of such with specific links and details. Otherwise it should be considered no more than minor WP:VANDALISM act and punished accordingly. --AXONOV (talk) ⚑ 11:02, 24 March 2021 (UTC)

The text you added to Malassezia talked about and referenced a primary research study, and cited the primary research paper that resulted from this (though you seem to have ended up with two references to the same paper). The "Science Daily" link is churnalism: at the bottom it says "Story Source: Materials provided by NYU Langone Health / NYU School of Medicine.", in other words, a press release from the institute about their latest research. Please find a review paper, or textbook, or other respected secondary source for this material. I don't know how you can say "I also see no evidence there was primary sources anywhere". I note that you are edit warring on that article, and that this does not usually end well. I suggest you try to listen carefully to what multiple parties have advised, rather than making claims that a long-standing guideline is fundamentally flawed. -- Colin°^Talk 11:26, 24 March 2021 (UTC)

@Colin: ...The text you added to Malassezia talked about and referenced a primary research study Stop trying to take this discussion into a different venue. If you feel truly impartial do it please in respective discussion: Talk:Malassezia#WIP because this is a bit different topic. The problem discussed here doesn't concern whether sources were legit or not. It's about misapplication of WP:MEDRS in order to abuse the right to remove sourced (non-verifiable as per WP:BURDEN, in bulk or not) content as "unreliable" from Wikipedia without burden of actually providing details on why and where, in which provision, for which text etc.. --AXONOV (talk) ⚑ 12:43, 24 March 2021 (UTC)

I'm not "trying to take this discussion into a different venue". You yourself linked to the revert of your primary-sourced content at Malassezia, and you yourself asked "I also see no evidence there was primary sources anywhere". I've explained your text fails WP:WEIGHT, which is policy. Verifiability is only a necessary but not sufficient condition to include text. If your problem is not so much with policy as what you perceive as editors abusing policy or guideline, then I suggest a visit to Wikipedia:Administrators' noticeboard/Incidents to report the miscreants. But I don't fancy your chances of retaining editing privileges for long. I do recommend you take a different approach and back down on the accusations of abuse. The editors who are saying you are wrong have considerable experience in this area. -- Colin°^Talk 12:59, 24 March 2021 (UTC)

I see that in fact, you are already enjoying yourself at ANI. -- Colin°^Talk 13:18, 24 March 2021 (UTC)

The discussion above is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

Nature article on "the systemic production of falsified research"

I thought that folks would want to be aware of this.

• Else, Holly; Van Noorden, Richard (2021-03-23). "The fight against fake-paper factories that churn out sham science". Nature. 591 (7851): 516–519. Bibcode:2021Natur.591..516E. doi:10.1038/d41586-021-00733-5. PMID 33758408. Retrieved 2021-03-29.

Apparently much of the sham research is landing in what could be considered MEDRS journals. I hope that our reliance on secondary & tertiary sources would help alleviate this, although if bogus primary research gets taken seriously in reviews & surveys of the literature, this would be problematic. Peaceray (talk) 17:12, 29 March 2021 (UTC)

All the more reason to insist on high sourcing standards, though ultimately if the citadel falls, Wikipedia will too, as all we can do is reflect "accepted knowledge". Alexbrn (talk) 17:15, 29 March 2021 (UTC)

Nanomedicine (Elsevier journal)

The journal Nanomedicine: Nanotechnology, Biology, and Medicine has some serious issues:

• https://scienceintegritydigest.com/2021/05/05/northeastern-university-professor-with-69-papers-on-pubpeer-has-resigned/

The accused is still listed in the editorial board.

The over 30 articles which reference the journal may need some extra checking. Nemo 05:58, 6 May 2021 (UTC)

Is genetics and population genetics covered by MEDRS?

We already have Wikipedia:Reliable_sources#Primary,_secondary,_and_tertiary_sources, so we should try not to use primary sources. We also have the essay Wikipedia:Identifying reliable sources (science). But I'm wondering if this guideline includes genetics and population genetics specifically as well. Bogazicili (talk) 15:20, 29 April 2021 (UTC)

• The essay Wikipedia:Biomedical information may be useful. The scope of "genetics and population genetics" is quite broad but I would think it is relevant if those genes are being discussed wrt health or ability. Is there a specific article conflict you had in mind. If so, WT:MED might be the better place to discuss. -- Colin°^Talk 07:43, 30 April 2021 (UTC)

Thanks. Just noticed many low quality sources in numerous articles across Wikipedia: Wikipedia_talk:WikiProject_Molecular_Biology/Genetics#Low-quality_population_genetics_content_across_Wikipedia Bogazicili (talk) 14:04, 30 April 2021 (UTC)

• As Colin said, it depends greatly on what information it is and how the information is being used. For example, discussing the genetic similarity between two similar bacteria/viruses - but not discussing how that affects humans at all (i.e. simply discussing the genetic code and not talking about transmissibility or virulence) would not be MEDRS. The second you start talking about how that genetics affects human illness/disease you must use MEDRS to cover all of the information in the tree - or at least a MEDRS must clearly make the connection between the genetic change and the change in humans - otherwise you're combining a non-MEDRS and a MEDRS to say something that the MEDRS itself doesn't say, but would require MEDRS. -bɜ:ʳkənhɪmez (User/say hi!) 14:07, 30 April 2021 (UTC)
• @Bogazicili: Unless genetics somehow linked to a human therapy WP:MEDRS doesn't apply per WP:BMI. Otherwise WP:RSCONTEXT should be taken into account. Try hard to avoid misapplying WP:MEDRS to something unrelated to what is listed under WP:BMI supplement. --AXONOV (talk) ⚑ 17:54, 9 May 2021 (UTC)
• Genetics is a field within the Biomedical sciences, so in general WP:MEDRS would apply to material in this realm of knowledge. Alexbrn (talk) 18:00, 9 May 2021 (UTC)

  Genetics is also a field within the fields of plant science, insect science, agricultural science, dog breeding, etc., in which case MEDRS wouldn't apply. I'm also not sure that we'd apply MEDRS if the question was human kinship (e.g., "Genetics studies show that the people of Ruritania are closely related to their traditional enemies, the people of Anti-Ruritania"). WhatamIdoing (talk) 01:21, 25 May 2021 (UTC)

Wikipedia:Biomedical information has an RFC

Wikipedia:Biomedical information has an RFC for possible consensus. A discussion is taking place. If you would like to participate in the discussion, you are invited to add your comments on the discussion page. Thank you. Aquillion (talk) 20:54, 25 May 2021 (UTC)

WP:SCIRS -- Wikipedia:Identifying reliable sources (science)

Recent IP edits added a need/benefit statement about SCIRS

Note that some content that does not require MEDRS may still benefit from reliable scientific sourcing (per WP:SCIRS):

• Their manufacture involves extensive use of rapid-throughput nanotechnology. This is a scientific claim, although not specifically a biomedical one

But that statement is about manufacturing process, not a scientific fact. The other issue User:Alexbrn raised with earlier text that said "need" rather than "may still benefit from" was that WP:SCIRS is just an essay. While some guidelines and policies do reference essays, we do need to be careful the reader isn't mislead or directed to some page that doesn't reflect community consensus and isn't widely accepted. There's also WP:CREEP to consider. We already link to Wikipedia:Biomedical information, which is an explanatory supplement, and not official guideline, and quite lengthy itself. If it isn't a biomedical claim, then for sure some other policy may apply, for example WP:BLP. Do we need to spell that out here? -- Colin°^Talk 16:06, 17 May 2021 (UTC)

Yes, accepted (tho maybe we could find another more "scientific" non-biomed claim?). I also agree that CREEP is a really relevant consideration. My (rather soft) rebuttal on this could be that this was a distinction that had me flummoxed (and to some extent still uncertain, as in the spurious manufacturing technology claim). And I've also seen the distinction raised several times on talk pages. So it sort of feels felt helpful to me, though others of course may disagree it turns out to be even more quirky than I'd thought. [Adding: While fully accepting the community-consensus points raised by Alexbrn and Colin, which I'd never really thought about before, fwiw, I have to admit to my ongoing editorial confusion on the subject - ouch!]. 86.186.168.168 (talk) 17:40, 17 May 2021 (UTC)86.186.168.168 (talk) 17:03, 17 May 2021 (UTC)

oppose spelling it - WP:MEDRS is bloated enough. The WP:SCIRS is already linked from the See also. AXONOV (talk) ⚑ 19:01, 17 May 2021 (UTC)

I agree with the IP's edit summary notes that this shouldn't be a vote. Perhaps Alexander_Davronov could unbold his comment. See Wikipedia:Polling is not a substitute for discussion. I'm not sure how "as in the spurious manufacturing technology claim" helps the discussion: it reads a bit like an accusation of bad faith on my part, but perhaps you mean something else? I agree we could find an example that is clearly scientific but non-biomedical. And we could also find an example that fitted WP:BLP. Is that worth doing here, particularly when we have Wikipedia:Biomedical information to go into detail. For what it's worth, I think the example "They are purple and triangular..." is superfluous, contrived and not encyclopaedic in tone. So that could go. IP, I'm not opposed to mentioning SCIRS with an example, though not much convinced it is needed either, but I think that attempting to find the right words or agree on it being needed is better done on the talk page, than edit summaries.

SCIRS is (quite openly) a copy/paste job of MEDRS with tweaks. Anyone reading it after MEDRS will be frustrated with deja-vu. I agree there are discussions about what is or is not in-scope for MEDRS, but they tend to occur more in conflict areas and get more attention/oxygen than represents actual every day editor confusion. An example of something fitting SCIRS might perhaps be so obvious the reader wonders why they need to be lectured about it, or so hair-splitting that the reader wonders why anyone would care, given both guidelines take a similar approach towards applying core policies. And some things belong so much in multiple domains that many guidelines or policies apply to different degrees (e.g., the whole Covid lab leak debacle is a right old mix of politics, biomedical science, politics, forensics, politics, crime, politics, current affairs and eventually political history). -- Colin°^Talk 20:26, 17 May 2021 (UTC)

@Colin, thank you very much for taking the time to give your thoughts so clearly (something I really appreciate). Believe me, absolutely no "spurious" accusation was intended in my own artless stream of thoughts (misunderstandings being another notorious forum pitfall). You raise several points, some of which are frankly beyond my own limited ken of WP internal culture (not my preferred field). As I tried to indicate above, my intention was to address a real source of confusion that that I've experienced, and that I felt other contributors to MED content might conceivably share. But I seem to have been naive, and I'm also very aware of the invasive potential of CREEP... And, personally, I feel I should step back from this particular question. As regards the fictional Mr Foster and his spurious pills, fwiw, my own take was that it's the key guidance that really needs to be crystal clear (and if humour is a valid tool, so be it?). I agree that WP:BMI is a lengthy page, but again (just speaking personally) I've found it genuinely helpful to clarify current guidance. By contrast, I sort of thinked I understanded the ruole of SCIRS, but now I din't... 86.186.168.192 (talk) 11:03, 18 May 2021 (UTC)

If you are still looking for an example of non-biomedical content, then "The high proportion of unsaturated fatty acids are also responsible for the relatively poor shelf life, as they cause sea buckthorn oil to turn rancid quickly" (sourced to a review article in a journal whose head a Wikipedian disliiked) was once disputed as being promotional(!). WhatamIdoing (talk) 05:42, 25 May 2021 (UTC)

Now, I wish you'd wiki-linked sea buckthorn oil, as I wanted to look that up to see what it was. I'm still not convinced MEDRS needs to contain (lengthy!) sentences that aren't biomedical and have different sourcing needs. The explanatory supplement seems a more appropriate place for dealing with that kind of thing, and MEDRS concentrate on how to source biomedical claims. As an aside, I see the reference you refer to and it does read a bit promotional of this wonder oil. However, the fact claimed is part chemistry, part food/cosmetic retail/consumer issue, and not really a medical one despite being a medical journal. Other than that the author of a medical paper is assumed to be clever, is that really so much better a source than, say, a cookery book? Would editors who care about science facts prefer the source was some chemistry or consumer-product science/engineering source? I suppose the fact that certain fats go rancid quickly isn't controversial. -- Colin°^Talk 09:07, 25 May 2021 (UTC)

I think we assume that chemists know something about chemistry, and that the peer-review process, however flawed it may be, is more likely to uncover scientific problems than the book-publishing process.

I don't think the oil is used much for cooking, so a cookery book is probably not a relevant alternative. It's mostly used in cosmetics. IMO a book about cosmetics (especially if it focused on manufacturing) would also be reliable for this claim. It is not controversial and there's no "medical" content in it.

That article also had a dispute over whether journal articles were reliable sources for the color of the substance, and I believe that at least two editors learned that skin tone is about color and not analogous to medical concepts such as muscle tone. WhatamIdoing (talk) 02:38, 27 May 2021 (UTC)

grantome

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

Is it OK to use grantome.com to cite an NIH grant abstract? Charles Juvon (talk) 15:22, 12 June 2021 (UTC)

It's a search engine returning primary sources, so would be rather odd to use - where's the WP:WEIGHT, would be the question? This sort of question would anyway be better asked at WT:MED (if noticeboard input is wanted); this page is for discussing changed to the WP:MEDRS guidelines. Alexbrn (talk) 17:03, 12 June 2021 (UTC)

The discussion above is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

Unreliable social science research gets more attention than solid studies

See https://www.sciencemag.org/news/2021/05/unreliable-social-science-research-gets-more-attention-solid-studies

Be suspicious of journals with high impact factor. Nemo 12:51, 24 May 2021 (UTC)

Social science is unscientific by definition. Wikipedia is too politically biased to be useful for medics. 124.169.147.24 (talk) 22:57, 22 June 2021 (UTC)

Social and behavioral sciences are not unscientific, they're just not hard sciences. And Wikipedia errs to the side of caution. E.g. a MD commented on Quora that Wikipedia chooses for one therapy method and declares it as the true therapy, while for MDs there are eight or nine different therapeutic options. tgeorgescu (talk) 23:32, 22 June 2021 (UTC)

I fail to see how what you describe is "erring to the side of caution". What if Wikipedia chose the wrong one out of the nine therapies to declare as the one true therapy? Or, even if that one therapy were the best one for a plurality or majority of patients, what harm might be done to those of the minority that read that that therapy is the only option there is, not knowing that there are actually 8 or 9 other therapies available, one or more of which might with better for those individuals? meseemyth that erring on the side of caution would be to AVOID taking any particular hard stand on any particar treatment, and to just present all of the treatments neutrally. Firejuggler86 (talk) 15:15, 3 July 2021 (UTC)