Wikipedia talk:Identifying reliable sources (medicine)/Archive 3

Archive 1

Archive 2

Archive 3

Archive 4

Archive 5

→

Archive 10

Draft replacement for Periodicals, Books, Online

Here is a draft replacement for the sections WP:MEDRS #Periodicals, WP:MEDRS #Books, and WP:MEDRS #Online. It attempts to take all the above discussion into account. It's a bit rushed and wordy, and could stand some improvement, but I thought I'd get it out the door sooner rather than later. Comments are welcome in #Comments on Draft replacement for Periodicals, Books, Online below. Eubulides (talk) 00:59, 21 October 2008 (UTC)

Sections of draft replacement

Each of the sections of the draft replacement are intended to be top-level sections in WP:MEDRS. They are 3rd-level sections here, so that it's nested properly in the talk page. Eubulides (talk) 00:59, 21 October 2008 (UTC)

(start of draft replacement)'

---

Search

Search engines are commonly used to find biomedical sources. Each engine has quirks, advantages, and disadvantages, and may not return the results that you need unless used carefully. It typically takes experience and practice to recognize when a search has not been effective; even if you find useful sources, you may have missed other sources that would have been more useful, or you may generate pages and pages of less-than-useful material. A good strategy for avoiding sole reliance on search engines is to find a few recent high-quality sources and follow their citations to see what your search engine missed. It can also be helpful to perform a plain web search rather than one of scholarly articles only.

Pubmed is an excellent starting point for locating peer-reviewed medical sources. It offers a free search engine for accessing the MEDLINE database of biomedical research articles offered by the National Library of Medicine at the U.S. National Institutes of Health. Although PubMed is a comprehensive database, many of its indexed journals restrict online access; an additional site, PubMed Central, provides free access to full text. There are basic and advanced options for searching PubMed. Clicking on the "Review" tab will help you narrow your search to reviews. The "Limits" tab can let you further limit your search, for example, to meta-analyses or to freely-readable sources.

Other useful search engines include:

• The Cochrane Library contains a database of systematic reviews and meta-analyses, and is a key resource in evidence-based medicine. Its reviews are generally considered to be of very high quality.

• Google Scholar covers all scholarly sources. It is useful not only as a sanity check for Pubmed searches, but also to cover topics outside Pubmed's core purview, such as the sociological or cultural aspects of medicine.

• Google Books can also be quite useful for medical searches: it can let readers peek at a few sentences in books even when full access is not granted, and can help editors find reliable sources quickly, either by looking at the book's references or by citing the book itself.

• EMBASE is a high-quality index that often generates better results than Pubmed. Unfortunately, it is proprietary and requires a subscription.

• CINAHL is a proprietary index on nursing and allied health care.

Accessing sources

A Wikipedia article should cite the best and most-reliable sources regardless of whether they require a fee or a subscription. When all else is equal, it is better to cite a source whose full text is freely readable, so that your readers can follow the link to the source. Some high-quality journals, such as JAMA, publish a few freely-readable articles even though most are not free. A few high-quality journals, such as PLoS Medicine, publish only freely-readable sources. Also, a few sources are in the public domain; these include many U.S. government publications, such as the Morbidity and Mortality Weekly Report of the Centers for Disease Control and Prevention.

When searching for sources, it is wise to skim-read everything available, including abstracts of papers you can't fully access, and use that to get a feel for what reliable sources are saying. However, when it comes to actually writing a Wikipedia article, it is generally not a good idea to cite a source after reading only its abstract, as the abstract necessarily presents a stripped-down version of the conclusions and omits the background that can be crucial for understanding exactly what the source says. You may need to visit a medical library to access the full text, or ask somebody at the WikiProject Resource Exchange to either provide you with a copy or read the source for you and summarize what it says; if neither is possible, you may need to settle for using a lower-impact source or even just an abstract.

Citations as documentation

Citations should document precisely how to access sources. Normally, medical citations should contain a Pubmed identifier (PMID). It is good practice to also supply a digital object identifier (DOI) if available. A common practice is to supply a uniform resource locator (URL) to a source if and only if its full text is freely readable. If the {{Cite journal}} template is used, all this information can be supplied with the "pmid=", "doi=", and "url=" parameters, respectively. It is also helpful to mention whether a source is also available on Pubmed Central; with {{Cite journal}} this can be done with the "pmc=" parameter. For example:

{{cite journal |author= Bannen RM, Suresh V, Phillips GN Jr, Wright SJ, Mitchell JC |title= Optimal design of thermally stable proteins |journal=Bioinformatics |volume=24 |issue=20 |pages=2339–43 |year=2008 |pmid=18723523 |pmc=2562006 |doi=10.1093/bioinformatics/btn450 |url=http://bioinformatics.oxfordjournals.org/cgi/content/full/24/20/2339}}

produces this:

Bannen RM, Suresh V, Phillips GN Jr, Wright SJ, Mitchell JC (2008). "Optimal design of thermally stable proteins". Bioinformatics. 24 (20): 2339–43. doi:10.1093/bioinformatics/btn450. PMC 2562006. PMID 18723523.{{cite journal}}: CS1 maint: multiple names: authors list (link)

If a source is available in both HTML and some other form, normally the HTML form should be cited, as it is the most likely to work on a wide variety of browers.

Biomedical journals

As mentioned above, the biomedical literature contains two major types of sources: primary publications describe novel research for the first time, and review articles summarize and integrate a topic of research into an overall view. In medicine, primary sources include clinical trials, which test new treatments; secondary sources include meta-analyses that bring together the results from many clinical trials and attempt to arrive at an overall view of how well a treatment works. It is usually best to use reviews and meta-analyses where possible, as these give a balanced and general perspective of a topic, and are usually easier to understand!

Peer-reviewed medical journals are a natural choice as a source for up-to-date information for medical articles. They contain a mixture of primary and secondary sources, as well as less technical material such as biographies. Although almost all such material will count as a reliable source, not all the material is equally useful. Journal articles come in many types: original research, reviews, editorials, book reviews, correspondence, biographies and eulogies. Research papers are primary sources; although they normally contain previous-work sections that are secondary sources, these sections are typically less reliable than reviews. A general narrative review of a subject by an expert in the field makes a good secondary source that can be used to cover various aspects of a subject within a Wikipedia article. Such reviews typically contain no original research but can make interpretations and draw conclusions from primary sources that no Wikipedia editor would be allowed to do. A systematic review uses a reproducible methodology to select primary studies meeting an explicit criteria in order to answer a specific question. Such reviews should be more reliable, accurate and less prone to bias than a narrative review. However, a systematic review's focus on answering one question limits its usage as a source on Wikipedia.

The 2003 Brandon/Hill selected list includes 141 journals suitable for a small medical library. Although this list is no longer maintained, the listed journals are of high quality. The core general medical journals include the New England Journal of Medicine, The Lancet, Journal of the American Medical Association (JAMA), Annals of Internal Medicine, British Medical Journal (BMJ), and Canadian Medical Association Journal. Core basic science and biology journals include Science, Cell, and Nature.

Medical textbooks published by the academic press are excellent secondary sources. Ensure the book is up-to-date, unless a historical perspective is required. Doody's maintains a list of core health sciences books, which is available only to subscribers.

Popular press

The popular press is generally not a reliable source for science and medicine information in articles. Most news articles fail to discuss important issues such as evidence quality, costs, and risks versus benefits. Articles in newspapers and popular magazines generally lack the context to judge experimental results. They tend to overemphasize the certainty of any result, for instance presenting a new experimental medicine as the "discovery of the cure" of a disease, or an every-day substance as the "cause" of some dreaded disease. Newspapers and magazines frequently publish articles about scientific results before those results have been peer-reviewed or reproduced by other experimenters. Such articles may be based uncritically on a press release, which can be a biased source. They also tend neither to report adequately on the scientific methodology and the experimental error, nor to express risk in meaningful terms.

A news article should not be used as a sole source for a medical fact or figure. Editors are encouraged to seek out the scholarly research behind the news story. One possibility is to cite a higher-quality source along with a more-accessible popular source, for example with the "laysummary=" parameter of {{Cite journal}}.

On the other hand, the high-quality popular press can be a good source for social, biographical, current-affairs and historical information in a medical article. For example, popular science magazines such as New Scientist and Scientific American are not peer reviewed but sometimes feature articles that explain medical subjects in plain English. As the quality of press coverage of medicine ranges from excellent to irresponsible, common sense and the general guidelines presented in the verifiability policy and general reliable sources guideline should be considered in determining whether a popular press source is suitable for these purposes.

Popular science and medicine books are usually tertiary sources, but there are exceptions. Self-published or books published by vanity presses are generally not subject to any form of independent fact-checking or peer review and may not be reliable sources.

---

(end of draft replacement) Eubulides (talk) 00:59, 21 October 2008 (UTC)

References

Comments on Draft replacement for Periodicals, Books, Online

(Please put comments here.) Eubulides (talk) 00:59, 21 October 2008 (UTC)

Wikischolar citation template generator is now broken, has been for a while. Would be great if someone could re-create it. The text also gives the impression that google books is only good for finding other references, and I have reworded to avoid discouraging the use of books themselves as references; though books should in many disciplines be used slightly more carefully, their use in general shouldn't be restricted (natually, depending on publisher and oversight; that may be missing or assumed in my adjustments).

My apologies if I've stepped on your toes Eubulides, I've made some other adjustments to the text to place more emphasis on the reliability over the accessibility (I feel very strongly that this is the best way to write an article). If people disagree, then please go ahead and revert to the original wording and discuss!

Another comment - "if neither is possible, you may need to settle for using a lower-impact source or even just an abstract" - this actually discourages writing articles in my mind. I very rarely have access to the full text, and often am insufficient of an expert to judge an article based on the specifics of methods and statistical analyses. I've written many an article based on abstracts alone, and have received few complaints (but given the dearth of expertise on wiki, perhaps that says something about the system rather than my writing). An abstract should normally include the most salient conclusions, and generally any significant methodological flaws, or follow-on sources should raise these issues rather than editors.

I'm also of the opinion that discarding an article or placing qualifications on the mainspace page regards it's usefulness (i.e. "...but these conclusions should be regarded with caution because of a lack of control group.") is very dangerous grounds, very close to original research, and am dubious about its inclusion. We're not Nupedia with expert oversight and the ability to criticize conclusions, and to enter into this territory sets a shaky precedent in which we'll have another layer of unofficial expertise or authority - regular users, admins, and "those who can discard studies on the basis of methodology". TimVickers, Eubulides and many of the heady powereditors who are regulars on MEDRS I would trust with this judgement, but it is begging, just begging for trouble as soon as a content dispute arises where one side has a very strong POV. These are comments about articles that I think should be made by verifiable sources, not by us, and if it's a serious flaw then very shortly after the original publication a qualification, retraction or criticism should be published by a relevant expert in a relevant field. Just my thoughts, grist for the mill, and I'm always delighted to see what the excellent contributors on MEDRS are going to do to destroy my wikiworldview this week. You know for the longest time I didn't know there was a difference between policies and guidelines? WLU (t) (c) ^{(rules - simple rules)} 01:11, 21 October 2008 (UTC)

WLB, if you look above on this page to Steps towards consensus, you will see that we have engaged in a discussion lasting over a month, with an informal mediator, on the issues you bring up now. Please take the time to read through the reasoning of the participants. It has been very hard on us all and it would indeed be disappointing if it were all for nothing and we were to start over at square one. Regards, —Mattisse (Talk) 02:27, 21 October 2008 (UTC)

oops, my bad. I will read tomorrow if I've the time and until then will refrain from edits and suggestions. Please feel free to revert all changes including my above comments. I had thought this a separate issue. WLU (t) (c) ^{(rules - simple rules)} 02:51, 21 October 2008 (UTC)

No problem. Your changes had good ideas. I took the biggest point and moved it into a new section #Citations as documentation above. I also edited the changes further. Thanks! Eubulides (talk) 06:13, 21 October 2008 (UTC)

@WLU: Making improvements to a page is a good thing. Just edit the page like normal. :-) That's the whole point of wikipedia, and it's what we've been working towards. If anything happens (I hope it won't, but it might), drop me a line and I'll help sort it out. --Kim Bruning (talk) 17:49, 21 October 2008 (UTC) The rule on wikipedia is: someone has to do the consensus homework, else you may have ...some... trouble finding consensus at times. ;-) And I'll admit I'm rebelling a little: why should it always be the (informal) mediator who gets to do all the work? People should learn to do the homework themselves for a change! That way the workload is spread out over several people, and the mediators job is a lot lighter. In fact, why need a mediator at all? A stitch in time saves nine: If you try to understand the current consensus _before_ you get into an edit war, rather than after, well sure, it might cost you a couple of hours (only at first, with practice you can get it down to a couple of minutes), but it will save you a month of torture at the hands of an evil mediator! O:-)

PubMed and PubMed Central

Above suggested wording states PubMed Central as an "alternative" of PubMed - but in reality both hosted by NLM. They each provide a different webpage to search the database and whilst one shows PMID abstracts (which will mention any PMC links if available), the other only shows archived copies of articles. "additional" resource tool rather than "alternative" might be a better choice of wording ? David Ruben ^Talk 00:13, 22 October 2008 (UTC)

Thanks for catching that. I changed "alternative" to "additional" in the above draft. Eubulides (talk) 07:34, 22 October 2008 (UTC)

I'm old fashioned, I like to use history and diffs and other such wiki tools. None of this new fangled "ink", "draft" and "dead tree" stuff for me ;-). Could you do l'il old me a favor and add the changes to the page itself, so I can check them in my own old fashioned way? O:-) --Kim Bruning (talk) 01:47, 23 October 2008 (UTC) try split it into multiple edits, one paragraph at a time, perhaps.

The proposal has only been out for a couple of days; I'm a bit leery to install it now. But if you want to see the diffs, and the new version, please check here and here, respectively. The diffs aren't that useful, I'm afraid; the change is too big. Eubulides (talk) 03:05, 23 October 2008 (UTC)

That's why you make the change in steps, so that your diffs actually make some kind of sense. :-) --Kim Bruning (talk) 10:15, 23 October 2008 (UTC)

But that's for next time. I've unreverted you. Let's see what edits people make. :-) (I found some odd stuff in a hidden text, so I made it slightly friendlier already, I haven't touched visible text yet.) --Kim Bruning (talk) 10:20, 23 October 2008 (UTC) I hope you won't unrevert-war with me. That'd be a novelty! ;-)

Yes, PubMedCentral is not an alternative to PubMed in any sense at all; PMC is an archive of articles posted as being the result of mostly NIH sponsored investigations from about 2007/8+, with some earlier--its a wonderful thing to have these open access articles and will become more and more useful, but they are not a complete record of anything. PubMed provides abstracts only for all published articles in biomedicine. The place to search for material is unquestionably PubMed, using PMC as a convenient source of full articles for the ones it has.DGG (talk) 05:17, 23 October 2008 (UTC)

PubMed Central manuscripts vs final versions

Consider the following citation:

Turetsky BI, Kohler CG, Indersmitten T, Bhati MT, Charbonnier D, Gur RC (2007). "Facial emotion recognition in schizophrenia: when and why does it go awry?". Schizophr Res. 94 (1–3): 253–63. doi:10.1016/j.schres.2007.05.001. PMC 2571079. PMID 17583481.{{cite journal}}: CS1 maint: multiple names: authors list (link)

This uses the "pmc=" parameter of {{Cite journal}}, so that if you click on the article title, you visit the PubMed Central manuscript for that article. This manuscript, though, presumably differs in minor ways from the final version of the article (which is viewable only with a subscription).

So, my question is: is such a citation a reasonable one? Or should it be altered to make it clear to the reader that the URL is to something other than the final version of the paper? There is an argument for convenience and for encouraging people to read freely-available versions of papers; but there is also an argument for making it clear which is the final (and presumably best) version of the paper.

Should WP:MEDRS have some guidance about this issue?

Eubulides (talk) 20:45, 23 October 2008 (UTC)

See Author Manuscripts in PMC. I think the differences should usually be trivial, possibly on a par with the difference between the PDF and HTML forms. But there is a risk that final editorial changes were significant. Where both are free, this implies that editors should be recommended to read and cite/link the journal's final edition and not include the "pmc=" version. If the PMC is the only free edition, but supports the text of the article, then I'd recommend using that. Does anyone know how much they tend to differ? It doesn't seem worthwhile to link to a paid-for version merely because it has had a final touch of copyediting. Colin°^Talk 21:43, 23 October 2008 (UTC)

I do think though that journals own versions of articles (assuming free to access) tend to have better inclusion of tables and charts than the version rendered at PMC. {{cite journal}} therefore links the title to the url parameter if given, rather than the pmc value if it is given too (i.e. the pmc link will do, but url to "original" journal takes preference) David Ruben ^Talk 00:24, 24 October 2008 (UTC)

I would not link the "author manuscript" with the PMC tag. There may be significant differences between the submitted manuscript and the final version that has undergone extensive peer review. JFW | T@lk 00:32, 24 October 2008 (UTC)

PMC is not necessarily manuscripts; a number of publishers deposit full versions, formatted in PMC's special XML. And when PMC has manuscripts they have invariably undergone full peer review--the difference is the lack of publishers' copyediting. There is consensus, though, in the academic community that the published version is the one for official citation--and consensus on WP that we always also give a convenience citation to the free version if available. Incidentally, as for the differences see my "Open access and accuracy: author-archived manuscripts vs. published articles " in Learned Publishing, 20:3, July 2007 , pp. 203-215 open access.and the refs listed therein. They are very rare--nobody has every demonstrated anything substantial. DGG (talk) 01:05, 24 October 2008 (UTC)

That's a very useful article! We can even reference it ;-) --Kim Bruning (talk) 02:29, 24 October 2008 (UTC)

Wow. I've never seen that happen before in an article, never mind a guideline page. I asked "Does anyone know how much they tend to differ?" and and within a few hours an editor has responded (effectively) "Well, actually I'm a expert on this topic and here's the freely available published/reviewed answer." Great.

BTW, I find it deeply ironic that Wiley's Epilepsia want me to pay $35 to read their open access policy. Colin°^Talk 08:10, 24 October 2008 (UTC)

Heh. It's not as common as it used to be. --Kim Bruning (talk) 16:33, 24 October 2008 (UTC)

Thanks, I made this change to cover the topic and cite DGG's paper. But there's a tasty irony here. As far as I can tell, Ingentaconnect, the publisher of DGG's paper, does not provide a stable URL to the full final version of the paper. Instead, one must go through a couple of levels of indirection, and that gives you a URL with session info in it, compromising both portability and privacy (so much for "open access"). My edit therefore contains this citation:

Goodman D, Dowson S, Yaremchuk J (2007). "Open access and accuracy: author-archived manuscripts vs. published articles" (PDF). Learn Publ. 20 (3): 203–15. doi:10.1087/095315107X204012. Retrieved 2008-10-24.{{cite journal}}: CS1 maint: multiple names: authors list (link)

Here the DOI is to the final version, but the URL http://dlist.sir.arizona.edu/1968/01/OAandA%5FGDY.pdf points instead to the authors' preliminary manuscript. I sure hope that the paper's results apply to the paper itself! Eubulides (talk) 08:59, 24 October 2008 (UTC)

How about modifying the cite journal template so that it tells the reader they're getting a rough copy? ☺ Coppertwig (talk) 00:40, 26 October 2008 (UTC)

Journals

It seems unfortunate/unbalanced for Wikipedia that there isn't actually a central guideline covering the various journal database and access issues above (unless I've missed it). To cover all the main databases, for different editors and points of view (pubmed of course is misleadingly titled/described 'cos it includes a large number of non-medical journals). Which could then split off into specific databases/topics if length dictated it. Avoiding cart before horse or tail wagging dog. EverSince (talk) 11:27, 24 October 2008 (UTC)

I look forward to reading your new page! :-) --Kim Bruning (talk) 16:28, 24 October 2008 (UTC)

Just found in this dispatches reoport on reliable sources, hidden in the basement.. Wikipedia:Wikipedia Signpost/2008-06-26/Dispatches.. 15:12, 25 October 2008 (UTC)

Also Dispatches 2008-06-30: Sources in biology and medicine and Dispatches 2008-07-28: Find reliable sources online which as per talk page of the former were initially one guide but then split for dispatch purposes. I asked at citing sources a few months ago about where an actual guideline along those lines might go, but no response. EverSince (talk) 16:21, 25 October 2008 (UTC)

I don't think we should restrict references to particular databases, but I personally would strongly object to the citation of articles from journals not indexed on Medline. I have a preference for freely accessible sources, but if there is a better source that is not freely accessible then the second one still wins... JFW | T@lk 18:58, 25 October 2008 (UTC)

there are 3 types of sources not indexed on Medline: one is journals not regarded as relevant to scientific medicine, a second is local journals outside the US, and a third is journals that have material relevant to medical topics, though in other sciences. None of them can be assumed to be not freely available. The problem in RS of course is for the first when one deals with fringe medicine: the major alternative medical journals that have some pretense to science are included, the lesser ones and the frankly non-scientific are not. In practice, we have been using inclusion in Medline a criterion of at least possible suitability for citing in this topic when one wishes to cite a possibly rational approach--if one wishes to cite what people believe, the spectrum is of course much wider. Local non US/Canada journals can have a high standard, but this cannot be assumed--a rough check is whether they are included in Excerpta Medica, even though that actual database is not widely available outside major medical schools. And journals in any area of physical or social science or humanities can be relevant--an example is that the journals dealing with medical ethics are included in Medline; the general philosophical journals are not, though they may have articles on the subject of just as high quality.

But this does not say anything about availability: many less-developed worlds journals are available open access, hough not in Medline. and many that are do not have even the abstracts freely available--this is particularly true for review articles. and I hope everyone realises that many open access versions are available outside PMC--the only way of getting most of them all is to check Google Scholar or Scirus as well.DGG (talk) 02:54, 27 October 2008 (UTC)

At the end of the day, Medicine and Medline represent only part of the knowledge that may exist for a given article[1][2] - including potentially at the scientific core of a subject. I know those aren't necessarily free to access, I'm just trying to illustrate. EverSince (talk) 20:53, 27 October 2008 (UTC) p.s. this point has of course been well made already by Kim's pointer to the citation diagram[3] EverSince (talk) 20:59, 27 October 2008 (UTC)

The above points about non-medical journals/articles potentially being relevant (whether or not indexed on medline, and even if they are they may also be accessed/linked via a non-medical database) seem to question how this guideline can legitimately only refer to medical journals/textbooks in the lead... EverSince (talk) 16:17, 1 November 2008 (UTC)

(watches the tumbleweeds go by)...I guess the above issues about the lack of a neutral cross-disciplinary guideline on finding/assessing articles, and the illegitimacy of this guideline being founded on a priori privileging of reliable medical sources over reliable non-medical sources, can be added to the list of critical issues that remain unaddressed/ignored. Another one is that a sourced critique of press sources is included in this guideline - tendencies, things they miss out, exaggerate etc - but not a similar critique of medical sources. Here's a long list of the sorts of points that might be relevant, not saying these specific sources are necessarily ideal but they seem to support the presence of widespread issues & tendencies equivalent to those covered for press articles:

In the 150 years since Virchow produced his principles, medicine has strayed from this vision [20]. Despite television images of trapped hurricane survivors searching for food and the knowledge that 44 million Americans (most of them working) do not have health insurance, medical research continues its biomedical trajectory in search of expensive “magic bullets” and more sophisticated interventional technologies, rather than understanding the social determinants of health...Why then is social science often excluded? For one, academic medicine has been preempted by the glamour of technology and by the rewards it brings to those who discover and employ it [22]. For another, social scientists are unwelcome when they discover unpleasant facts, such as life circumstances trumping medical care in determining the health status of populations and that disparities in health care are part of the system rather than oversights [1]. These discoveries not only threaten medical hegemony, but they challenge the larger social order[4]

If structural violence is often a major determinant of the distribution and outcome of disease, why is it or a similar concept not in wider circulation in medicine and public health, especially now that our interventions can radically alter clinical outcomes? One reason is that medical professionals are not trained to make structural interventions. Physicians can rightly note that structural interventions are “not our job.” Yet, since structural interventions might arguably have a greater impact on disease control than do conventional clinical interventions, we would do well to pay heed to them.[5]

When 'medical facts' must be derived from 'magical beliefs' in the centres of biomedical science, the state often intervenes to criminalize practitioners of alternative medicine. But, when profits are to be made on the fact that 'the magical' sells in alternative medicine, the state also makes it possible to shift ownership of medical knowledge, sometimes by way of the randomized controlled trial and the pursuit of active ingredients. The possibility of relocating the label of 'crime' is explored in this paper by way of an inquiry into processes that enable this shift in ownership, and a relocation of what constitutes medical 'fact' versus 'belief'.[6]

studies have found high error rates in references in biomedical journals[7]

Selective reporting of trial results exists for commonly marketed drugs[8]

[Medical anthropology] use[s] the voices and stories of the poor to explain the impact of structural adjustments...I argue that [this] masks the real absence of the voices of the poor and their suffering on the world stage. There is no international public sphere within which these voices might be heard; rather, there is a set of claims about justice and human rights.[9]

[in] contested areas of science-society discourse...militaristic metaphors are still part of a pervasive, but by no means inevitable, mode of science and policy communication.[10]

disease in the Western world has been primarily constituted by the imagery of war...it derives its immediate nourishment and historical root from a corresponding language of war within medicine itself[11]

Medical journals are an extension of the marketing arm of drug companies[12]

The eight questions that I pose in the opening paragraph – having to do with blindness and deafness, mild depression and slow running, black racial features and plain facial features, obesity and anorexia – have become, even in today’s world of limited resources and imperfect technology, principal lightning rods for debate over the final limits to medicine ... treatments for conditions such as deafness, obesity, or anorexia may actually be a form of cultural genocide[13]

Under the Medical Gaze serves as a powerful illustration of medicine's power to create and inflict suffering, to define disease and the self, and to manage relationships and lives.[14]

Three decades ago, Ivan Illich argued polemically that the medical establishment was “medicalising” life itself [8], and in the 1990s Lynn Payer described widening the boundaries of illness as “disease mongering” [3], highlighting the role of pharmaceutical companies. Today's debate about this phenomenon, while still maturing, both acknowledges the axiomatic interest of corporations and professionals in maximizing turnover and appreciates that well-informed citizens may choose to embrace the medicalisation of health problems previously regarded as troublesome inconveniences.[15]

The unequal distribution of power in contemporary society is reflected and reproduced in medical ideology. The present article analyses some articles from Israeli medical journals in order to show the ways in which biomedicine—the dominant medical ideology—is reinforced through hegemonic discourse. The central ways by which this is achieved are medicalization—which includes the desocialization of disease and the explanation of social phenomena in medical terms—and the affirmation by the Israeli medical literature of national, ethnic, class and gender relationships of domination. Analysis of the Israeli example provides useful insights about biomedicine's desocializing role, as the disregard for the social dimension of disease is particularly telling in a society characterized by several cleavages which determine a clearly unequal distribution of power and resources.[16]

"Social conditions such as poverty and crime have all too often been attributed to mental illness, biology, and genetics. This has created a kind of nihilism on the part of mental health professionals, which has proven to be an impediment both to research and health care delivery. These deterministic views of race-based psychology have continued to appear in the literature and continue to exert a pernicious influence."[17]

the dominance of biomedicine is delegated rather than absolute, these processes reflect the growing accommodation on the part of alternative practitioners to the reductionist disease theory which is compatible with capitalist ideology.[18]

The "Medical-Industrial complex" has led to the commercialization of health care well beyond what traditional practitioners would consider ideal. Medicine is being treated as a business, with cost curtailment measures and profit margins often dictating physicians' choices[19]

the transformation of continuing medical education into an enterprise for drug marketing. The chore of teaching doctors how to practice medicine has been handed to the pharmaceutical industry.[20]

By accepting only advertisements for drugs and medical devices, medical journals have accepted an exclusive and dependent relationship with corporations... Advertisements and other financial arrangements with pharmaceutical companies compromise the objectivity of journals.[21]

pharmaceutical industry organizations often front for sections of the pharmaceutical industry by retaining public relations consultants and sometimes by coordinating the funding of psychiatric research. An important line of agenda setting communication is via a two-way exchange between PR/advertising and the psychiatric profession/researchers. Public relations firms are often used to disguise the sources of drug company funding for psychiatric research and there is also a constant barrage of drug company advertising in psychiatric journals aimed at persuading psychiatrists to use specific drugs. The reverse flow, from psychiatry to PR, involves the utilization of psychiatric research, and psychiatric expertise, in public relations campaigning.[22]

the erosion of professional values and medical education by commerce shows no sign of slowing. The latest scandals involve Medscape[23]

One of the nation’s most influential psychiatrists earned more than $2.8 million in consulting arrangements with drug makers from 2000 to 2007, failed to report at least $1.2 million of that income to his university and violated federal research rules, according to documents provided to Congressional investigators.[24]

Leading medical journals seem to be having a difficult time disentangling themselves from the pharmaceutical and medical device industries. If they cannot stop printing articles by scientists with close ties to these businesses, they should at least force the authors to disclose their conflicts of interest publicly so that doctors and patients are forewarned that the interpretations may be biased.[25]

residents indicate that they do not have the time nor the mentors to deliver effective cross-cultural care, and are not evaluated on their ability to do so[26]

both EBM [Evidence-Based Medicine] and CCM [cultural competence in medicine] have been accused of fostering stereotypes and have therefore suffered some degree of backlash. EBM experienced a backlash over fears of “cookbook medicine,” or clinical stereotyping, while CCM experienced a backlash over fears of cultural stereotyping.[27]

the issue about the predominance of the neurosciences has less to do with truth than with the current economical and legal climate and hence it is imperative that other approaches to mental illness are allowed to participate in the process of understanding..." "Indeed, it could even be said that it would be unethical not to allow other approaches to participate in the common epistemological enterprise. Without such wider conceptual participation, the haematoma auris or the drapetomania stories will happen again."[28]

EverSince (talk) 21:34, 23 November 2008 (UTC)

Too long, didn't read.

The fact that there isn't a neutral guideline to reliable sources in multiple fields is a red herring. Nothing in this guideline -- I repeat, nothing -- prohibits the editor from using any and all types of reliable sources for non-scientific information. It's only when you're talking about actual scientific facts, such as whether this molecule interacts with that molecule, or whether this medical sign is associated with that medical condition, that we're recommending that the editors rely on medical/scientific sources instead of, for example, a newspaper article written by a journalist that had never heard of the condition until he started the interview with a local patient.

Can you give me an example of a real problem? For example, can you type up an example of something you'd like to include in an article, with a non-scientific source that you'd like to cite, but which you think is deprecated under this guideline? Or are there no actual problems in practice, but you just don't feel like this guideline sufficiently represents your personal POV about the field of medicine in general? WhatamIdoing (talk) 19:53, 24 November 2008 (UTC)

I understand that intention regarding the press, and I'm not talking about deprecating. Those aren't my words, they're quotes from sources. I don't think the lack of a more comprehensive guideline to help editors find/assess sources is irrelevant. The real problem is that this guideline frames itself as about medicine and the medical, and it privileges medical/biomedical journals as "ideal" or "natural", and it guides people towards medical standards & databases. But then there's this bait and switch(caption bill?) (I don't mean in the deliberate advertising sense, just logical structure) where the terms "science" and "scientific" get conjoined to medicine & medical, and thus the implicit claim that there are medical sources which constitute the scientific facts (not necessarily, in fact), and then the "non-scientific" - actually a vast realm of non-medical science and factual perspective (as sourced above). EverSince (talk) 11:00, 25 November 2008 (UTC)

So you haven't actually encountered a situation in a live article in which this guideline, as written and reasonably interpreted, is actually causing any problems? This is just a hypothetical, somebody-might-misunderstand issue? WhatamIdoing (talk) 18:52, 25 November 2008 (UTC)

No it's not about potential misunderstanding of what the guideline says, it's bias in what it actually does state - which several editors in various ways have repeatedly identified over a long period of time now and which I've again tried to outline and source above. Are you disagreeing that it's biased, and if so how, or are you suggesting it doesn't matter? It's not necessary to sidetrack into trying to establish examples of institutional bias in order to address its presence in policies (and also, btw, in the favored citation template that doesn't accept DOIs & doesn't really seem to work with URLs, but only Pubmed IDs for articles). EverSince (talk) 11:58, 26 November 2008 (UTC)

Generally, I favor writing guidelines so that they can be correctly interpreted by an idiot, because if you're writing, for example, a school weapons policy, it's pretty safe to assume that at some point there will be a pointy-haired idiot interpreting the policy. (True example: most American schools have a weapons policy that technically requires them to report to the local police department that their own police officers carry duty weapons every single time any officer sets foot on campus.)

However, in this case, I'm leaning towards a "ain't broke/don't fix" approach. The guideline does not supersede the normal rules. It is meant to provide helpful advice in limited circumstances. Its limitations are apparently clear to everyone, because nobody has reported any actual problems. So I don't see any point in adding a paragraph that explains what everyone already seems to know (e.g., that government policies towards people with mental disorders aren't scientific facts or figures). WhatamIdoing (talk) 17:58, 26 November 2008 (UTC)

I agree guidelines should be as simple as possible. I note it has previously been alleged that this entire guideline represents instruction creep by trying to detail how to implement policies. I understand from what you are saying that you don't think it does so in a biased way, but is at most limited, in ways that are appropriate for Wikipedia and not significant. I don't understand the last sentence as I do'nt see who has suggested adding a paragraph along those lines.

While that may be your view, it doesn't specifically address/refute the specific points that have been raised and reasoned and sourced, regarding bias and policy conflicts in the guideline (e.g. the privileging of medical journals/databases over non-medical ones; outlining typical flaws in press coverage but not about typical flaws in the medical literature). I don't see how it can be said that correcting those would necessarily make the guideline more complicated. And again I think it is unreasonable to now suggest that examples of problematic impact are required before something in the guideline can be changed (and I'm not thereby saying there aren't any). EverSince (talk) 20:33, 26 November 2008 (UTC)

Sorry, but that last point doesn't sound right to me. Every change I've made to this project page has been motivated by a real problem in a real article, a problem that I could (and often did) point to when explaining why the change was useful. As far as I can tell, no specific changes are being proposed now, if any are eventually proposed, I suggest accompanying them with specific examples of why the changes are useful. Eubulides (talk) 21:12, 26 November 2008 (UTC)

Is the issue that these guidelines are biased in favor of the medical literature? Press reports are used for sources in medical articles by well meaning editors. An explanation of why news reports are not good sources for medical statements addressed that frequently found problem. To get into flaws in the medical literature, methodological or whatever, is a huge topic unto itself. The guidelines do address possible misuse of medical literature by distinguishing between primary, secondary and tertiary sources, as well as stressing the importance of using recent sources, review articles etc. —Mattisse (Talk) 22:04, 26 November 2008 (UTC)

Problematic article

The Aspartame controversy article has some pretty lousy self-published or non-PubMed sources. It needs a good thinning. The whole controversy was started by one woman, and she's the one who keeps it alive, but she's not a reliable source in any sense, and her supporters (a few radical fringe MDs) aren't good sources either. -- Fyslee / talk 02:19, 1 November 2008 (UTC)

This "one woman" claim is blatantly not true, the controversy has been going on since the early 1980s. By the way, the article is about a regulatory and political conflict. This article is only partially about medicine/science: NPOV trumps SPOV. MaxPont (talk) 17:58, 19 November 2008 (UTC)

Well, that shouldn't be an issue: news sources discuss the regulatory/political issues, whereas scientific sources are most appropriate to discuss the scientific issues. Seems pretty straightforward. MastCell ^Talk 19:59, 25 November 2008 (UTC)

Therapeutics Initiative

Wondering what others think about the Therapeutics Initiative as a reference source for medical articles. Another editor disagrees with its use. I presume this would be classified as a secondary source.

http://ti.ubc.ca/en/letter69

--Doc James (talk) 22:10, 31 October 2008 (UTC)

A quick reaction: that source looks reliable. It's peer-reviewed and it is a secondary source. However, it's not a medical journal in the usual sense (it's not PubMed-indexed, for example), and it would be better to cite its sources directly. Its References section looks good. Eubulides (talk) 22:21, 31 October 2008 (UTC)

I assume that you mean "it would be better to actually find and read and cite its sources directly"? WP:SAYWHEREYOUGOTIT still applies. WhatamIdoing (talk) 01:33, 1 November 2008 (UTC)

Yes, that's what I meant, and thanks for clarifying. Eubulides (talk) 02:02, 1 November 2008 (UTC)

A researcher and university professor (ICBSeverywhere) has taken issue with an article from this source and written at length finding fault with this citation and the called it misleading. She also stated, The "Therapeutics Initiative" IS an anti-pharm activist group. She posted once near the bottom of the thread and I quoted a personal e-mail about the same topic up a little higher in the thread. http://wiki.riteme.site/wiki/Wikipedia:Mediation_Cabal/Cases/2008-10-09_Attention-deficit_hyperactivity_disorder#Lets_resolve_one_issue_and_go_from_there Ironically, this issue came to light at Doc James Med Cab. http://wiki.riteme.site/wiki/Wikipedia:Mediation_Cabal/Cases/2008-10-09_Attention-deficit_hyperactivity_disorder --scuro (talk) 05:40, 11 November 2008 (UTC)

it's a tertiary source primarily summarizing other reviews. The members of the review panels do not impress me as necessarily being authorities--and they say they use the same panel for all drugs whatsoever. [29] DGG (talk) 18:17, 4 December 2008 (UTC)

sources that are only available through paid subscriptions and not mentioned anywhere else

I have found sources that are only available through paid subscriptions. When one does a google search no other mention of the article can be found beyond the links to subscription service. Is that an acceptable source?--scuro (talk) 05:45, 11 November 2008 (UTC)

That depends. "Subscription" is somewhat orthogonal to "reliable". "Not mentioned anywhere else" has a weird whiff, but could be OK. Can you be more specific? Eubulides (talk) 06:14, 11 November 2008 (UTC)

Subscription journal articles are invariably available free in major libraries, and can almost always be obtained in interlibrary loan through any public or school library. They have always been accepted as references in Wikipedia, just as printed books have. WP is not an encyclopedia limited to things one can research without using libraries. There are a few cases of purely electronic subscription-based services that are not available otherwise, and these might have some problems--the situation also occurs with some financial services and newsletters, especially market research newsletters, which are in practice not available to the public in any reasonable way. If these were proposed as references they would have to be individually considered--in some cases we might simply not regard the material as actually published or reliable. But journal articles, it does not make any difference--be sure to cite the printed version also, which you can get from the reference, or, if not, from PubMed. DGG (talk) 09:25, 23 November 2008 (UTC)

It is a difficult issue. Quite often the results of journal articles are taken out of context and used in a misrepresentative way in an article. I find this commonly in the use primary sources in medicine-related articles, but also with review articles when I have access to the original. I do not know what the solution is but I think it is major problem. I do not have access to off-line library sources, not being a student and working hundreds of miles from land. —Mattisse (Talk) 19:17, 25 November 2008 (UTC)

Just ask someone with full-text access to the citation to verify the claims made on Wikipedia. A lot of editors have this sort of access; you could ask at WT:MED as a starting point. MastCell ^Talk 19:56, 25 November 2008 (UTC)

Hmm, thanks, WP:MEDRS suggested only WP:LIBRARY for that; I made this edit so that it now also suggests WT:MED. Eubulides (talk) 20:10, 25 November 2008 (UTC)

Yes, although in practice I have never found sourcing help there. —Mattisse (Talk) 21:38, 26 November 2008 (UTC)

a montage of correspondence, on reviews and citable sources

I've been corresponding with a researcher about the topic of reviews. I've collected her thoughts and organized them, taking ideas from different e-mails and stringing them together. Hopefully I have done her justice. She has broadened my knowledge about quality of sources and I thought some may find what she has to say of interest.

Review articles just review the literature, so they are not better sources than research reports; they are just a good place to get a general overview of what's new (when they are good). Research findings are primary sources, so they are always superior. If you can read the report, you can evaluate the validity of the finding first-hand. Reviews are hearsay. Reviews are excellent sources - when they are published in peer-reviewed academic journals. In my field, for example, there are entire journals devoted to this kind of work. They have names like "Current Directions in..." and "Annals of..." and are published by organizations like the American Psychological Association and the Association for Psychological Science. Any literature is static, too, and reviews are meant to summarize the recent literature.

The problem is that the term "review" could be used to describe any piece of crap. Anyone can WRITE a review. Not just anyone can get a literature review PUBLISHED (at least in a "real" journal). Everybody does literature reviews as part of graduate education, but published review articles are written by individuals that have published several important findings in the field, especially recent ones. The quality of a review varies just like studies, but they are usually good when published in appropriate journals. There are journals that will publish any article if you pay them. They are academic & sometimes peer-reviewed, but not of high quality. There are journals created by activist groups. They are not academic, not peer-reviewed (by the usual definition of "peer"), and usually qualify as propaganda. NOBODY uses these in science.

Websites are never appropriate in my opinion, unless it's a government organization like NIMH or NSF. Anybody can say whatever they want on a website. Statements and summaries by the NIMH and NSF are some of the best information available with the most thorough and strongest citations. They are not technically the kind of literature review that I described above, but they are reviews in a loose sense.

She would most likely speak to this if there is enough interest.--scuro (talk) 21:58, 29 November 2008 (UTC)

The problem with primary sources is their relative impact. Reviews are excellent in separating chaff from corn; they string together the findings from primary sources into useful theories that can then be tested. In diseases where numerous hypotheses exist, review articles can balance the various hypotheses against each other and consider their plausibility.

Wikipedia can't engage in these tasks, and reviews tend to be much more useful in writing encyclopedia articles. I'm not sure whether your source is speaking specifically about the suitability of sources for an encyclopedia.

I have stated before that the choice of review is an editorial decision. That's because some reviews are pieces of stunning brilliance (usually those in the core journals, by avowed experts with wide clinical experience in the subject), and some reviews are pieces of stunning lunacy (usually in smaller journals and by semi-controversial characters who ride their hobby horses). JFW | T@lk 22:40, 29 November 2008 (UTC)

I've never been a fan of primary sources in wikipedia medical articles. Furthermore, like the vast majority of wiki users, I don't have the skills to, "evaluate the validity of the findings firsthand". On the otherhand do the vast majority of users have the skills to judge the quality of a review? As noted, quality varies from "crap" to "stunning brilliance". Yet, reviews are labeled as ideal sources. Newspaper articles could be more reliable then some reviews. If Wikipedia is an encyclopedia that anyone can edit, shouldn't the article, Reliable sources (medicine-related articles), have better guidelines for contributors to determine the quality of a review?--scuro (talk) 01:22, 30 November 2008 (UTC)

• The advice from the researcher who criticized reviews and preferred primary sources is good advice, if you are a researcher. But it is not good advice if you are a novice in the subject, that is, if you are a member of Wikipedia's target audience.
• I agree that it would be better if WP:MEDRS gave good advice about assessing the quality of a review. Its current assessment section, WP:MEDRS #Assess evidence quality is not that useful for that purpose: that section's subject is mostly primary studies, which articles shouldn't be emphasizing anyway. I would support a rewrite of WP:MEDRS #Assess evidence quality to make it more useful to Wikipedia editors (WP:MEDRS's target audience). Of course this will take some work. ...

Eubulides (talk) 06:57, 30 November 2008 (UTC)

Agree with Eubulides, though would like to clarify that the advice is only good for researchers doing research. If researchers want to write for Wikipedia, they would be advised to choose the same type of sources as everyone else. Read the primary literature, sure, but read and cite the secondary literature on Wikipedia, as a general rule. That's the way we can confidently say "X is a common cause of Y.(Smith 2008)". Editors using primary studies may produce horrible non-encyclopaedic prose like "An analysis of 200 patient histories for all hospital admissions in Essex for Y in 2005, showed X to be a common cause.(Jones 2006)" Colin°^Talk 09:07, 30 November 2008 (UTC)

Here's the problem with the arguments given by Eubulides & Colin: The goal here is to produce an ACCURATE Wiki entry, not put together a list of "further reading"! How easy a work cited is for the audience to evaluate is entirely IRRELEVANT. Citations are evidence that supports a statement made. Good citations do so. Bad citations do not.

The only reason to discuss the criteria of a "good source" is to make it easier for people who are EDITING the entries to quickly eliminate some statements made (without reading every source first-hand).

Reviews are filtered through the reviewer, so they are ALWAYS hearsay. Research-reports are first-hand, so it is ALWAYS better to cite them. That does not mean that the research is always GOOD; it simply means that the source is less biased than a second-hand source.

ALL editors should cite sources that support their statements, and be prepared to defend those sources, PERIOD.

If you are not able to evaluate your OWN sources, then you should not be contributing to the Wiki entry in the first place. If you are unable to evaluate the sources of other contributors (either yourself or with the help of someone with more expertise in a topic), then you shouldn't criticize it. ICBSeverywhere (talk) 18:33, 30 November 2008 (UTC)

I think you misunderstand the reason why we place so much emphasis on reviews. You are quite wrong that reviews "must be hearsay". Quite the opposite, reviews filter out the "noise" (especially in heavily researched areas, where lots of research leads to dead-ends or is of questionable quality). We cannot allow discussions to get bogged down by claims of one study vs the next - contradictory results are obtained all the time, and reviews are exactly the tool to deal with such phenomena.

I agree that sources need to support their statements, but a corollary of this is that if there is no reliable source (i.e. a review) we must be prepared to drop a certain subject, however much it tickles our collective fascination.

You have very few article edits, but you seem to be fishing in the same pond as Scuro. Could you clarify what the disagreement is about, and how any of us could assist in dealing with this disagreement? JFW | T@lk 20:07, 30 November 2008 (UTC)

I think what is misunderstood here is my use of the word "hearsay". Reviews are reports about the work of others - secondary sources. Your statement that reviews "filter out noise" is a gross overgeneralization, implying that all literature reviews are equal. They are not.

Literature reviews are NOT a way to reconcile findings that contradict one another, although the author may clarify why findings appear to contradict one another, saving a reader some work. Meta-analysis is a better way to do this, however. My point here is that review articles are summaries and everything in them is filtered through the author. They are SECONDARY sources.

I do not understand why anyone would prefer secondary sources, which would make the entry MUCH more removed from evidence than would primary sources. However, I did not, and would not, say that literature reviews should not be cited.

I maintain that ALL sources are open to scrutiny and a "first pass" is the legitimacy of the publication in its field. In science, "peer-reviewed academic journal" is the standard for that first pass. Beyond that, methodology must be examined.

I also think that many contributors to this discussion have misunderstood the purpose of citations. The purpose of citing sources is to provide evidence that what one says/writes is accurate, not to give people something else to read.

I fail to see the relevance of my association with Scuro OR the number of article edits that I have made. ICBSeverywhere (talk) 23:11, 30 November 2008 (UTC)

Your beef is with WP:RS, not with us. Please go there. Encyclopedias shouldn't be examining methodology. JFW | T@lk 23:17, 30 November 2008 (UTC)

I agree with JFW's remarks on reviews. Secondary sources aren't perfect, of course, but they're better than the alternative of relying only on Wikipedia editors' judgment. Moreover, we have a Wikipedia policy that prefers secondary sources like reviews to primary sources like reports of experiments (see WP:PRIMARY). We can't overturn that policy in this guideline. Eubulides (talk) 20:17, 30 November 2008 (UTC)

The "pond I'm fishing in", is that all reviews are not equal, yet wiki makes little distinction about the quality of reviews. You can have a "crap" review not worth much more then the cost of toilet paper, yet under current guidelines it may be considered an ideal or near ideal source. That ain't right. Also, propaganda often is made to look real. Simply stating that, "secondary sources aren't perfect", ain't the way to deal with this gaping hole.--scuro (talk) 21:21, 30 November 2008 (UTC)

I do not think that it is possible to put together a policy that can address the quality of sources published in peer-reviewed academic journals, but I do think it is entirely possible to limit citations to sources that are peer-reviewed to reduce the amount of propaganda that could occur in a free-for-all Wiki.

That said, NO source is error-free and NO source should be beyond questioning. ICBSeverywhere (talk) 23:11, 30 November 2008 (UTC)

Scuro, could you have a stab at generating a list of useful discriminators that will help editors distinguish between a sterling and a faecal review? JFW | T@lk 22:24, 30 November 2008 (UTC)

She didn't really hit upon one of the biggest problems with reviews in the context of Wikipedia. That is, one can cite a review which actually cites one or two little articles. Also, as she hinted, the reader doesn't get the straight facts because the paper is perceived through the possible bias of the reviewer, who may not understand the science going on as well as the author. This can go both ways, but JFW has insisted that the author's credentials (ie publication history) doesn't matter as much as whether a paper is a review or not, or whether it's in a journal that's a household name even among laypeople (BMJ, Lancet, NEJM, ect). An inflexible insistence upon reviews can therefore be harmful to lay readers, who often can't even access the full-text of the review to find out what that paper is basing its assertions on. In academia, citing secondary and tertiary sources is frowned upon for exactly this reason. It adds an extra step finding the real underlying data. Systematic reviews help to overcome this to problem to some degree.

Phenylalanine brought this issue up over at WP:RS/N a while ago. I suggested that he cite the reviews and then indicate what they were basing their arguments on. As you can see from this section of the paleolithic diet, he agreed with me and did that. It adds wordiness, but I think it is often worth it. II | (t - c) 23:38, 30 November 2008 (UTC)

So there is broad consensus that reviews are not a "widget" of uniform quality?..and that wikipedia needs to address this issue?--scuro (talk) 12:52, 1 December 2008 (UTC)

All sources vary in quality. These guidelines already prefer reviews from reputable medical journals, which should filter out the "crap" reviews. But really this is not WP's most pressing problem. An analogy: "Eat in a restaurant rather than from a burger van, if you want quality food, safely prepared". We're getting comments above along the lines of "Well I've seen some really dodgy restaurants, and food inspectors often find poor hygiene when they inspect them." Totally ignoring the burger van is using week-old fat and has no running water or toilet facilities for staff. It is easy to find bad WP articles that cite primary sources that superficially appear to support the text. In my experience here, I've yet to see the same problem with reviews being widely misused. I agree with almost nothing in II's first paragraph above, and don't even understand the "An inflexible..." sentence. Academic practice to sourcing isn't relevant on WP and thinking our papers should be researched and sourced the same way as an academic review shows a big misunderstanding over WP fundamentals. II once again claims reviews add "bias", as though the primary research paper contained virgin facts, unsullied by another's opinion. Primary sources are widely misused on WP for at least three reasons:

1. Cherry picking. Picking some studies rather than others.
2. Extrapolation. Taking basic science, animals studies or case notes and extrapolating the findings to the public at large.
3. Abstracts. Editors without access to the paper's full text can often glean something from the abstract in a research paper. The abstract of a review often gives away only the scope. Some of the comments above pontificate about methodology or quality but many primary studies are cited by folk who have only read the abstract. That is no more going to the source of the information than watching a movie trailer is of a film.

ICBSeverywhere, your lack of edits doesn't devalue your opinions but does cast doubt over your experience of applying existing WP guidelines and policy when writing articles. Remember that only a tiny proportion of topics on WP are controversial. Writing an article sourced to reviews is much easier than trying to do so from primary research papers. Most folk disputing WP guidelines have an issue with a controversial aspect of a topic. What is often forgotten is that it is practically impossible to source an entire topic to primary research. WP:NOR does not allow you to fill in the blanks. Colin°^Talk 20:31, 1 December 2008 (UTC)

Scuro, I think the person you're talking to doesn't quite grasp the goal here. She writes, for example: "Research findings are primary sources, so they are always superior."

Sure: I want the primary sources. No -- strike that, I want the original dataset, on my own computer, and I'm going to have a long, personal chat with the people that generated it. I'm also going to drop a note to a couple of people that have worked with these individuals, and to a couple of people that work in the same field.

But are primary sources always superior?

Certainly not: If you gave my grandmother a research paper about her high blood pressure, she would be utterly lost.

Here, the goal is not to find out the Truth™. It's to find out what is currently "known". We're writing an encyclopedia. That means that we are assembling and summarizing the state of human knowledge. Secondary sources are the best way to do that. That's why the entire encyclopedia (and not just medicine-related articles) is (supposed to be) based upon secondary sources. Ideally, they'll also be high-quality secondary sources, but as you have noticed, the quality variable is independent of whether the source is primary or secondary. WhatamIdoing (talk) 02:39, 2 December 2008 (UTC)

I have seen many examples of the misuse or primary sources. There are also problems with reviews. The issue can be quality but it can be pov pushing too. I have never seen an example in wikipedia of a bogus research paper. I have seen very questionable reviews. As ICB states, anyone can write a review. They are always hearsay. Reviews are easier to manipulate towards a biased point of view. I have also seen reviews from 25 years ago being used as citations. If reviews are a "state of the nation report" on our current understanding about a topic, then they should have a well defined shelf life. To me it just seems that reviews have been put up on a pedestal, that any review will be deemed as good by association to the designation of ideal source. This is the way most contributors see reviews, no matter what the quality. That is not reality, and this will cause problems over time.--scuro (talk) 12:46, 2 December 2008 (UTC)

Can you give some examples of reviews in a quality journal that "manipulate towards a biased point of view"? And why should you think this is "easier" for a review than for a research study? We already recommend against using old sources, so 25 year-old sources would be unlikely to be a good source, review or not. Being a review doesn't exclude the source from any other check recommended by MEDRS, WP:RS or WP:V. For example, a review in a journal of an organisation seeking to promote a particular POV might be dismissed for being a biased publication, as with any source. Even if the world was populated only with high quality reviews and high quality research papers, we'd still prefer reviews as sources. They are different beasts, and there are aspects of reviews that make them vital sources on WP, and those attributes are simply missing from primary studies. Colin°^Talk 18:41, 2 December 2008 (UTC)

• "I have never seen an example in wikipedia of a bogus research paper." This doesn't match my experience; I often run into uses, or attempts to use, research papers that are highly questionable. A few days ago, for example, another editor wanted to cite Geier et al. 2008 (PMID 18817931) in Causes of autism to support the fringe claim that mercury poisoning a significant cause of autism. This paper is coauthored by Mark Geier, who has serious credibility problems (see Mark Geier #Ethics). I'm afraid this sort of thing happens all too often: editors get excited or interested in a recent primary study of a medical topic and want to throw it into an article, without having the context and quality control that a good review would provide.
• Now that we have a recent example of a primary study misuse, can we also see a recent example of review misuse in Wikipedia?

Eubulides (talk) 20:58, 2 December 2008 (UTC)

Dated and low-quality reviews

Okay, on the premise that reviews which are seen as an ideal source and have limitations. These limitations get short shrift on the guideline page. I will attempt to find examples of:

• reviews that contributors used long past their "best before" date
• reviews that are not reviews but more propaganda
• reviews of questionable quality

If other contributors can easily find examples of such reviews, it would be appreciated if you shared that with us. :)--scuro (talk) 12:03, 3 December 2008 (UTC)

I'm not sure that I'd bother looking for outdated reviews, because that's every bit as much of a problem with primary sources as it is with secondary sources. But examples of low-quality and promotional reviews, especially if there are actually good primary sources on that point, would be desirable. Everyone is in favor of killing low-quality and biased references. WhatamIdoing (talk) 19:28, 3 December 2008 (UTC)

Then we are agreed that the limitations of outdated reviews should be spelt out more clearly on the med. reliable source page. Following wiki processes, old reviews can be removed but this can be time consuming and frustrating. It has been a source of conflict in the past. A contributor will find a review and argue that the informtation on the review is relevant, when in reality it clearly no longer is. Clearer guidelines may nip these sort of problems in the bud.--scuro (talk) 20:16, 3 December 2008 (UTC)

Clear guidelines regarding date parameters for reviews would be helpful. It would hopefully give weight when discussing this issue when complaining about an "old review" on an article talk page. I find that editors often don't seem to consider the "best before" date when using reviews as sources. Also, the use of primary sources is prevalent. "Some claim that Blah."<primary source> —Mattisse (Talk) 20:36, 3 December 2008 (UTC)

• WP:MEDRS already says "Look for reviews published in the last five years or so, preferably in the last two or three years." If that's not clear, can someone please suggest a specific wording change that would be clearer?
• It's not that hard to find reviews, even recent reviews, that Wikipedia should not cite uncritically. One example would be Nissen & Wolski 2007 (PMID 17517853), published in NEJM no less. This meta-analysis has been heavily criticized; just look at the PubMed record's "Comment in:" field for starters. I'm not saying that the review shouldn't be cited; only that its results should not be presented uncritically.

Eubulides (talk) 21:14, 3 December 2008 (UTC)

The guideline is clear IMO. In addition, I see no issues raised that are specific to reviews. In other words, if one reads this guideline in conjunction with other sourcing guidelines and policy, there doesn't appear to be a problem that needs to be fixed. Remember these are just guidelines. There can be perfectly valid reasons for citing a 10-year-old review. For example, it discusses a notable therapy that nobody (apart from a few cranks) uses any more, but which once had a following. Colin°^Talk 21:55, 3 December 2008 (UTC)

Yes, it does tell you what wikipedia desires. But, what if a review falls outside of this parameter, which pragmatically speaking is where disagreement usually happens on the talk page. Are there guidelines that tell us when a review is clearly out of date? I remember a contributor getting rather heated because they wanted to use a 1977 review on ECT (http://www.ncbi.nlm.nih.gov/pubmed/900284) to make the case that ECT causes brain damage. Never mind that no one uses devices from that time period currently. This may seem obvious to all of you that a 30 year old review would be highly questionable. But, not everyone editing wikipedia has the marked intelligence of those contributing to this thread. I say, spell it out. Granted there are exceptions for every rule, but a general guideline or two indicating what is not acceptable may be just as useful to the garden variety editor, as guidelines which state what is desirable.--scuro (talk) 00:09, 4 December 2008 (UTC)

That is a good example of the kind of referencing encountered. And not every reader carefully looks at the dates of all references and evaluates accordingly. —Mattisse (Talk) 00:24, 4 December 2008 (UTC)

Scuro, the guideline already says that editors should use up-to-date evidence. See WP:MEDRS#Use_up-to-date_evidence. We have an entire section dedicated to that specific issue. Were you unaware of that section? WhatamIdoing (talk) 03:02, 4 December 2008 (UTC)

Perhaps if I used the term "update" the guidelines, that would be more palatable. I did read that section, WAID. It could be clearer. What happens with a review of say 10 years? 20 years? 30 years of age? There are no guidelines there. The section tells us what wikipedia wants. It doesn't tell us what is not wanted. And as a contributor who deals with this sort of stuff issue on a regular basis, you inevitably hear, "you make up the rules as you go along". And what can a contributor say at that point? There is nothing to show what wikipedia doesn't want. So the newbie reasons if an ideal source can be 5-7 years old, then a review that is 10-20 years is good, and one 30 years is okay. It still is a review after all. We have newbie contributors who thus get turned off of the process of researching information and using wikipedia...just my 2 cents worth.--scuro (talk) 03:25, 4 December 2008 (UTC)

How about if we append "Avoid older reviews." to the end of the first bullet of WP:MEDRS #Use up-to-date evidence? Eubulides (talk) 04:23, 4 December 2008 (UTC)

Simple...sure, it's better then what is there now.--scuro (talk) 04:36, 4 December 2008 (UTC)

OK, done. Eubulides (talk) 04:46, 4 December 2008 (UTC)

On further reflection, I'd be happier with a more general statement. "Avoid older reviews" could be (and should not be) interpreted as "Delete the 1985 review, and add in the 1983 primary paper that the review cited favorably." What we want is more along the lines of "Just because it's a review doesn't mean that it's exempt from the normal rules about using up-to-date evidence." WhatamIdoing (talk) 19:57, 4 December 2008 (UTC)

Likewise, I'm unconvinced by this new sentence. Good use of sources requires editorial judgment and an indepth knowledge of the topic. I've read almost everything of significance written about Tourette syndrome in the last 12 years, and if a five-year-old review is still a top notch, accurate, well written review, there's no reason for me replace it with, for example, a newer NEJM review which is available online and contains an incorrect definition of the condition! Blanket statements about dates aren't as helpful as statements that lend to editorial judgment; I think we can find a better way to phrase this, and don't recommend removing sources based on date alone. SandyGeorgia (Talk) 20:33, 4 December 2008 (UTC)

All good points, thanks, and I reverted that change. Suggestions for better wording are welcome. Eubulides (talk) 21:12, 4 December 2008 (UTC)

These things should not be phrased in terms of rules. This is a guideline. There's going to be wikilawyering when they're phrased as hard rules. Just yesterday I ran into Mattisse saying that a 2002 American Journal of Nutrition review is unciteable because it is not within the 5 year date.[30] Yet this particular 2002 review has free full-text. A 2005 review (PMID 16021987) has essentially the same conclusions, but it lacks full-text. II | (t - c) 20:13, 4 December 2008 (UTC)

What I've done in cases like these are to use a single <ref> and cite both reviews, the newer one first, with the older one marked as being older but freely readable. Something like this inside the <ref>:

• Williams AL, Girard C, Jui D, Sabina A, Katz DL (2005). "S-adenosylmethionine (SAMe) as treatment for depression: a systematic review". Clin Invest Med. 28 (3): 132–9. PMID 16021987.{{cite journal}}: CS1 maint: multiple names: authors list (link) A freely readable but older review supporting the same claim is: Mischoulon D, Fava M (2002). "Role of S-adenosyl-L-methionine in the treatment of depression: a review of the evidence". Am J Clin Nutr. 76 (5): 1158S–61S. doi:10.1093/ajcn/76/5.1158S (inactive 2023-12-13). PMID 12420702.{{cite journal}}: CS1 maint: DOI inactive as of December 2023 (link)

As it happens this particular example is not a good one, as the first source is actually freely available on the net via an authorized copy, so that a simpler citation would be:

• Williams AL, Girard C, Jui D, Sabina A, Katz DL (2005). "S-adenosylmethionine (SAMe) as treatment for depression: a systematic review" (PDF). Clin Invest Med. 28 (3): 132–9. PMID 16021987. Retrieved 2008-12-04.{{cite journal}}: CS1 maint: multiple names: authors list (link)

Eubulides (talk) 21:12, 4 December 2008 (UTC)

Argh! I'm with Sandy (and with II) on this one. Editorial judgement trumps "rules". I support a general warning to stick to modern sources ceteris paribus, but if an older source is better we'd be doing the reader a disservice by sticking to a newer source that happens to be junk. JFW | T@lk 20:55, 4 December 2008 (UTC)

Yes, thanks, I reverted that bit of advice as mentioned above (just before you made the comment). Eubulides (talk) 21:12, 4 December 2008 (UTC)

"This statement is more than five years old and is provided solely for historical purposes. Due to the cumulative nature of medical research, new knowledge has inevitably accumulated in this subject area in the time since the statement was initially prepared. Thus some of the material is likely to be out of date, and at worst simply wrong. For reliable, current information on this and other health topics, we recommend consulting the National Institutes of Health's MedlinePlus". -- This is an example of a blurb from the NIH.

I understand why wikipedia wants reviews in the last 5 years. I understand that some reviews a few years older might still have value. I understand that in inactive fields, reviews have longer shelf life. I understand that seminal reviews also have value. But generally speaking, at some point a review is no longer a review. Time makes the review obsolete and it becomes archival history. This is clearly a time sensitive limitation that taints information and MEDRS states nothing about this. If wikipedia reflects societies current understanding about a topic, what are we doing romantically clinging onto the notion of a review as an ideal source, which allows scientific understanding from a generation or two ago, to meet little resistance from getting onto the page? --scuro (talk) 02:54, 5 December 2008 (UTC)

Here is the passage that Scuro want to remove from the ADHD "The controversies have involved clinicians, teachers, policymakers, parents, and the media with opinions regarding ADHD that range from those who do not believe it exists at all to those who believe that there is genetic and physiological basis for the condition, and also include disagreement about use of stimulant medications in treatment." It is in the past tense and is not a review per say.

Here is the reference. http://www.ahrq.gov/clinic/epcsums/adhdsum.htm

The reference from the therapeutics initiative is there aswell that is contested. Doc James (talk) 03:39, 5 December 2008 (UTC)

That's a nice little tidbit Jmh, it might even have some merit but your wrecking my train of thought here. I am talking about reviews in general, and specifically about how wikipedia MEDRS makes no mention about one of the chief weaknesses of reviews. Over extended periods of time they can become virtually useless, and although guidelines suggest around 5 years max, what is the unknowledgeable contributor to make of an "ideal source" that is a fair bit older? --scuro (talk) 05:31, 5 December 2008 (UTC)

Scuro, I have two points for you:

• A review stays a review, even if it's a hundred years old. It does not quit being a review when it gets out of date. It does not magically transform into primary research, or into tertiary summaries. It remains a review. What it quits being is a good source for current information. These are entirely separate considerations.

• This guideline advises -- note that this is a direct quote from the guideline -- "Look for reviews published in the last five years or so, preferably in the last two or three years". Now why do you keep asserting that "MEDRS states nothing about this"?

--WhatamIdoing (talk) 05:33, 5 December 2008 (UTC)

Yes, a review, is a review, but over time, excellent often does morph into questionable or even useless. Take the 1970's ECT review, where they no longer use that machinery currently. That review is now virtually useless. For us it makes sense not to use it. But what of vast majority of the scientifically uniformed contributors? What will they think when they see "good evidence" from a review that is 10, 20, or 25 years old? A review is an "ideal source" after all. Isn't a goal of wiki to educate? When I read MEDRS I see that much of what is in that article is educational. The article tells how to determine what makes a good review. Why can't that same page also educate the reader about the chief limitation of a review?--scuro (talk) 12:57, 5 December 2008 (UTC)

Because age isn't the "chief limitation of a review" any more than it is an aspect to consider for any source, be they primary research papers, books, newspapers, etc. Please credit our "scientifically uniformed contributors" with some common sense. I think this "issue" has been discussed enough. Colin°^Talk 15:13, 5 December 2008 (UTC)

Not sure what you mean about crediting "scientifically uniformed contributors". A current FAC uses a year 2000 source for data on medication types, strategies and doses. The source may be originally a good one, but many current medications in use today were not being used in the year 2000 and prescribing practices have changed since then. Is this what you mean? —Mattisse (Talk) 16:02, 5 December 2008 (UTC)

And therefore, your argument is that the topic being discussed has changed significantly in the last 8 years and so a year-2000 source is unlikely to be very useful. What has this got to do with medicine-related articles or reviews that isn't equally valid for any topic on WP that is still actively studied? An eight-year-old article on mobile phone technology is also unlikely to document the "state of the art" accurately any more. Colin°^Talk 16:59, 5 December 2008 (UTC)

I am referring to the origin of this thread regarding dated reviews. So you are saying that the issue of an 8 year old review for medications that do not include the most recent and frequently used is irrelevant, Colin? Don't understand this argument of yours. —Mattisse (Talk) 22:02, 7 December 2008 (UTC)

I'm saying that out-of-date sources is a general problem that isn't medicine-specific. Let's try to focus MEDRS on problems that are specific to this area. This issue you raise isn't "irrelevant" and it sounds like your example is a genuinely problematic use of old sources -- but that isn't a problem that MEDRS needs to rule on. Colin°^Talk 22:16, 7 December 2008 (UTC)

(Undent) Scuro, this guideline already says that recent reviews are preferable. Now why are you insisting that this guideline doesn't say that recent reviews are preferable? WhatamIdoing (talk) 19:16, 5 December 2008 (UTC)

Gray areas

"Because age isn't the "chief limitation of a review" any more than it is an aspect to consider for any source, be they primary research papers, books, newspapers, etc". - as I understand it, the function of a review is to summarize what is known about a topic, at any given moment in time. MEDRS speaks of the review cycle. Renewal, is part of that cycle. New reviews are needed as our understanding about a topic expands. A review documents recent findings and then more research is done. It follows a cycle. Primary research can get dated by other research, BUT time is not inherently woven into the structure of research. That is a major difference.

"Scuro, this guideline already says that recent reviews are preferable. Now why are you insisting that this guideline doesn't say that recent reviews are preferable"? - Well lets look at grey areas or what the guideline doesn't cover. Lets say there is an old review on a specific topic. Lets also say that there has been primary research done since then but no new review. Which is the better piece of evidence? Is the review still the better source or is the primary research better? How about another example. What if there are several reviews that state different things? Does the age of the review matter in such a case? Can the viewpoint of an older review be used for "minority opinion" if it conflicts with a recent reviews? How about if a VERY OLD review states something that several new ones don't even touch upon. Is that unique bit of information good or must we see that in context of what is known since then? Before you come up with your answers, here is another question...are the answers obvious to all? The guidelines could possibly miss these issues because they only focus on what is wanted from reviews. --scuro (talk) 14:28, 6 December 2008 (UTC)

• I don't think the guideline can come up with advice that is definitive for every circumstance; there will always be some gray areas. We can make gray areas smaller, but we have to do so carefully, as the cost of shrinking gray areas is often greater than the benefit.
• For all your examples, the answer is, "it depends". In your first example, if the review is quite old and says X, and if reliable primary studies since 2006 all agree on not-X in their previous-work sections (because of new evidence that came out in 2006), then the article shouldn't claim X and cite the review. Conversely, if the review was done last year, has a lot of authors at respected institutions, and is published in a leading journal on its subject, whereas the more-recent primary studies were all published by fringe sources, then you go with the review.
• To move this discussion forward, it would help if specific wording-changes could be proposed to address the concerns. I tried to do that with a simple "Avoid older reviews." and got shot down for the good reasons cited above. Can you propose better wording?

Eubulides (talk) 19:57, 6 December 2008 (UTC)

Scuro, you're making points similar to the ones that Paul gene and I (among some others) made earlier. There's nothing about a review per se which makes its conclusions more reliable than the conclusions of a research article. Reviews are less common than research articles, so they can be dated. And research articles are not just primary sources, they are also secondary sources. Reviews are also sometimes very general, and when a review is general, it has to stuff a bunch of information in only a few pages. That means it gives it a light treatment -- just a sentence or two -- when sometimes we want to give topics a detailed treatment on Wikipedia. Research articles tend to give focused attention to a particular line of research. Their introduction/discussion sections together often give fairly lengthy reviews of the topic. Scuro's example of areas where reviews are dated and research articles are recent is a good one. Eubulides suggests that we should only use such research articles if they "all agree", but that imposes an unrealistic burden upon those wishing to present well-sourced information on Wikipedia. My suggested wording would be: "Research articles may provide secondary information on an area of research. Citing these research articles may be preferable if reviews on the topic are dated or too limited in scope." The concern, as always, is that one might cite some "fringe" research article for an overview of a topic. But this concern applies to reviews as well, and shouldn't invalidate citing good information. II | (t - c) 22:50, 6 December 2008 (UTC)

It might be worthwhile backtracking to why we've focused on review articles. The bottom line is that we don't want editors to use their own prioritization and synthesis of the primary literature - it's too easy for an even moderately sophisticated editor to mine the literature and create a "reliably sourced" article that violates WP:WEIGHT. One workaround to prevent this sort of abuse (which I've found to be relatively common) is to insist that our use of the primary literature mirrors the synthesis and weighting applied by experts in the field. Review articles give us a window to see what experts in the field think about the state of the primary literature. They're not magical or infallible, and they're probably not even necessary for every single sourcing question - but reviews are very useful to prevent editorial mining and abuse of the primary literature to circumvent WP:WEIGHT. If that's a problem, then we should go to review articles to solve it. If there's no such problem, then there's no problem. I think this all falls under the rubric of Wikipedia:Use common sense. MastCell ^Talk 23:37, 6 December 2008 (UTC)

I'm not sure I agree. Citing a bunch of research articles individually to synthesize a point involves much more work than to than citing one research article which makes a fairly bold claim, sometimes even secondarily through its own synthesis. I feel like the bigger concern has to do with undue weight. I think combating this sort of undue weight through restricting the citation of research articles is not right. If I felt like it, I could cite both recent reviews and research articles in decent journals which say there's a connection between thimerosal and autism. The argument would not be that these are original research, but that it would be "undue". Anyway, what do you think about my suggested wording above? I don't really like relying upon common sense in what is really a bureaucracy. People cling to rules. II | (t - c) 07:30, 7 December 2008 (UTC)

ImperfectlyInformed, you've made this point over and over, and I continue to disagree that the "introduction" and "discussion" sections of research articles should be put on the same level as review articles. They tend to be focused particularly on information directly relevant to the topic under research, and may completely ignore alternative theories. This is not a rare occurrence.

If a review makes a bold claim, and the evidence doesn't stack up, then this should be addressed through the usual channels. You are correct that reviews sometimes make bold claims based purely on clinical experience and "gut feeling" of clinicians who have studied a condition for most of their career. If this ends up in the Wikipedia article, and others disagree that this should be included, it can be discussed on the talk page. On the whole, that is only a minor problem with reviews.

People do indeed cling to rules, which is why we have WP:IAR. JFW | T@lk 07:49, 7 December 2008 (UTC)

If you read my suggested wording, it doesn't say that research articles and review articles should be given the same priority. It says: "Research articles may provide secondary information on an area of research. Citing these research articles may be preferable if reviews on the topic are dated or too limited in scope. II | (t - c) 08:13, 7 December 2008 (UTC)

Reviews also have degrees of quality. A systematic review for example is often better then a literature review. The Cochrane collaboration being one of the most famous sources of these.

Systematic reviews that are recent are preferable to a single research article. As would large reviews from groups such as the NHS. I have seen research articles selective pick from the available research only that that supports the conclusions of their study.

If a review or primary research article is wrong /dated then it should be easy to find other sources that refute its conclusions. And this should be the discussion that takes place on the talk page of the article. WRT the review which showed ECT caused brain damage one could easily word things: "a review from 1975 indicted that ECT might cause brain damage, further research however showed that this was not the case" etc. This type of info might be interesting under the history section of an article. One acknowleges the review and then refutes its conclusions.--Doc James (talk) 14:23, 7 December 2008 (UTC)

A few points on the above (sorry if a little off-topic)

1. These are guideline for writing a comprehensive medical article, not just for those parts that are contentious. To that end, a systematic review is often not "better" (in terms of usefulness, not quality) than a literature review, because its scope is very narrow. They are very good at what they do, but one is unlikely to be able to source an FA-comprehensive article to Cochrane and his mates.
2. Try to avoid referring to medical literature in the article body. Unless a paper is itself highly notable (e.g., Wakefield's infamous MMR), it should not be mentioned (as a paper). To the reader, a "review" is a guide to what's on telly tonight -- so avoid that jargon. A review is just a form of literature. Would you say "a paperback from 1975..."? I'm unable to read the full text of the review on ECT, and know little about the subject, but it looks very much like that review is of case studies and is very explicitly one author's opinion. Why should it be mentioned at all?
3. Prefer to state the facts with an appropriate level of confidence as supported by your sources. The reader wants to know if ECT causes brain damage. Not whether one author said this and another author said that. Or if one small study found this or a larger study found that.
4. If it is necessary to refer to the research, talk about studies and meta-analyses (though that term usually needs explained) rather than papers and reviews. There are places where historical, recent and ongoing research should be explicitly discussed in a medical article, but they are fewer and more discrete than is typical on WP (both Wikidocs and POV pushers are guilty here IMO).

Colin°^Talk 19:10, 7 December 2008 (UTC)

Re the proposed wording "Research articles may provide secondary information on an area of research. Citing these research articles may be preferable if reviews on the topic are dated or too limited in scope.": this gives too much leeway in the troublesome direction of editors citing primary studies instead of reviews. For example, in the scenario where a high-quality mainstream review was done last year, and more-recent research articles were put out by fringe sources, an editor could argue that the fringe sources trump the review. How about the following wording instead? It would be inserted after the 3rd bullet of WP:MEDRS #Use up-to-date evidence.

• In a lightly-researched area lacking recent reliable reviews, it may be possible to summarize non-review sources; however, extreme care should be taken, as these sources often tend to focus narrowly on one subtopic to the exclusion of alternative theories, and it is all too easy when combining this material to synthesize it to advance a position. It is often better to say nothing, or to say that evidence is lacking, than to attempt to summarize a poorly-reviewed medical area.

Eubulides (talk) 20:24, 7 December 2008 (UTC)

I'm not keen. This page can guide readers to what is best (and therefore what is "better" when comparing sources offered by editors who are in conflict) but we should be careful not to restrict editors beyond what policy requires. I would not want to see this guideline being used to remove text sourced, for example, to an NHS page or a reputable charity information page, merely because it wasn't a review in a peer-reviewed medical journal. While an editor without access to medical journals is unlikely to achieve FA, they should not be excluded from contributing. There are many medical articles that are utterly unloved and could benefit from extra material. emphasise the spectrum of quality, and range of article types, rather than just say "use this; avoid that" black & white. Colin°^Talk 21:11, 7 December 2008 (UTC) (Though I think some sources (newspapers) are so far down the quality spectrum that "avoid" is quite justified. Colin°^Talk 21:12, 7 December 2008 (UTC))

"I'd like us to emphasise the spectrum of quality, and range of article types".-I like that better already. Not all reviews are equal. If we get beyond the ideal review and inform the reader of indicators that may demonstrate reviews further down on the spectrum, I think that is a step in the right direction...stuff like that would be helpful to me as an editor and I am sure to others.--scuro (talk) 00:29, 8 December 2008 (UTC)

spectrum analysis of quality

Here is a crack at spectrum analysis of determining quality in Reviews.

1. not peer reviewed <--> peer reviewed by independent body
2. not published <--> published in well-known and well-respected academic journal
3. reviewed literature is not academic <--> reviewed literature comes from well known and well respected journals
4. review is several or more review cycles old (unless it is a seminal work) <--> review from previous or last review cycle
5. author's focus is narrow <--> authors focus is wide
6. author is unknown in field <--> author is widely known expert in field

Whadya think?--scuro (talk) 04:30, 9 December 2008 (UTC)

Some points are unnecessary. Peer review is usually anonymous but also independent; it would be tautological to insist on "peer review by independent body".

How do you propose to establish whether an author is known in his field? Could this not simply be something we can resolve by consensus, rather than trying to cast this in stone? JFW | T@lk 10:31, 9 December 2008 (UTC)

What may be obvious to you or me, many not be obvious to the majority of contributors. Probably a majority of contributors have no real idea what a review is. This is why the MEDRS article is most likely largely an educational article. Such a guideline would educate but it would also help any contributor determine what a bad "review" is. Are there not examples of reviews, or something that walks and quacks like a review, where the "peer review" is neither anonymous or independent? Consensus is not always an easy thing to achieve, especially when a majority of the contributors do not have the background to make an educated judgement. Couldn't a guideline based on a model such as this save countless wikipedian man hours and make articles more accurate? I agree that is not a simple task to determine whether an author is well known in his field. --scuro (talk) 12:45, 9 December 2008 (UTC)

Several of these items are wrong, or at least misleading. For example, a narrow focus is great if you want to get an in-depth look at a narrow question, but absolutely lousy for getting a broad perspective of the general situation -- and vice versa. Neither of these styles is better than the other.

It's not all about the source; it's also about how you use it. This is, once again, not an issue that is specific to medical reviews. This is every bit as true if you're trying to write about George Washington as it is if you're trying to write about Disseminated intravascular coagulation. WhatamIdoing (talk) 02:18, 10 December 2008 (UTC)

If you see more errors or misleading items, point them all out. We can't deal with the unknown. I'm no expert here but as I know it, if the focus of analysis is on a very narrow question, then they wouldn't call it a review, now would they? That would make your comparison between "broad" and "narrow" moot. But that doesn't seem to be the thrust of your disagreement. You seem to think that parameters that help identify poor reviews are not a good idea because some of those parameters are the same parameters that one would use for ordinary sources? Did I get that right?--scuro (talk) 07:17, 10 December 2008 (UTC)

A review can indeed focus on "a very narrow question", and may be just as narrow as a primary research paper. For example, a systematic review of all studies of drug X on condition Y is no wider than each of those studies. On the other end of the spectrum, a review could attempt to summarise our understanding of an entire medical condition, or aspects of a whole medical speciality. I don't disagree with you noting that focus is a "spectrum" issue. The focus of a source is likely to be reflected in the extent of its use in an article. However, focus isn't a quality issue, which I think is WhatamIdoing's complaint. But the other issue, which I agree with WhatamIdoing, is that this guideline should concentrate on areas specific to medical sourcing, rather than sourcing any subject that has academic literature. Colin°^Talk 08:22, 10 December 2008 (UTC)

(Undent) Scuro, I numbered your suggestions for convenience. Here are my comments on each individual item:

1. Peer review is an issue of quality, however, many documents, such as consensus statements by major organizations or government guidelines, are not generally considered peer reviewed and are still great sources. Thus this proposed bit of advice would mislead editors into incorrectly rejecting high-quality sources.
2. Per WP:V, unpublished documents cannot be used anywhere on Wikipedia for any purpose. Presumably this is meant to read "self-published". However, consensus statements by major organizations or government guidelines are generally considered to be self-published and are still great sources. Thus this proposed bit of advice would mislead editors into incorrectly rejecting high-quality sources.
3. I'm not entirely sure what this means, but academic sources are not necessarily the best. I would not necessarily elevate the work of a university-based researcher over the work of an intramural NIH researcher simply because the one holds a post at a university and the other holds a post with the US federal government. Thus this proposed bit of advice would mislead editors into incorrectly rejecting high-quality sources.
4. Age is not a matter of quality. (Age dramatically limits how you can appropriately use a source of scientific information, but an amazingly well-conducted review doesn't become a bad one in fifty years; it just becomes an outdated source.)
5. Focus is not a matter of quality.
6. The effectiveness of an author's publicity and self-promotion efforts is not a matter of quality. They are often used as proxies for quality, but famous researchers make mistakes, and unknown people publish brilliant work.

Importantly, not a single one of these proposed items is specific to medical reviews, which is what I thought you set out to do. You could have written exactly the same thing about primary research papers, or about Egyptian archaeology, or about computer architecture. Those areas, too, benefit from peer reviewed papers, published in reputable media, by knowledgeable researchers. It is not appropriate for MEDRS to tread so heavily on the territory of WP:RS and WP:V as to provide advice that applies to nearly every one of Wikipedia's two and a half million articles. WhatamIdoing (talk) 21:05, 10 December 2008 (UTC)

• 1. Peer review is an issue of quality, however, many documents, such as consensus statements by major organizations or government guidelines, are not generally considered peer reviewed and are still great sources. Thus this proposed bit of advice would mislead editors into incorrectly rejecting high-quality sources. - We are talking about reviews...no? Peer review is a major component of a review.

• 2. Per WP:V, unpublished documents cannot be used anywhere on Wikipedia for any purpose. Presumably this is meant to read "self-published". However, consensus statements by major organizations or government guidelines are generally considered to be self-published and are still great sources. Thus this proposed bit of advice would mislead editors into incorrectly rejecting high-quality sources. - again the focus is on reviews and not government sources, is it not? If we are talking spectrum, I have seen self published reviews. This would be at the bottom of the spectrum.

• 3. I'm not entirely sure what this means, but academic sources are not necessarily the best. I would not necessarily elevate the work of a university-based researcher over the work of an intramural NIH researcher simply because the one holds a post at a university and the other holds a post with the US federal government. Thus this proposed bit of advice would mislead editors into incorrectly rejecting high-quality sources. Does a review not examine recent findings? Again, we are talking specifically about reviews, no? To make things clear, reviews examine literature and not people or insitutions.

• 4. Age is not a matter of quality. (Age dramatically limits how you can appropriately use a source of scientific information, but an amazingly well-conducted review doesn't become a bad one in fifty years; it just becomes an outdated source.) - If a review gives us a synopsis of what we know at any given time, would not a 50 year old review be a historical document, instead of a living document like the updated NIH pages, or research which is not dated by function. Do you believe that a 50 year old review cited outside of it's historical context, is an ideal source?

• 5. Focus is not a matter of quality. - No, but it can demonstrate poor quality.

• 6. The effectiveness of an author's publicity and self-promotion efforts is not a matter of quality. They are often used as proxies for quality, but famous researchers make mistakes, and unknown people publish brilliant work. - Who said anything about publicity or self-promotion?

The issue is, are there reviews which are not ideal..or are all reviews which don't fit the parameters on MEDRS not ideal? Also, how do you determine if a review is a poor review, or are all reviews that are not ideal, poor reviews.--scuro (talk) 16:53, 12 December 2008 (UTC)

On points 1, 2, 3: When a significant organization issues a paper on the current state of medical understanding about <fill in the name of your favorite medical issue here>, it is generally considered a very important literature review -- and it is not generally peer-reviewed (it is instead written by the peers), it is generally self-published, and the people issuing the document may or may not have any academic credentials.

Are you aware that the definition of a scientific "review" properly includes any scholarly paper that primarily discusses other people's research? It is not limited to papers published in some Journal of Medical Reviews or a paper that gets a subtitle of "A review". Any scholarly paper whose major point is, "We looked over what everyone in the field has been writing recently, and here's the current state of things" is a review.

4: Age is not a matter of quality. As I said above, outdated sources are dramatically limited in their utility. They are ideal sources for what was understood fifty years ago; they are not ideal sources for what is understood today. Once again, this issue is also not specific to either medicine or to reviews. Fifty-year-old primary research papers are also dramatically limited in their utility in exactly the same manner. It would be stupid to single out reviews for concerns about being out of date; that's why MEDRS has a separate section that addresses this issue for all sources. You will find a related statement at WP:RS ("However, some scholarly material may be outdated, superseded by more recent research..."); what we do here is provide some specific guidance about how to determine what "outdated" usually means in medicine-related articles.

5: A lack of focus does not demonstrate poor quality -- ever. A paper that reviews general concepts and issues in Heart disease is not automatically of lower quality than a paper that considers only the use of a single medication in a single type of heart disease for a single group of specially selected patients. These kinds of papers have different strengths, but neither is actually worse than the other. A good general literature review can be of much higher quality than a poorly conceived systematic review (and vice versa). Quality is independent of the narrowness of the subject.

6: Whether or not the author is "unknown" or is "widely known" often has much more to do with efforts at publicity than anyone wants to admit. For example, the University of California system provides a substantial pay raise to "world famous" professors, and consequently (according to a friend that is a professor of chemistry), many senior professors in the UC system deliberately court fame as part of their career plan. Linus Pauling, a scientific celebrity, wrote some embarrassing nonsense about DNA once upon a time; the much less well-known (at the time) Watson and Crick got it right. Whether or not you the author has a good reputation does not make any specific paper be a high-quality source.

Your first three points appear to be based on not knowing what a review is; your second three points are both wrong and not specifically applicable to reviews. There might be a useful way to educate the inexperienced editor about how to identify good sources, but IMO this list is not actually going to be helpful. WhatamIdoing (talk) 21:14, 12 December 2008 (UTC)

WhatamIdoing's points above - Numbers 1 & 2 above - Your points seem to stem from a misunderstanding of what Consensus statements are. These are statements of position. Although they are good sources for a layperson to get an idea of where scientists stand, they are not evidence and should not be used as sources. These statements usually cite literature. THAT LITERATURE should be cited as evidence.

Number 3 - To echo Scuro, "source" refers to publication, not author. Most NIH, NIMH, & NSF work is completed by university researchers. These agencies fund research through grants, which are distributed by a committee made up of experts in the field to which the grant money is allocated. The research conducted by those who receive the grants is published in peer-reviewed ACADEMIC journals. Often, the agencies summarize information for patients, physicians, and educators.

Number 4 - The quality of a review is not reason for the guideline on age. The age of a review IS a reason to question the review's RELEVANCE. If significant knowledge has been gained since a review was written, the review can no longer tell you the state of the field.

Number 5 - There seems to be a disagreement about the meaning of "focus", and your comments seem to stem from a misunderstanding of what a review is. An article that examines 2-3 reports IS NOT A REVIEW. Likewise, an article that answers a very, very specific question is not actually a "review", either, although it may be called a "meta-analysis". Reviews are articles that discuss the state of a field (which may be broad or somewhat narrow) by reviewing ALL of the relevant, recent literature. The terms "relevant", "recent", etc. are subjective; I doubt you can set guidelines for them.

Number 6 - The public's knowledge of a researcher is not a measure of a scientist's expertise and I don't believe that anyone suggested that it was. Scientists know who the leaders in their field are (even if they all think they are included in this group). It's true, though, that people not working in the field are unlikely to know. Since Wiki will never be written entirely by people who actually know the field, using only sources from peer-reviewed, respected, academic sources takes care of this. Why? Because only people who are leaders in the field have reviews published in these sources, whose editors are leaders in the field.

None of those guidelines needs to be "specific to medical reviews". Science doesn't work all that differently in different fields. ICBSeverywhere (talk) 17:57, 13 December 2008 (UTC)

You might be interested in knowing that Wikipedia has a long-standing definition of source in WP:V that is rather broader than your personal definition: "The word "source", as used in Wikipedia, has three related meanings: the piece of work itself, the creator of the work, and the publisher of the work. All three affect reliability." WhatamIdoing (talk) 20:13, 13 December 2008 (UTC)

Consensus statement by well respected authorities in the field are great sources for Wikipedia. Wikipedia is not a review itself but an encyclopedia. If we were trying to write a review then yes one should quote the primary research however we are not. I think a good example of what sort of references we should use in the medical parts of articles is Uptodate. For those who have access if you look at the references they use they often reference sources such as the WHO, the FDA, the AAP, etc..

Science is about ideas not people. Just because you are famous does not make you right or your ideas unapproachable. It is the quality of the work that matters period. For the none medical parts of the articles such as the section on history, cultural significance, social implications, references requirements are less stringent.--Doc James (talk) 19:10, 13 December 2008 (UTC)

Agree. We don't need to cite the "evidence". We aren't academic researchers. In fact, we are discouraged from weighing the evidence and encouraged to cite secondary sources from experts who have weighed the evidence. A consensus statement (like PMID 18823325) is just about the best source we could use. ICBSeverywhere, I think your definition of review matches what others are calling a literature review. What you call a "meta-analysis" is just a scientific technique (not a form of literature) -- which tends to be published in a systematic review. Both are reviews, though they are somewhat different beasts. And focus is an independent issue from whether a review cites sufficient sources. Colin°^Talk 19:48, 13 December 2008 (UTC)

Going beyond the issue of what exactly is a review, I agree with what ICB states. A review is hearsay as is even a consensus statement. Why is that point important? Because while these sources can generally be excellent references for wikipedia, they can also be poor or questionable sources. When someone, or a group of people interpret information there is always the possibility of unintentional or purposeful bias. MEDRS generally speaks to what makes a good source, there is little information to determine the quality of a source when one or more key characteristics fall outside of the parameter stated in MEDRS. ICB and I have both spoken to this issue of how to determine the quality of a source. You can fault what we have stated on several grounds, but we have attempted to make Wikipedia a better source of information by coming up with criteria that weeds out poor and questionable sources. We have made an effort to fix a perceived fault with wikipedia. I'd like to hear if others agree that this gray area is a problem, and if so, can it be fixed?--scuro (talk) 16:10, 14 December 2008 (UTC)

So what I understand you saying is that one should go directly to the primary research? Unfortunately there are lots of problems with that. SSRIs in depression is a perfect example. You have something called publication bias were the company funding the research only publishes results that are in their favor. So that if 16 trial are done 10 are not published and show negative results, 6 are published of which 5 show positive results and one shows negative results. Those doing a review are much better then you and I at finding all the trial that have been done. In this example the meta analysis of all trials found no clinical benefit for SSRIs in mild / mod / and severe depression.

In most areas of research there are trials that show both positive and negative result. A review analysis the over all picture to get useful clinical answer. If you were to ask clinicians weather they would prefer to base their treatments on reviews or primary research virtually all would go for the reviews.Doc James (talk) 17:39, 14 December 2008 (UTC)

So what I understand you saying is that one should go directly to the primary research?-This is what ICB does, but I think you are asking specifically about Wikipedia. I agree that primary sources can also be biased and that I have personally seen many more examples of improperly referenced primary sources then reviews. Pragmatically speaking, it makes sense that reviews are better for Wikipedia. Wikipedia calls them ideal sources. The problem becomes the gray areas where unintentionally, or purposefully biased secondary sources have undue weight on wikipedia. These questionable or poor secondary sources would have little to no weight in academic circles because academics can easily separate the wheat from the chaff, not so for the bulk of Wikipedian contributors. There is a disconnect here. Wikipedian questionable SS have a life on wikipedia where they wouldn't within the academic institutions wikipedia sees as ideal.

Can I focus all on the questions asked previously: We have made an effort to fix a perceived fault with wikipedia. I'd like to hear if others agree that this gray area is a problem, and if so, can it be fixed? --scuro (talk) 18:32, 14 December 2008 (UTC)

Which question? Do you mean should we have specific guidelines outlining how to determine if a review is good or not? Doc James (talk) 18:56, 14 December 2008 (UTC)

The gist of what we are getting at is: can lesser reviews, questionable reviews, and even biased reviews live as citations on wikipedia? Second question, if you perceive this to be a problem, should it be fixed?--scuro (talk) 20:13, 14 December 2008 (UTC)

Scuro, it would really help your cause if you avoided loaded terms like "hearsay" (which is more than just "second hand"; it implies "inadmissible") and examined your own writing for bias (such as repeating the possibility that a review introduces bias, rather than the normal case which is where a review is more likely to reflect consensus than an arbitrary primary source). I can only see two ways that editors can identify poor sources within a given category of source. Firstly, they can draw on their own expert knowledge (which WP does not require). Secondly, they can judge the source using the same criteria as any source for any subject. Other than discussing the types of medical literature, and their strengths and weaknesses, I don't see how MEDRS can help. If you think there's a "fault with wikipedia" in its inability to judge sources, then your first stop should be WP:V or WP:RS. Article talk pages are the place where collaborative judging of sources should occur. And misuse of sources is a far bigger problem than the use of poor sources. Lastly, the readers of WP are reading second or third-hand information. Who would you rather "interpret information"? An anonymous WP editor or a scholar publishing in a scholarly journal? WP:NOR forbids us from all but trivial "interpretation". Colin°^Talk 20:51, 14 December 2008 (UTC)

I believe Scuro is using this term in something akin to the legal concept of Hearsay, which has nothing to do with rumor and innuendo. If they gave you a receipt at the store when you purchased something, and you present the receipt in a court case, that receipt is "hearsay", meaning (with lots and lots and lots of nuance and case history to make things complicated) "a statement that was made outside of the courtroom". A review is not direct testimony about what you personally did, saw, and heard yourself (that would be a report, not a review); if the review were given under oath in a courtroom instead of published in a journal, it would be considered "hearsay". It's not supposed to be an insulting term.

(Other readers of this page may be amused to know that "Learned treatises" are officially exempted from the hearsay rule in the modern legal rules.) WhatamIdoing (talk) 21:15, 14 December 2008 (UTC)

Yes but it all comes back to determining what a "lesser review, questionable review, and biased review" is. These are all subjective terms. You cannot say that any review you do not like the conclusion of is biased. Or just because you do not like other work the reviewer might have published it is biased.

Now it was mentioned that the importance of the researcher should be used as a criteria. This is not verifiable. Therefore it cannot be used as a criteria.

There are entire classes in University on assessing the quality of a review or a primary research paper. This is a complicated process. Will see if I can find a paper on it.--Doc James (talk) 22:04, 14 December 2008 (UTC)

Here is two articles from the BMJ about assessing quality. http://www.bmj.com/cgi/content/full/315/7103/305 http://student.bmj.com/back_issues/0200/education/24.html --Doc James (talk) 22:08, 14 December 2008 (UTC)

Greenhalgh is already cited by MEDRS. I'd be happy for her entire series of articles on papers to be included as external links for MEDRS. Colin°^Talk 23:37, 14 December 2008 (UTC)

Good idea. I added the entire series, as citations, with Greenhalgh's book in a new section WP:MEDRS #Further reading. I suppose it's a bit much to add all this at once, but I figured why not? It's a great series. Eubulides (talk) 09:39, 15 December 2008 (UTC)

Might be a bit subtle, including them as references, when we're really offering them as further reading. How about linking to this page that contains the series, as an external link. There's also a JAMA series "Users' Guides to Evidence-based Medicine", which has been archived here. That appears to be very must practitioner-focused, but might be useful. Lastly, there's this personal website, assembled by Andrew Booth from Sheffield's School of Health and Related Research. It contains both those series and a collection of other related papers, most of which are freely accessible. Would links to the first two be appropriate? Colin°^Talk 12:42, 15 December 2008 (UTC)

Thanks for the pointers; I added the first two to WP:MEDRS #Further reading. Eubulides (talk) 16:58, 15 December 2008 (UTC)

Webster's on,

• hearsay evidence: evidence based not on a witness's personal knowledge but on another's statement not made under oath.
• cause: a principle or movement militantly defended or supported

Collin, are we not to focus on content and not the contributor? By focusing on the contributor are we not in danger of imparting bias?--scuro (talk) 05:41, 15 December 2008 (UTC)

I'm aware what "hearsay" means in both the general "gossip" definition and the legal one. The contents of hearsay evidence are generally inadmissible and not regarded as truth. It is a loaded term, where "secondary source" would have been plain. If you don't want to listen to advice, ignore it. Colin°^Talk 08:27, 15 December 2008 (UTC)

The point I have been making is that all evidence interpreted is hearsay. Interpretation always creates the possibility of bias, either intentional or unintentional. I fail to see how these concepts are controversial, they are just fact. If you want to infer meaning and judgement into these concepts, the fault is not mine. I totally agree that there are many more problems with primary sources and that the vast majority of reviews do not have a significant bias problem. Even if you consider this a small problem though, it is still a problem, and wikipedia will be a lesser source as long it remains as is. There are many guidelines about what makes a poor primary sources and how they shouldn't be used. On those grounds it is much easier to challenge a primary source. A review is much harder to challenge. Yes, you can challenge any review in wikipedia but consider this, a review is considered an ideal source. What will you base your challenge on, your own personal expertise? Some fault in the review?...well unless the review is blatantly biased this will be difficult. What are the criteria that would allow you to find fault in a review? If the review did not meet the criteria of what wikipedia sees as a good review? Answer that question and we are half way home. Reviews are different from other sources and there are criteria that can be used to judge reviews. I've posted ideas, others are now posting further citations. A few obvious and general guidelines should not be that difficult to agree too. MEDRS tells us what makes a good review, but tells us little about what to watch out for.--scuro (talk) 12:13, 15 December 2008 (UTC)

So what you are trying to figure out how to "challenge a review" or suppress the use of a review as a reference? If it is out of date then a newer review could be used. You could find another review that says something different and juxtapose the comments to provide balance. Otherwise it might be a good idea to change your opinion on a topic. I think a much bigger problem on Wikipedia is pages with no references or references to none scientific sources. The discussion of the quality of a review must take place on an individual bases on the pages talk page. What Eubulides has added is a good approach the discussion.Doc James (talk) 13:34, 15 December 2008 (UTC)

The material recently added to WP:MEDRS #Further reading should provide ample resources for editors wishing to challenge a review. Eubulides (talk) 16:58, 15 December 2008 (UTC)

I don't think that Wikipedia's preference for secondary sources obliges editors to use low-quality secondary sources any more than a decision to eat plant-based foods obliges vegetarians to eat hemlock or castor beans. These are independent considerations. We reject low-quality sources whether they are primary, secondary, or tertiary sources. Editors can use their judgement to determine what counts as a low-quality source (in consultation with each other on the article's talk page); if there are specific problems, there are plenty of people willing to help here, at WT:MED, at WT:PHARM, or at WP:RS/N. WhatamIdoing (talk) 21:04, 15 December 2008 (UTC)

Yes James, I do come across crap reviews as I edit, and I believe all crap citations should be removed. They diminish the quality of wikipedia. I don't believe that one should "juxapose" good reviews with bad reviews to provide "balance". So James, you go about your business and you remove the crap review, but once challenged about removing a review, you have a dilemma. A review is an ideal source, the first question any wikipedian is going to ask is why remove an ideal citation? You will say, because it sucks eggs and they may very will say, no it doesn't. At that point you are snookered because there are little to no criteria/ comments about what makes a bad review. Time to take it up the ladder? Which rung? Hmmm...you could take it several places but the question becomes do they have the background to make a good decision? So you are thinking ahead, and steer it to the WikiProject Medicine talk page because you know there are a few smart cookies there who actually know there stuff. Does that solve your problem?..Perhaps but perhaps not. If you are lucky one of the smart ones gets involved...if not...? In the end ANY final judgement will be subjective based on the personal criteria that the judge(s) believe to be right, they have nothing else to go on. Odds are that if you got 3 judges, they would not rule unanimously on the reason why a "gray area" review is usable or not useable. Use another 3 judges and you could very well have a different result. Go to wiki reliable resources page, and you could have a different outcome. Consistency and fairness could well suffer under the current approach. I do agree that you can't come up with a specific list of rules but I don't agree that you can't come up with some generalized guidelines that would make wikipedia a lot more streamlined, useful, helpful, and most importantly accurate.--scuro (talk) 22:44, 15 December 2008 (UTC)

Rating reviews

I am having a hard time understanding what exactly you mean in the above paragraph? You want to remove "crap reviews"? How do you know that the reviews are poor? Some of the rating criteria that have been proposed are not significant. Like who the author is or the focus of the question. The date I would agree has some importance. Peer reviewed is of course important. There are rating scales for review quality but they can be difficult to carry our. Here is one for example for a systematic review http://www.uwo.ca/cns/ebn/Scales/metaanalysis.pdf

These are subjective questions and no review is going to be perfect. If you are wanting a unanimous rule I do not think you will find one. What one does is compare what research is out there and discuss which is the best. There are no cut and dried answer in medicine. Information should be provided so that people can assess the risks and benefits. That is what it means to provide a NPOV. Doc James (talk) 23:13, 15 December 2008 (UTC)

That's exactly what I was going to ask: How do you know that it's a low-quality source? And why the heck can't you just give your specific, exact reasons for the specific, exact source? If you have solid, specific, concrete reasons for believing any source to be low quality, then you should give those reasons, and not try to resort to "Well, I classify it under MEDRS section 2 part 4(a) as probably being..." WhatamIdoing (talk) 23:18, 15 December 2008 (UTC)

I would make a subjective judgement about the review. Would it be done any other way by any contributor?...unless wikipedia had guidelines? That is how all judgments are made in wikipedia on reviews, even at wikiproject Medicine. There is nothing objective to hang your hat on. Back to our example, while I may think the review is crap, most likely the person who cites the review will have a different subjective understanding about why the review is good. Where do we compare notes? We can go to MEDRS and find characteristics of the review that are good and that's all we can do independently. We can even agree that some characteristics of the review do not meet the characteristics of a good review while other characteristics do. What then, does that make it a bad review? Does that give us grounds not to use the review, or does it mean it is no longer an ideal source but is still a good source? What of the other gray areas? We can still both be working in good faith but now we need help. We can bring this problem to a half dozen different spots on wikipedia and we won't get a consistent judgement, it' a complex thing and guidelines are sparse on what makes a poor review. Eventually if one party has unjustifiably been wronged by a poor judgement, the judgement may get overturned with another process by those who have the background to do so. That is if they would hear an appeal, a review is an ideal source you know. Can you see that how it now stands, there will be a lot of wasted time and frustration for anyone involved in challenging a review?--scuro (talk) 05:28, 16 December 2008 (UTC)

This talk page is supposed to be a forum for improving WP:MEDRS but I fear that the current thread has strayed from that goal. I suggest that we focus by proposing specific wording changes to WP:MEDRS. Eubulides (talk) 06:22, 16 December 2008 (UTC)

Scuro, do you know the saying about complex problems having simple solutions? A common mistake here is to think these are guidelines on identifying accurate/truthful/correct sources rather than on reliable sources. We can't offer a guide to help pinpoint all the mistakes or shortcomings an individual paper or statement might contain, but we can identify the types of sources that generally contain accurate/truthful/correct statements. Being reliable means that one can take their statements on trust, depend on them and reasonably hope not to be disappointed. Do you see the difference? Since this is the encyclopaedia anyone can edit (anonymously), we can't expect or trust editors to be good at identifying the truth (whatever that may be). We can verify that text is sourced reliably. Once we go beyond that, then it involves good-faith editor discussion and consensus forming on a case-by-case basis. Colin°^Talk 08:24, 16 December 2008 (UTC)

There is nothing wrong with identifying "reliable". So, identify all the key characteristics of a reliable source. Perhaps that is already accomplished, perhaps it needs to be touched up. Next, indicate that reviews can be less reliable the more these key characteristics stray from the ideal. Finally, state that reviews missing certain characteristics are not at all reliable. That would work for me.--scuro (talk) 12:34, 16 December 2008 (UTC)

I do not think that what you want is possible. There are far too many variables to provide a comprehensive list that would work in every single instance. Sometimes editors have to use their best judgment instead of mindlessly following a checklist. We have already provided general advice that is likely to fit most cases. If we attempt to make it more specific, we will make it less useful, less applicable to any given situation, and more likely to have it misapplied. There will always be situations that are not covered by the guidelines: even if we list a hundred criteria, from "source of funding" through "authors' criminal record", there will be a case that is not considered. Editors must use good judgment! (I thought of this discussion when I read WP:PEREN#Define_reliable_sources today.)

Furthermore, your putative problem is not specific to medical reviews. While you claim that you only care about evaluating reviews, every single complaint you've made about being unable to prove that your personal estimation of a review's value applies at least as much to the difficulty of evaluating primary sources. I will instead suggest this principle to you: if you can't articulate to other editors any reason for rejecting a secondary source that is more specific and more logical-sounding than "I don't like it", then you should not reject it. If, on the other hand, you can name reasons that seem reasonable, appropriate, proportionate, and consistent to both yourself and the other editors of an article, then you have very likely made a good decision. WhatamIdoing (talk) 20:54, 16 December 2008 (UTC)

I think it is time we wrapped up this discussion. It is going around in circles, it discusses issues well beyond the scope of this guideline alone, and seems to revolve about one editor sowing seeds of confusion. Can we move on please? JFW | T@lk 22:21, 16 December 2008 (UTC)

Bestbets

A few of us are wondering if Bestbets are sufficient to be used as references for Wikipedia? They often ask questions that are not looked at elsewhere in the literature. If better sources can be found I think they should be. But I think that this is a reasonable source to add to pages that might have few or no references. see http://www.bestbets.org/home/bets-introduction.php Would like to get others comments. --Doc James (talk) 13:37, 21 December 2008 (UTC)

I agree that BestBETS often addresses problems that are otherwise very hard to find in the medical literature. At the same time, it is a website without a formal peer review process. I'm unsure if it is the right kind of source... JFW | T@lk 13:45, 21 December 2008 (UTC)

Today for example I replaced the quote that cranberry juice is effective for UTIs to no evidence supports the use of cranberry juice for UTIs. see http://www.bestbets.org/bets/bet.php?id=1324 I could reference the 1998 Cochrane collaboration study but the best bets people assessed newer literature aswell. --Doc James (talk) 16:18, 21 December 2008 (UTC)

No proper peer review? Eeeek.--scuro (talk) 20:04, 24 December 2008 (UTC)

Journal supplement

The November issue of Epilepsia had a 133-page supplement on the Ketogenic diet. It contains nearly 40 short review papers. The introduction states:

The first International Symposium on Dietary Treatments for Epilepsy and Other Neurological Disorders was held in Phoenix, Arizona from April 2–5, 2008. This supplement comprises a summary of presentations at the symposium, describing advances in dietary therapies for epilepsy from both clinical and basic science perspectives...

...The major topics discussed in both plenary and mini-symposium presentations are contained with this volume in succinct mini-reviews...

Are such papers given the same editorial / peer-review scrutiny as reviews appearing in the main journal? I should very much like to use some of these as sources. They are up-to-date and together form a comprehensive resource. Any concerns? Colin°^Talk 21:45, 22 December 2008 (UTC)

Journal supplements are usually treated with peer review, but conference presentations themselves are usually not peer reviewed. I suspect the sources are reliable, but would be superseded by actual secondary sources if these are available. JFW | T@lk 22:21, 22 December 2008 (UTC)

Thanks, JFW. Just so I'm clear, you are saying these "mini-reviews" are conference presentations (and not reviews, based on earlier conference presentations). If I hadn't read the introduction to the supplement, I wouldn't be aware of the conference aspect and would just think this was a supplement on "Ketogenic diet and treatments". How are we to tell, especially if the review is found via a PubMed search, whether a review is peer-reviewed or not? BTW: by "mini-review", they don't mean 1/2 page. They range in size from 2–10 pages. I've added a link, above, to the publisher's page. Colin°^Talk 08:39, 23 December 2008 (UTC)

Generally speaking, I don't invariably trust PubMed's opinion as to whether something is a review. Sometimes PubMed labels an article a review when it is not a review, and vice versa. (Usually PubMed is right, but not always.) When in doubt, I get a copy of the journal containing the article, including the table of contents, any introductory editor remarks, and so forth. PDF copies sometimes give more info than HTML copies. This is a lot more work, though. Eubulides (talk) 19:50, 23 December 2008 (UTC)

Many of the papers describe themselves as reviews. I don't really have a problem with working out if the paper describes original research or previously published research. The concern I had was with spotting whether the paper had undergone the same scrutiny as conventional articles. One thing I have spotted, which I suppose comes from the nature of these papers, is that even if not describing previously unpublished research, they are often describing recently published research by the same author. I'd be concerned about possible bias if the editorial or peer review process was skipped, and the paper was merely an unedited opinion piece. Colin°^Talk 22:39, 23 December 2008 (UTC)

As your introduction indicates, these papers are based on presentations given at a conference. The content of such talks is usually not submitted for peer review. Therefore, derivative publications are likely to receive less peer review than papers submitted directly as reviews. Still, I'd be surprised if the editors of Epilepsia published these papers without any peer review whatsoever. JFW | T@lk 20:26, 23 December 2008 (UTC)

RSN

I've posted a question at RSN that relates to this guideline. It's here, and involves a patient's personal website about iguanas and chronic fatigue syndrome. Those that are familiar with Wikipedia's standards for medicine-related articles are invited (begged) to respond. WhatamIdoing (talk) 02:21, 26 January 2009 (UTC)

Citing sources you haven't read

About recent changes that I've just reverted: Editors may not cite sources that they have not read. This guideline must not provide advice that directly contradicts the main page on citing sources. WhatamIdoing (talk) 02:54, 24 February 2009 (UTC)

Reply

User:WhatamIdoing doesn't seem to understand Wikipedia:SAYWHEREYOUGOTIT. It puts the burden on the editor to verify that the original source gives that information, but it certainly does not preclude you from citing it after checking that it really gives that information.

“ It is improper to obtain a citation from an intermediate source without making clear that you saw only that intermediate source. For example, you might find some information on a Web page that is attributed to a certain book. Unless you look at the book yourself to check that the information is there, your source is really the Web page, which is what you must cite. The credibility of your article rests on the credibility of the Web page, as well as the book, and your article must make that clear. ”

Therefore, I'm going to revert that removal, and add a link to the Wikipedia:SAYWHEREYOUGOTIT making clear the onus for verifying that the original source concurs with the review rests with the editor making that change. Xasodfuih (talk) 04:16, 24 February 2009 (UTC)

Xasodfuih,

Please stop making significant changes to this guideline without discussing them. Here's the disputed text:

When a statement taken from a narrative review is supported there by only one or two original research articles, it's recommended to cite both the original work(s) and the narrative review, especially if the original articles were published in higher impact venues than the narrative review. Citing the original article(s) is a convenience for the reader, and avoids diminishing the importance of the results; citing the review gives the result some additional credibility beyond that conferred by the (often anonymous) reviewers of the original work.
Some journals specialize in the publication of third-party summaries of reviews published elsewhere, in particular summaries of systematic reviews, often accompanied by a short commentary. Unless such summaries/commentaries add significant additional insight, citing them instead of the original review must be avoided because this practice can confuse the reader as to the actual review authors. If these summaries/commentaries are cited along the original review, label them clearly as summary/commentary in the footnote.

Note that your text says only to cite the original publications. It does not mention ever bothering to read the original publications. Citing a paper without reading it is (1) prohibited and (2) what your text appears to recommend.

And again: this is a guideline. Your changes do not have consensus. Please stop editing the page until you've made a proposal here and determined, at minimum, that there's no active opposition. WhatamIdoing (talk) 05:31, 24 February 2009 (UTC)

I have added the caveat you inist on. Getting hung up on a single word ("cite") when the point here is choosing the right sources seems small potatoes to me. You could have rephrased it or added the caveat yourself instead of reverting. Xasodfuih (talk) 05:45, 24 February 2009 (UTC)

Case reports

Have had a lot of issue lately with editors using case reports for justification of just about everything. IMO these should not be allowed as evidence. I think this should be make perfectly clear.--Doc James (talk · contribs · email) 04:02, 24 February 2009 (UTC)

Case reports are basically necessary when writing about extremely rare diseases. (And that reminds me, I've got to get back to exactly that problem at Hyperhidrosis.) WhatamIdoing (talk) 05:32, 24 February 2009 (UTC)

This issue is similar to that of using other primary sources (RCTs etc). The problem is not with using them when other sources don't exist, but with overusing them to write an original review, especially one that contradicts published reviews. Template:Sciencereview can used to tag wiki articles that have this problem. Xasodfuih (talk) 06:06, 24 February 2009 (UTC)

uptodate.com, WebMD

I don't see anything written about these or medical sites in general. Should we added them as reliable sources? Xasodfuih (talk) 05:37, 24 February 2009 (UTC)

See Wikipedia talk:Reliable sources (medicine-related articles)/Archive 2#Uptodate.com. Colin°^Talk 12:49, 24 February 2009 (UTC)

Perhaps a 1-2 sentences summarizing the conclusion from there (read but don't cite directly because ... ) should be added to the guideline? After all, the guideline bothers to explain how to use google books... Xasodfuih (talk) 13:35, 24 February 2009 (UTC)

OK, how about something like this? Let's remove the last sentence of Biomedical journals and books and move it into into a new section Tertiary sources, which would appear after that section (and before Popular press), and would contain the following text:

"Biomedical encyclopedias published by established medical publishers are often of good quality but the information in there may be too terse for detailed articles. Other tertiary sources include online medical encyclopedias such as WebMD and UpToDate, along with journals such as Evidence-based Dentistry (ISSN 1462-0049) that publish expert third-party summaries of reviews published elsewhere. In general, tertiary sources can be useful guides about the relevant medical literature and how much weight to give different sources; however, as much as possible Wikipedia articles should prefer secondary sources."

Eubulides (talk) 18:25, 24 February 2009 (UTC)

Sounds good overall, although whether a specialized encyclopedia entry cites reviews or original research articles depends on the amount of literature available on the topic. Look at the free sample pages here: "Josephine domain" cites two primary sources, whereas "Jews and Genetic Diseases" cites at least one review. (By the way, there's no concise article about this on the wiki. Ashkenazi Jews#Specific diseases and disorders covers a narrower topic, but for the other populations, I don't see anything.) The same choice of sources is also true for most medical textbooks. Surely this book is much better source for writing about Machado-Joseph disease/Spinocerebellar Ataxia Type-3, but that distinction has nothing to with whether one of these books is a "secondary" source and the other "tertiary". I suspect the same observation holds about the choice of sources in WebMD and uptodate.com, although these two sites have different target audiences.

So, all of the above go together as tertiary sources if you insist on that classification, which I don't find very helpful: "The term tertiary source[1][2] is a relative term. What is considered tertiary depends on what is considered primary and secondary. A tertiary source may thus be understood as a selection, distillation, summary or compilation of primary sources, secondary sources, or both.[3][4][5]" (and the rest is unsourced) If a review cites another review it's automatically tertiary? In that line of reasoning the NHMRC report in water fluoridation is tertiary source, because it explicitly cites the York report in support of its conclusions (and for efficacy they explicitly say they've looked only one newer primary study). I would not bother drawing a line between secondary and tertiary sources for medical articles; it will lead to more artificial distinctions instead of considering the quality and level of coverage of source. In the case of third-party summaries of reviews published elsewhere it is easy enough to give a general recommendation, but we should do the same for uptodate.com/WebMd based on how good they are (and easy to cite), as opposed to WP:ARTIFICIALDISTINCTION between secondary and tertiary. Xasodfuih (talk) 05:24, 25 February 2009 (UTC)

Another observation is that some specialized encyclopedias may have a beefy overview section that are quite suitable as review source. Look at this one (using Amazon's look inside). Xasodfuih (talk) 08:23, 25 February 2009 (UTC)

OK, thanks for the comments; how about this idea instead? Add a new section Websites after Popular press, with the following content:

"Internet websites contain a wide range of biomedical information ranging from factual to fraudulent, with high percentage being of low quality. Peer-reviewed medical websites such as WebMD and UpToDate can be useful guides about the relevant medical literature and how much weight to give different sources; however, as much as possible Wikipedia articles should cite the literature directly."

Also, create a new paragraph after the 2nd paragraph of Biomedical journals, with the following content:

"A few journals, such as Evidence-based Dentistry (ISSN 1462-0049), publish third-party summaries of reviews and guidelines published elsewhere. If you have access to both the original source and a helpful summary, it is good practice to cite both sources together."

Finally, append the following example just before the last paragraph of Formatting citations:

"If you are citing a source along with an expert summary, it is helpful to list them together, with the main source first to indicate that it is more authoritative. For example:

• Griffin SO, Regnier E, Griffin PM, Huntley V (2007). "Effectiveness of fluoride in preventing caries in adults". J Dent Res. 86 (5): 410–5. doi:10.1177/154405910708600504. PMID 17452559. S2CID 20230525.{{cite journal}}: CS1 maint: multiple names: authors list (link) Summary: Yeung CA (2007). "Fluoride prevents caries among adults of all ages". Evid Based Dent. 8 (3): 72–3. doi:10.1038/sj.ebd.6400506. PMID 17891121.

"

Eubulides (talk) 00:43, 26 February 2009 (UTC)

I support adding to the guideline the 1st & 3rd paragraphs you proposed above. For the 2nd paragraph you propose I'd prefer that the 2nd sentence read: "If you have access to both the original source and the summary, and you find the summary helpful, it is good practice to cite both sources together (see [#formatting citations] for further details)." Xasodfuih (talk) 11:10, 26 February 2009 (UTC)

New guidance on citing twice

I have removed the following, which appears to be the personal opinion of Xasodfuih.

When a statement taken from a narrative review is supported there by only one or two original research articles, it's recommended to cite both the original work(s) and the narrative review, especially if the original articles were published in higher impact venues than the narrative review. Citing the original article(s) is a convenience for the reader, and avoids diminishing the importance of the results; citing the review gives the result some additional credibility beyond that conferred by the (often anonymous) reviewers of the original work.

Some journals specialize in the publication of third-party summaries of reviews published elsewhere, in particular summaries of systematic reviews, often accompanied by a short commentary. Unless such summaries/commentaries add significant additional insight, citing them instead of the original review must be avoided because this practice can confuse the reader as to the actual review authors. If these summaries/commentaries are cited along the original review, label them clearly as summary/commentary in the footnote.

WP:SAYWHEREYOUGOTIT strongly recommends that in situations like those outlined in the above two paragraphs, the editor citing the original source must not simply copy the reference from the review or summary, but must also check what the original source/review says.

WhatamIdoing has already removed an earlier version of this text. It has not been discussed by the community and is likely to be contentious. There has never been any requirement on WP to cite two sources when one alone is a reliable source. We are not an academic journal and have no obligation to cite the original research out of courtesy/acknowledgement to the original researchers, nor do we have the same issue with regarding the citation of a review as "lazy research". The middle paragraph seems to come from a recent issue at Water fluoridation‎ and I don't think it is a common problem. We must avoid knee-jerk updates to guidelines based on current events. I encourage Xasodfuih to patiently wait further discussion on this text and avoid restoring the contested text (even with minor changes). Colin°^Talk 08:57, 24 February 2009 (UTC)

WhatamIdoing removed it for a minor reason, which I had already addressed when you removed it. You however have contested the essence of the changes. There are two issues here, and I would like them to be discussed separately, so I'm making subsections: Xasodfuih (talk) 11:14, 24 February 2009 (UTC)

Actually, I oppose it for more reasons than simply its conflict with WP:SAYWHEREYOUGOTIT. The conflict with pan-Wiki standards deserved an immediate removal. I wouldn't have risked an edit war for my other reasons, which are WP:CREEP, risk of WP:SYN, failure to document actual practice (a requirement for guidelines), and a sense of it very commonly being an unnecessary, make-work standard. When it's necessary or appropriate, editors can do that on their own. This is not a reasonable standard for 99% of sources. WhatamIdoing (talk) 18:32, 24 February 2009 (UTC)

"Cherry picking" from narrative reviews

When a reviewer is citing many sources to make a point, it's indeed his WP:SYNT, so attributable mostly him, and it may also be impractical to cite all of his primary sources. However, when single statements are extracted from narrative reviews where they are supported by a single source, it sometimes makes sense to cite the original article along with the review, especially when:

1. the original paper may be published in a more prestigious venue; I sometimes dismiss narrative reviews in crappy venues, and so should you, given that review journals easily get a higher impact factor than the rest. After all, this guideline does name some prestigious journals, so they're not all equal on the wiki, and I'd worry if they were. If the review is in a (much) lesser venue than the original, citing just the review lowers the credibility.
2. the reviewer is citing a primary study of his own, so he's not really a 3rd party for that fact; see PANDAS for a train wreck written from a review written where the majority of the substantive references in the review are from the same research group.
3. the sentence is not put by the reviewer in his abstract or conclusions, so the weight of the statement may not be apparent; statements that are supported from a single source may not be incredibly important, so some extra scrutiny is welcome as to what they're based on.
4. the original paper writes at length about the issue, and gives us an abstract, which 99% of the time it's free to access, which makes it easier to verify articles. We already have a statement that encourages open access journals if quality is not compromised. Well, practically all abstract are open access, even if the full text is not. Giving an extra footnote that eases verification can't be a bad thing.

Finally, this is merely a suggestion in a guideline. Nobody will make a big fuss if this isn't always followed; 90% of the time reviewers at FA/GA do not read the sources (which is why we have FAs like anabolic steroids or paleolithic diet, but I'm digressing), so we're not going to see articles failing from standards in any numbers because of this recommendation. Xasodfuih (talk) 13:36, 24 February 2009 (UTC)

Your opening analysis (multiple sources vs single sources to make one point in a review) is over simplistic. Sometimes an author cites several studies because all those studies confirm the same point. No synthesis is involved, merely weight of evidence. Sometimes an author cites one study but is in fact deriving considerably from his own experience/views to build on that study. And so on. It really is not our job to second-guess what the reviewer is doing with his sources. It is not the purpose of this guideline to teach editors how to criticise and analyse every point in a review paper. Friendly talk-page discussion on these finer points is great, but is really not expected of editors.

Point 1. WP demands that we cite the source we read (and read in full, not just an abstract). There is no requirement on WP to double-up a citation because it might be inadequate. If you believe the review is "crappy" either because you think it is full of mistakes or was published in a seriously weak journal, then don't use it.

Point 2. Don't confuse "primary/secondary source" with "first/third party". They are separate issues and WP:PRIMARY is not the relevant policy. We have a WP:NPOV policy. An incestuous review is not necessarily an unreliable one, particularly for uncontroversial topics and/or topics with only a few researchers.

Point 3. I'm not clear about this one. Reviews rarely have useful abstracts. They usually just say "We review the literature on ...". Again, you talk about "statements that are supported from a single source" which is a simplistic way of judging the quality of a statement.

Point 4. Don't go there. Abstracts are not substitute sources any more than a movie trailer is a substitute for watching the movie.

The issues above ignore the benefits of a fact merely being stated in a peer-reviewed review paper provides over the "same" fact in the original research paper. It also assumes there is a 1:1 mapping from statement in a review to the citations in the review and therefore the cited sources are equivalent. This is often manifest on WP when someone wants to prove a fact to the reader, they say "study X found Y". This isn't encyclopaedic. An encyclopaedia just says "Y". We state facts and cite reliable sources that back up those facts. We should be less concerned with whether those facts came from research paper A or papers B, C and D.

Here's what I wrote at the Water Fluoridation FAC on this topic:

It is far better to cite the review by X rather than the study of Y in the example you give. When you cite the review, we gain "review author X (an expert) believes study Y is worth citing and shows Z is true, and the folk who peer-reviewed the review (also experts) believe study Y is worth citing and shows Z is true, and the editors of the journal that published the review also believe...." If you just cite primary study Y, all we have is "wikipedian abc thinks study Y is worth citing..." which isn't worth anything really. Secondly, and especially so for such an old study, we gain immensely from citing recent reviews because they indicate that the primary study is still relevant, hasn't been superseded or discredited. You rightly say you wouldn't do this in an academic paper, but academic papers are written by (known) academics, not anonymous wikipedians. We benefit from the extra authority that the review provides, even if it is just a literature review and isn't systematic.

Colin°^Talk 13:45, 24 February 2009 (UTC)

I find your theoretical presentation somewhat tendentious given that the study Y is peer-reviewed as well, and if it's published in a better venue, it's usually peer-reviewed by more qualified experts. Furthermore, you don't seem to understand that I'm proposing to cite the primary source in addition not instead of the review in limited circumstances. So, it's the review authors, and the reviewers of the review that "approve" of us citing the primary source in that context (not wikipedian abc), and the reader can verify that chain of expert authority if he wishes (by looking in the review to see that indeed it cited that primary study we also cite in that context). Look for instance at the last sentence from Fit for Life#Controversy—an entry into a specialized encyclopaedia is a review, but it's a narrative one. You'd have to look in the book to see who was in the advisory board, i.e. who peer-reviewed the review. For anonymous peer-reviews, you don't even have that info and have to go by the journal's reputation. Someone may question whether an encyclopaedia entry like that is biased, question where the claimed scientific evidence is coming from, etc. Can you honestly say that the additional link to the primary paper cited in that case is not helpful? Xasodfuih (talk) 16:37, 24 February 2009 (UTC)

Both the primary research and the review could be published in the Lancet for all I care. They are different kinds of sources and the former is much weaker for our purposes. The secondary source (review) generally widens the scope of the conclusions and assesses it against any similar studies, if any. Look at the conclusion of your diet paper: "In summary at identical energy intake and similar substrate composition, the dissociated (or 'food combining') diet did not bring any additional loss in weight and body fat." Notice the "did not". They are saying "When we tried it, it didn't work". They are judged on whether their study was well designed, the data analysed correctly and the conclusions reasonable. They do not claim that "Dissociated diets (including those promoted by FFL) do not achieve weight loss greater than a balanced diet." A review might and it might cite that paper. It is for the review author to judge whether that study is compelling enough for him to generalise.

I'm aware of what you are requesting and repeat that additional citations are not how WP works. The purpose of citations is to say where you got the information that backs up the text. We do not use citations to prove a point; we expect our sources to prove the point. This is such a key point and why so many POV pushers want to cite primary studies: because it helps them prove a point when they can't find a reliable secondary source to back them up. This is the mistake made in the Fit for Life example. It is as though "The Gale Encyclopedia of Diets" is not considered a sufficient source for the text of the final sentence. Citing the primary paper is actually unhelpful. The primary paper makes no mention of "Fit for Life" so it is WP:OR for us to use the analysis of a particular diet type to criticise the particular diet that FFL promote. If the Gale Encyclopedia does that then fine. In addition, the study is open to criticism (small number of participants, in-hospital setting, possibility that the form of dissociated diet studied is different to that used by FFL, only obese patients studied and not those merely overweight, etc). Therefore, someone could argue with the WP editor who cited it as to whether it is relevant. It really needs a degree of expertise to know if it is relevant, and on WP none of us are trusted as experts. Such arguments disappear if you just cite the Gale Encyclopedia. Assuming the Encyclopedia is considered reliable and written by experts, then we build on their expertise in judging the primary literature, not ours. Colin°^Talk 17:32, 24 February 2009 (UTC)

• I don't think anybody's objecting to the practice of citing primary sources on occasion alongside a review; the disagreement is more on how desirable and often this should be done. I agree with Xasodfuih that there are some cases (a weak 3rd-party commentary which cites one strong systematic review to support a point) where it's better to cite the lower-order source on a point. I also share Colin's unease with a guideline that recommends citing primary sources when a review cites only one or two primary sources; this will lead to massive overciting in Wikipedia, and most of it will be unnecessary bloat. In a topic that is well covered by reviews, I prefer citing a primary source directly only if the study is worth discussing directly in the text. I realize that there are other valid styles but the guideline shouldn't be imposing the styles in an area were we don't have a reasonable consensus.
• Also, often an editor has access to the review but not to the source the review cites; if the review is reliable it's not reasonable to insist that the editor go dig up the primary source from a medical library somewhere.
• Xasodfuih also suggested adding text about third-party summaries. Here I agree that some additional guidance could be helpful, and have proposed some wording in #uptodate.com, WebMD above.

Eubulides (talk) 18:25, 24 February 2009 (UTC)

I fully agree that this guideline should not be imposing this standard on all editors. Xasodifuih should feel free to do that (so long as s/he reads every single paper) whenever it is wanted, but we simply should not impose this on every editor. WhatamIdoing (talk) 18:39, 24 February 2009 (UTC)

Okay, we've agreed to disagree on this one. Thank you all for explaining you reasoning in detail. I won't try to impose my preference on you via the guideline; I consider this matter closed. Xasodfuih (talk) 03:47, 25 February 2009 (UTC)

This is one of the disagreements that User:Paul gene and I had originally, so you're not the only one. I don't like obfuscating the original sources either. II | (t - c) 18:36, 26 February 2009 (UTC)

Note that in standard style guidelines, it's often encouraged to cite the original source while also noting "as cited in ..." -- this could be easily done on Wikipedia, where the primary source is cited, but the footnote also contains: "As cited in: [review]". I don't see why not. People often assume that everyone has great access to medical journals, and it's just not so. Hiding a weak source behind a review is not necessarily a good thing. II | (t - c) 00:21, 11 March 2009 (UTC)

I think our bigger concern is hiding a lay source behind any scientific paper. You should not cite an scientific paper when you've only read the press release. WhatamIdoing (talk) 04:45, 11 March 2009 (UTC)

Using journals that specialize in summaries of reviews published elsewhere

We give all sorts of advice on which sources to use and which not. I don't see why we should say something about this type of journals. If you feel that that something needs to be something else, please formulate an alternative. The fact that the issue doesn't come up very often means that it shouldn't be discussed at length, but since it has come up in a FAC, it means it's not negligible either. Xasodfuih (talk) 11:14, 24 February 2009 (UTC)

The issue that affects this guideline is whether such journals or such summaries of reviews are reliable sources. I'd be interested to hear what folk think about that (some examples of those journals/papers would be useful). It is irrelevant as to whether such articles "add significant additional insight" or whether we might "confuse the reader as to the actual review authors". You cite what you used as a source. I don't have a problem with the last sentence proposed, but it seems a bit pointless to append a citation to a summary of the actual source used (the main review) unless the summary is open-access but the review is not. Colin°^Talk 12:58, 24 February 2009 (UTC)

Some examples I've encountered recently are NPG's Evidence-based dentistry (not in JCR) and Elsevier's Dental Abstracts (prolly not in JCR either, have not checked). Xasodfuih (talk) 13:42, 24 February 2009 (UTC)

• I find it useful to cite summaries of reviews, for the same reason I find it useful to use |laysummary= when citing journal articles: the summaries are often far more readable than the full report, and this will help the typical Wikipedia reader. In the dentistry example that I believe prompted this thread, I've changed the citation to look like this:

• National Health and Medical Research Council (Australia) (2007). "A systematic review of the efficacy and safety of fluoridation" (PDF). Retrieved February 24, 2009. Summary: Yeung CA (2008). "A systematic review of the efficacy and safety of fluoridation". Evid Based Dent. 9 (2): 39–43. doi:10.1038/sj.ebd.6400578. PMID 18584000. {{cite journal}}: Unknown parameter |lay-date= ignored (help); Unknown parameter |lay-source= ignored (help); Unknown parameter |laysummary= ignored (help)

• This cites the main review (which is along), along with two summaries, one by the authors of the review and one by a 3rd party (Yeung). This is a fairly extreme case, but it's not inappropriate, as fluoridation is a contentious topic and the cited source is the most recent systematic review published on the topic.

Eubulides (talk) 18:25, 24 February 2009 (UTC)

• The whole point of my addition to the guideline in this matter was to support exactly the practice you described above, which we've converged on at Water fluoridation, i.e. label summaries as summaries, don't substitute them for the actual review when the summary is written by other authors because it can lead to confusion. Parenthetically, "laysummary=" is not a good choice if the summary has a PMID, but I think this is obvious enough that I didn't mention it. Xasodfuih (talk) 03:40, 25 February 2009 (UTC)

I think Xasodfuih has it right--There are two types of these--one is where they simply reproduce the abstracts from the original paper, as the authors of the paper wrote them. (as is the case for the abstracts in PubMed, and many other services). This can be a convenient source of information, but the original paper must be cited as well, even if you didnt actually see it--you just say so. But the ones written by others are their interpretation, and may or may not represent the material accurately. DGG (talk) 05:14, 27 February 2009 (UTC)

Medicine and the popular press

An interesting and relevant article from the New England Journal of Medicine on the difficulties and pitfalls inherent in popular-press coverage of science and medicine is here. I thought it might be relevant, and perhaps even useful as a source. MastCell ^Talk 20:34, 8 January 2009 (UTC)

Thanks, I added the phrase "news articles too often convey wrong or misleading information about health care" and cited that new source. Eubulides (talk) 21:14, 8 January 2009 (UTC)

I think "too often" is an exaggeration based on the anecdotal evidence in that article (he cites a handful of examples). So I'm changing it to "sometimes". Xasodfuih (talk) 11:36, 24 February 2009 (UTC)

He's slamming WaPo for instance for saying something along the lines "antidepressants don't work half the time", when he thinks that the fact that they do work half the time a great achievement, so in his view WaPo should have written the article with that spin. Spin on spin. Xasodfuih (talk) 11:39, 24 February 2009 (UTC)

Take a look at the current front page of Health News Review; it cites a recent ABC News story on "cooking" cancers, a recent ABC News story on platelet-rich plasma, and a recent CBS News story on lunchtime liposuction, all published February 16 or later, and all terribly misleading or wrong. It's quite bad in today's journalism, at least in the English-language press. Anyway, the cited source says "all too frequently" and it's straying to far from that to change it to "sometimes"; I reworded it at "all too often". Eubulides (talk) 18:25, 24 February 2009 (UTC)

"All too often" is a meaningless term. In my experience, it's a term that people use when they don't have accurate figure. It's non-falsifiable. They could say that once is too often. In a book, magazine, or legal writing, editors strike out terms like that and send the writer back to get the actual figures.

How often is "all too often"? What's the difference in meaning between "too often" and "all too often"? Is once "too often"? More than half? 20%?

All too often, this MOS is nonsense.

It's bad writing, because it's meaningless. In WP terms, it seems to violate WP:WEASEL and WP:PEACOCK. Nbauman (talk) 18:49, 24 February 2009 (UTC)

The phrase in the cited article is "all too frequently". The editors of the NEJM didn't strike it out and "send the writer back to get the actual figures". The word "too" is the key and is a matter of opinion which can't be derived from raw data. In the opinion of the NEJM article writer, and most of the contributors to this page, journalists say "wrong or misleading" things about health too often for us to consider them generally reliable. Sorry if that offends you but that's our opinion. Wrt WEASEL/PEACOCK, WP guidelines do not have to follow WP guidelines or policy (none of them do). Colin°^Talk 19:06, 24 February 2009 (UTC)

I read that Dentzer article when it came out in the NEJM, and I didn't think it was particularly useful -- just a lot of de gustibus judgments about how she disagreed with the way the press handled particular stories. I didn't think the Jim Lehrer NewsHour coverage of health care policy was that good, as others have argued [citations omitted].

You could just as easily say that the NEJM gets its peer-reviewed articles wrong "all too frequently". Even 1 is too frequently. You could say that any publication makes mistakes "all too frequently". You could say that doctors make mistakes or commit crimes "all too frequently". Therefore, the phrase is nonsense. If you want to write nonsense, go ahead.

It doesn't offend me that most of the contributors to this page believe that journalists say "wrong or misleading" things about health too often for you to consider them "generally reliable."

It does offend me that you say that without any supporting evidence, evidence of the quality that you would expect others to use -- evidence in peer-reviewed journals, academic studies, or reviews of those studies. Indeed, you say that despite contrary evidence.

Instead, you say that on the basis only of your own personal opinions and prejudices, against newspapers and against things that ordinary people read.

This entry makes broad, sweeping generalizations attacking newspapers, contrary to the evidence in the peer-reviewed literature. And you say that you can do that because WP guidelines are merely opinion, and don't have to be supported by WP:RS or anything else, including facts -- as long as it's agreed on by a majority of the editors working on the guideline.

If you want to ignore the scientific method that you profess to champion, go ahead. Nbauman (talk) 21:19, 24 February 2009 (UTC)

Please can you cite the study in the "peer-reviewed literature" that concludes that newspapers are consistently accurate in their reporting of health issues? We need "consistent accuracy" for a type of source to be judged "reliable", not the odd example of outstanding journalism. My "prejudices" are amply confirmed by reading NHS Choices. Colin°^Talk 21:36, 24 February 2009 (UTC)

• Like Colin, I don't know of any "contrary evidence" on this issue; that is, I don't know of any reliable source that contradicts our reliable sources on this particular point.
• I would favor changing "all too often" back to "too often"; the "all" isn't really needed here and is I think part of the problem. However, changing "too often" to "sometimes" would go too far from what is in the source, Dentzer 2009 (PMID 19118299).
• "Too often" means too often to be a reliable source in the WP:MEDRS sense. I think this is reasonably obvious in context, but if not, I suppose we could insert further text to clarify this.
• It's not just this source; for example, Schwitzer 2008 (PMID 18507496) writes that U.S. news articles are "often ... imbalanced" and that "We believe this imbalance may have a profound impact on the decision making of American consumers who rely on these news stories." Schwitzer goes on to say that "People may be misled, become anxious, or make ill-informed decisions based on" the kind of incomplete stories that often are published by news organizations.

Eubulides (talk) 21:47, 24 February 2009 (UTC)

(Undent) Nbauman, you write professionally. Think about the last few science-related pieces you've done. About how often did you, in your professional writing, rely entirely on what some other journalist wrote for a scientific/medical fact, without bothering to check it against a journal article, textbook, or other source? Never? Always? Something in between? WhatamIdoing (talk) 22:30, 24 February 2009 (UTC)

In my writings, I have never relied upon another journalist. My job is basically to check the facts with the doctors who did the original research, and confirm with them that I understand their work correctly. And I would never depend on my own understanding of a published source. I would talk to the author to make sure I got it right. Many news stories, and many WP articles, are misunderstandings of published sources.

Some of you here may be medical doctors or its equivalent, but you don't understand journalism, and particularly medical journalism, at all. You don't know how it works. I wonder if you've ever spoken to a journalist about medical coverage, or called a journalist to complain when you saw a story that was wrong. I doubt it, because you're engaging in prejudiced stereotyping.

There are techniques that some journalists, and particularly medical journalists, use to make sure they get their facts right.

The important point that you're missing is that there is a great variation in journalism, some of it reliable, some of it not reliable, and there are ways of distinguishing between them. That's what Schwitzer studies. This guideline completely brushes that research aside, and doesn't give any guidance on how to distinguish them. You're choosing ignorance. I can't stop you. Nbauman (talk) 23:38, 24 February 2009 (UTC)

So you always reject a journalist's summary or interpretation for this purpose in your professional work, but you think that we should encourage editors to use a lower standard than the one you yourself use. A newspaper article isn't a good enough source for scientific facts for you, but you keep arguing that it ought to be good enough for Wikipedia, right? WhatamIdoing (talk) 05:12, 26 February 2009 (UTC)

That sounds like a trick question, which you could have used against me no matter what I answered.

No, your interpretation is wrong.

First, I said that I would verify my understanding of any published source -- newspaper or peer-reviewed article -- with the original investigator.

I have a good sense of how accurate newspaper stories are, because I often use newspaper stories as part of my research, and when I talk to the investigator I ask him whether the newspaper story about his work was correct. Some are, some aren't. Some newspapers, like the Wall Street Journal, have a widespread reputation for accuracy. Doctors themselves read the New York Times. I've been amazed to find out sometimes that even leading investigators had read the NYT article, but not the original journal article, about something important in their own field.

That's why I say you're ignoring the data. You haven't spoken to investigators and asked them whether they thought the newspaper coverage of their work was accurate. You would find out that some publications are accurate and some aren't. Instead, you're cherry-picking the newspaper stories and opinion pieces that fit your prejudices.

Second, the standard of accuracy that I use for my published stories is very high, because the fact-checking is my responsibility alone. (And I get paid for it.) After I publish it, I can't correct it any more.

The standard of accuracy that everyone uses on WP is not that high, because other editors will review the changes after they're put on the Internet and correct it if necessary. That's the whole idea behind WP.

That's all consistent with my main point: There is good scientific data, published by authors like Schwitzer, in the academic peer-reviewed journals, the standard you profess to follow, that newspaper accuracy is more complex than you claim it is, and that many newspapers cover medicine very well. And yet you refuse to follow those academic standards in this MOS, but instead insert your own unfair prejudices against newspapers. Nbauman (talk) 06:51, 26 February 2009 (UTC)

This guideline does not say that all newspapers and magazines get everything wrong every time, and where it dings popular press is for widely agreed deficiencies, such as reporting risks out of context.

As I'm sure I have mentioned several times during these ongoing conversations, I have spoken with investigators on this subject. I know a few that simply refuse all interview requests after getting burned with egregious errors (like "We have isolated a new compound that might have interesting biological effects" being turned into "New drug will be on the market soon"): They just ignore requests and direct the occasional persistent person to the publicity office. I know more who will only answer questions in writing. I even know one person that agreed to an interview on condition of proofing the story: She corrected the proof, and then had serious factual errors introduced by an editor. Of the people that have had their work written up in three or more popular press articles, I know no one that says it was adequately and accurately represented in all of them.

Presumably these are the people that aren't returning your phone calls, so they're not in your sample set.

Wikipedia should aim higher than just getting most of the facts right most of the time. We should have a high standard of accuracy. There's no harm in our recommendation that people get it right the first time. WhatamIdoing (talk) 19:48, 26 February 2009 (UTC)

Some science journalism is accurate some is not. Before adding info to wikipedia one should verify the accuracy to a peer reviewed source. Up in the cold cold north we play a game were you site in a circle and whisper from one person to next to see what comes back to you. This recreates the accuracy of moving farther from the source.--Doc James (talk · contribs · email) 20:29, 26 February 2009 (UTC)

I finally got a chance

I finally got a chance to read Susan Dentzer's NEJM piece carefully. I realize now that WP:MEDMOS distorts Denzer's article through selective misquotation. MEDMOS says

news articles too often convey wrong or misleading information about health care.

Dentzer actually wrote

Often these messages are delivered effectively by seasoned reporters who perform thoughtfully even in the face of breaking news and tight deadlines. But all too frequently, what is conveyed about health by many other journalists is wrong or misleading.

In other words, Dentzer said what I've repeatedly said, that (1) some popular news sources are accurate, while (2) others are not. MEDMOS selectively paraphrases Dentzer to make it sound as if she made a one-sided statement supporting (2), even though she actually made a balanced statement.

So here on WP we have critics of the popular press selectively quoting and distorting a source, to make the point that they're trying to push -- that the popular press is unreliable -- rather than accurately reporting what the source herself actually believes -- that the popular press is diverse.

I always take it seriously when people criticize the press, because I have an obligation to find out if I'm doing things wrong and to correct my procedures if I am.

However, critics often charge the press with the very errors that they, ironically, are committing themselves. That seems to be the case here -- cherry-picking one sentence in an article, while ignoring the other.

I will be interested to see whether you keep that distortion of Dentzer in the entry, or whether you change it. Nbauman (talk) 02:35, 24 March 2009 (UTC)

The preceding comment greately misrepresents the main thrust of Dentzer 2009 (PMID 19118299), by selectively quoting one of the few positive things it says about the popular press. The bulk of Dentzer's piece is critical of the press: this includes not only its title "Communicating Medical News — Pitfalls of Health Care Journalism", but also the topics of most of its paragraphs, ranging from "The news media need to become more knowledgeable and to embrace more fully our role in delivering to the public accurate, complete, and balanced messages about health." to "More broadly, a problem that is worsening in this era of the 24/7 news cycle is the frequent failure to put new developments into any kind of reasonable context for readers or viewers.". The piece's concluding two paragraphs lead with "In my view, we in the news media have a responsibility to hold ourselves to higher standards if there is any chance that doctors and patients will act on the basis of our reporting.", which pretty much sums up Dentzer's opinion. The existing summary in WP:MEDMOS is fairly accurate, but like any brief summary, it is not perfect: if you can propose better (brief) wording that does not misrepresent Dentzer's piece, please do so here. Eubulides (talk) 06:45, 24 March 2009 (UTC)

I "selectively quoted" from the introductory paragraph of the article. The introductory paragraph is where writers state the main thrust of their argument clearly to frame and qualify what follows. Dentzer states at the beginning that often messages are delivered effectively, but often they are not. The article deals with examples in which they were not. Nowhere in the article does Dentzer say that her examples characterize all medical stories, or medical journalism in general.

Her argument is that bad stories are done because some reporters don't understand medicine. She acknowledges in the introduction that some writers do understand medicine.

If you're going to quote Dentzer, you should quote what she actually said -- the positive and negative. If we can't agree on a summary, then we should use the full quote:

Often these messages are delivered effectively by seasoned reporters who perform thoughtfully even in the face of breaking news and tight deadlines. But all too frequently, what is conveyed about health by many other journalists is wrong or misleading.

I would paraphrase it as:

Some articles in the popular press, particularly those by experienced reporters, are accurate, while others are wrong or misleading.

That's what she said -- positive and negative. To use the negative, and ignore the positive, is wrong and misleading. Nbauman (talk) 15:45, 24 March 2009 (UTC)

I think that our advice lines up with the main thrust of Dentzer's piece, but I also don't think it much matters. The advice from Wikipedia editors, to other Wikipedia editors, is still the same: avoid using the popular press as your sole source for a medical or scientific fact. Our advice would be the same even if Dentzer had never written that piece.

If you ever find a situation where our deprecation of sole reliance on the popular press actually prevents you from writing a good article, please let us know. Until then, even if it hurts your professional pride to have known limitations called out without a placating admission that not every journalist gets every fact wrong every time, the advice about preferring the scientific literature to the popular press stands. WhatamIdoing (talk) 19:34, 24 March 2009 (UTC)

If you are attributing a statement to Dentzer, then it does matter whether your statement "lines up" with the source or not. The question is whether you want to use a misleading summary or not. You are insisting on using a misleading summary in order to advance your personal prejudices about the press.

Some people say that what matters on Wikipedia is not truth, but verification.

Now you're saying that neither truth nor verification matter. I want to get that on the record. This discussion is getting too serious, and we need some entertainment.

(BTW, you're misrepresenting my position too (Straw man). I never said that anyone should use the popular press as the sole source. And I never said that every journalist gets every fact right every time; I cited Schwitzer.) Nbauman (talk) 22:21, 24 March 2009 (UTC)

The proposed text is not an improvement on what's already in WP:MEDRS, as it less-accurately summarizes Dentzer 2009 (PMID 19118299) than the current text does. Dentzer's primary thrust, from its title to its conclusion (which is in its last 2 paragraphs), focuses on limitations and drawbacks of the popular press when reporting medical issues. The proposed text contains 13 words on the strengths of the popular press, and 6 words on its weaknesses, which is a seriously out-of-whack characterization of Dentzer from a WP:WEIGHT point of view. I'm not saying that the current text in WP:MEDRS can't be improved (far from it) but it's not an improvement to replace it with such a misleading summary. Perhaps you could give it another try, which summarizes Dentzer more fairly? 04:58, 25 March 2009 (UTC)

The subject of Denzer's article is specialist vs. generalist medical journalists.

Dentzer makes a standard argument, which I've heard many times among science and medical journalists, which is that specialist medical journalists can cover medicine better than generalist journalists.

Dentzer says in the introduction that specialist ("seasoned") journalists can deliver the messages well. She then goes on to give examples of non-specialist or poorly-trained journalists who (she believes) got the story wrong.

In other words, she believes that some journalists (specialists) are reliable, and others (poorly trained) are not.

The editors of this guideline believe something different. They believe that the popular press is "generally" not reliable. That is not what Dentzer said. Dentzer said what I said, and what Schwitzer said, which is that some journalists in the popular press are reliable and some are not.

Any paraphrase or citation of Dentzer's article, that supports your believe that the popular press is "generally" not reliable, is a deceptive misquotation or misattribution of Dentzer.

Any reference to Dentzer must preserve the idea that some journalists in the popular press are reliable, and some are not.

I think it's ironic that the people who fault the popular press for not reporting the medical literature reliably, are themselves deliberately distorting the medical literature to support their own prejudices and attribute a view to the author that she doesn't state or believe. But that's what makes Wikipedia so entertaining. Nbauman (talk) 17:07, 25 March 2009 (UTC)

Please see #Popular press is generally not reliable below. Eubulides (talk) 17:55, 25 March 2009 (UTC)

Popular press is generally not reliable

• "Dentzer makes a standard argument ... that specialist medical journalists can cover medicine better than generalist journalists." This is a misleading summary of Dentzer. Dentzer never says either "specialist" or "generalist", or anything like that. Her main point is news articles' "frequent failure to put new developments into any kind of reasonable context for readers or viewers". She also mentions lack of training, but that is secondary, and she never argues for specialist (as opposed to generalist) reporters.
• "The editors of this guideline believe something different. They believe that the popular press is "generally" not reliable." This is pretty much what Dentzer 2009 (PMID 19118299) wrote (albeit more politely, which is appropriate for the NEJM). Here's Dentzer's main conclusion:

"In my view, we in the news media have a responsibility to hold ourselves to higher standards if there is any chance that doctors and patients will act on the basis of our reporting. We are not clinicians, but we must be more than carnival barkers; we must be credible health communicators more interested in conveying clear, actionable health information to the public than carrying out our other agendas. There is strong evidence that many journalists agree — and in particular, consider themselves poorly trained to understand medical studies and statistics.(Voss 2002, PMID 12084701) But not only should our profession demand better training of health journalists, it should also require that health stories, rather than being rendered in black and white, use all the grays on the palette to paint a comprehensive picture of inevitably complex realties."

• Again, I'm not saying the current summary of Dentzer can't be improved on, but it summarizes Dentzer far more accurately and fairly than the proposed replacement. Please try to think of a better proposal.

Eubulides (talk) 17:55, 25 March 2009 (UTC)

You said,

Dentzer never says either "specialist" or "generalist", or anything like that.

Dentzer said:

Often these messages are delivered effectively by seasoned reporters who perform thoughtfully even in the face of breaking news and tight deadlines.

What do you think "seasoned reporters" means? Nbauman (talk) 18:51, 25 March 2009 (UTC)

I think it means "experienced", and so does the dictionary. See definition 2b. Experience and training are different. Dentzer believes that a veteran reporter that has spent years reporting all kinds of health stories will do better than a newbie that took one class on the specific subject. WhatamIdoing (talk) 20:04, 25 March 2009 (UTC)

I was asking Eubulides. Nbauman (talk) 20:48, 25 March 2009 (UTC)

I agree with WhatamIdoing that "seasoned" means "experienced", not "specialist". But isn't this a side issue? Eubulides (talk) 21:44, 25 March 2009 (UTC)

It's not a side issue, it's a factual distortion of a source.

Do you agree, then, that Dentzer is saying:

Often these messages are delivered effectively by experienced reporters.

Nbauman (talk) 00:00, 26 March 2009 (UTC)

Certainly it's a side issue, as far as Dentzer is concerned. The main "factual distortion of a source" that is going on in this subthread is the overemphasis on Dentzer's pro forma concession that news articles often get it right. But that concession is not Dentzer's main point; it's not even a secondary topic in her piece. And it should not be the main point here, where we have many more fish to fry than Dentzer's relatively narrow topic. It's not at all clear that it should be mentioned here at all. Eubulides (talk) 01:51, 26 March 2009 (UTC)

Eubulides, please answer my question. Do you agree that Dentzer is saying:

Often these messages are delivered effectively by experienced reporters.

Nbauman (talk) 02:47, 26 March 2009 (UTC)

I don't agree that that is Dentzer's main or even secondary point, no. It is true that Dentzer briefly concedes that point before making her main arguments about the relatively low quality of news articles as sources for medical information. Here's what Dentzer writes:

"Often these messages are delivered effectively by seasoned reporters who perform thoughtfully even in the face of breaking news and tight deadlines. But all too frequently, what is conveyed about health by many other journalists is wrong or misleading. Some distortion is attributable to ignorance or an inability to interpret and convey the nuanced results of clinical studies. And some is due to uncertainty about journalists' proper role: Is our job to describe the bigger picture, or simply to report what is "new"? Should we present black-and-white versions of reality that lend themselves to stark headlines, rather than grayer complexities that are harder to distill into simple truths? ¶ I believe that when journalists ignore complexities or fail to provide context, the public health messages they convey are inevitably inadequate or distorted. The news media need to become more knowledgeable and to embrace more fully our role in delivering to the public accurate, complete, and balanced messages about health. With some additional skills, care, and introspection — and a change in priorities — we can produce coverage more in line with our responsibilities."

Dentzer goes on to give multiple examples of news articles getting it wrong and explaining why this occurs. Overall, it would be misleading to summarize her comments with "Some articles in the popular press, particularly those by experienced reporters, are accurate, while others are wrong or misleading.", which is the text most-recently proposed. Eubulides (talk) 03:13, 26 March 2009 (UTC)

What about Although some articles in the popular press are accurate, many others are inadequate, misleading, or even completely wrong. Tim Vickers (talk) 03:31, 26 March 2009 (UTC)

Yes, that's better than what we've got now.

But I think we should also help Wikipedia editors to figure out which popular press articles are likely to be accurate and which are likely to be wrong. Dentzer made the point that experienced medical journalists are often accurate. I think that point also belongs in the guideline. Don't you agree? Nbauman (talk) 03:43, 26 March 2009 (UTC)

Interesting idea, how would you do this in practice? For example, how experienced is the author of this story? Tim Vickers (talk) 04:16, 26 March 2009 (UTC)

I think that asking the editor to divine the reporter's level of (relevant) experience is asking too much.

Tim, the problem with your proposal is that a statement can be technically "accurate" while also being "inadequate" and consequently "misleading". WhatamIdoing (talk) 04:30, 26 March 2009 (UTC)

Tim, in practice, the way to help WP editors figure out which popular press articles are accurate or wrong is to link to the many free web sites that have been set up by collaborations of journalists, social scientists, doctors and academics for that purpose, such as this and this. These include evidence-based, peer-reviewed studies of the press coverage of medicine.

I wanted to do that, but I was outvoted by editors who insisted that the entire popular press was unreliable. If you start by saying that the entire popular press is unreliable, then you can't go on to explain how some popular press sources are reliable and others are unreliable.

In the case of the Bloomberg story on tuberculosis, you could start by rating it on these criteria. There are other ways of rating it. There are many ways of rating news stories, and they've been discussed extensively in the peer-reviewed medical literature, in BMJ and elsewhere.

You can tell the reporter's level of skill by how well he meets these criteria. (Actually, what matters is not his experience, but whether he writes an accurate story.) These criteria also help you figure out whether the article is adequate or misleading. Nbauman (talk) 11:36, 26 March 2009 (UTC)

Two of those seem useful links, particularly in light of the fact that we already advise people to use the "laysummary" parameter in cite journal. They could help with selecting articles for those so I've made an addition to the guideline. Do you have any other specific suggestions Nbauman? Tim Vickers (talk) 15:37, 26 March 2009 (UTC)

TimVickers, My other suggestion is that we include language in the guideline to explain to people the repeated findings from the peer-reviewed literature about how the reliability of the popular press is distributed:

Some popular news publications, and individual journalists, are reliable, while others are not. Science and medical reporters in major publications, such as Lawrence Altman of the New York Times, are usually reliable. Reporters who don't specialize in science or medicine, and reporters who must write quickly on tight deadlines, are less reliable. No source is always reliable, and Wikipedia articles should be checked against the original published research or peer-reviewed review articles for accuracy.

That's my summary of the peer-reviewed research on the reliability of popular journalism.

It's similar to the literature on outcomes of surgery, which depends on the individual surgeon and the institution. Nbauman (talk) 18:10, 26 March 2009 (UTC)

Could you cite a couple of examples of this peer-reviewed research? I'm not sure exactly what is being talked about. For more on TimVickers's proposed wording, please see #Popular press accuracy below. Eubulides (talk) 18:47, 27 March 2009 (UTC)

(Undent) Nbauman, you've just gone circular on us. How do you tell if the newspaper story is accurate? By determining whether or not the journalist "writes an accurate story". That's not helpful. (Are you aware that this section already says that the coverage ranges "from excellent to irresponsible"? Perhaps you forgot that we already acknowledge the existence of excellent work -- while still recommending a solid scientific source for any scientific fact.) WhatamIdoing (talk) 17:56, 26 March 2009 (UTC)

I haven't gone circular. I was responding to your statement that it would be too difficult to "divine the reporter's level of (relevant) experience."

I was making a parenthetical comment that it's not necessary to divine the reporter's level of experience. If he's accurate, it doesn't matter how much experience he has. Some young journalists without much experience in journalism do very good work. (I disagree with Dentzer on that point.) Nbauman (talk) 18:23, 26 March 2009 (UTC)

Okay, let's take this from the top: The question we are addressing is "Is this popular press source sufficiently accurate that I can just use it as a source for a specific, concrete statement of scientific fact, without checking it against the proper scientific literature?"

• Dentzer says you might be safe trusting an experienced medical writer. You disagree with Dentzer (but complain that we misrepresent her).
• You say (without sources) that you might be safe trusting an accurate medical writer. Apparently you mean something more like "has a reputation for accuracy" instead of "wrote this specific story accurately" (which would be circular).
• I say this is all irrelevant, because the Wikipedia editor doesn't know whether the writer should be considered experienced or accurate.

I also note that you acknowledge above that the facts should be checked against the proper scientific literature -- "Wikipedia articles should be checked against the original published research or peer-reviewed review articles for accuracy" -- which is exactly what we have recommended all along.

So what's the problem, again? Do you want the editor to first figure out the reporter's reputation before double-checking the facts in the literature? What exactly is wrong with our existing recommendation that the editor skip that potentially time-consuming and error-prone step and just go straight to the literature, since (1) the editor will have to check the literature anyway and (2) we'll still rely on the literature if there's a conflict between the two sources, no matter what the reporter's reputation is? WhatamIdoing (talk) 19:30, 26 March 2009 (UTC)

On Wikipedia, people use the popular media (particularly the BBC for some reason) as the source for statements in articles. I think it's acceptable to use the BBC, New York Times, Wall Street Journal, or other sources, if it passes the checklist that Schwitzer and others have offered that I linked to above. You treat it like anything else under WP:RS.

Because this is Wikipedia, anybody who has the knowledge and access to the journals can check the news story against the journals. If the original news story checks out against the original source, you leave it in. If you think the news story misrepresents the original research, you can change it.

Why not go straight to the original journal article? For many reasons. First, most news stories are accessible to the public, while most peer-reviewed journals are not. Several million people read the New York Times. JAMA has a circulation of about 300,000. Second, it's easier for a Wikipedia editor to misunderstand a journal article than a news story by, say, Lawrence Altman in the New York Times. Third, news stories that meet Schwitzer's checklist will get comments by other experts in the field who may point out weaknesses in the journal article.

I don't know if you're an MD, but I interview a lot of academic MDs who publish in the NEJM, Lancet, JAMA, and similar journals -- and they themselves often get their information about clinical studies first from the New York Times. I was at a meeting recently trying to find a doctor who could comment on the JUPITER study in NEJM, but everybody there -- published academics -- kept telling me that they had only read the New York Times story.

You seem to believe in a romantic world where scholars get their information directly from the peer-reviewed journals, and eschew the lowly New York Times. But that's not the real world of medicine. Even leading doctors get a lot of their information about new studies from the New York Times. Nbauman (talk) 21:52, 26 March 2009 (UTC)

BTW, here's an example of a news story on Forbes that adds something clinically important that wasn't in the original BMJ article. The BMJ article reported that women who quit smoking before their 15th week reduce their risk of premature birth and small birth weight to normal. The Forbes story got a comment from another OB/GYN who pointed out some limits to the BMJ article's conclusions, such as the fact that the improved birth weight didn't translate into actual improvements in newborn health, and that it didn't tell whether pulmonary disease also returned to normal.

So a news story that meets Schwitzer's criteria can be more reliable than the original journal article, by better defining the limits to the study. Nbauman (talk) 17:17, 27 March 2009 (UTC)

• The above examples all seem to be highlighting primary studies, which WP:MEDRS properly warns against. People in many fields get their first reports about notable results from the newspaper, but there's a huge difference between being alerted to a new event, and getting an accurate, authoritative, and in-depth study. It's fine if my doctor hears about a new drug trial first from the New York Times, but I certainly wouldn't want my doctor prescribing it for me based on the NYT article!
• I think much of the current dispute comes from the difference between news articles and Wikipedia. Wikipedia is not a newspaper, and it rightly frowns upon following new stuff too closely (see WP:RECENTISM). In the vast majority of medical areas, Wikipedia articles should focus on what's known reliably and reproducibly, and should avoid reporting the very latest studies, regardless of whether it cites the studies directly or cites news articles about the studies. Since news articles focus heavily on the latest studies, most news articles should be avoided for that reason alone.

Eubulides (talk) 18:47, 27 March 2009 (UTC)

Eubulides, you're not a medical doctor, right? Nbauman (talk) 20:58, 27 March 2009 (UTC)

My personal qualifications are irrelevant to this discussion. Eubulides (talk) 21:57, 27 March 2009 (UTC)

I think you ought to talk to your doctor about where he gets his or her information from, and how much he or she uses the peer-reviewed medical journals to get his information.

You seem to think doctors read the major journals every week and that's where they get their information from. If you knew more about the real world, you'd know that those doctors are few and far between. There are studies of where doctors get their information, and the journals aren't that significant. Nbauman (talk) 23:11, 27 March 2009 (UTC)

And others read the synopsis provided by the likes of Uptodate which reference the original research. We should be trying to create a open source Uptodate and to do this we need to primarily reference the literature much like they do.--Doc James (talk · contribs · email) 23:26, 27 March 2009 (UTC)

There are several services like that. Another popular one is Journal Watch.

But a surprisingly large number of doctors get their medical news from the New York Times. (They look up the journal articles as necessary, and sometimes it isn't necessary.) Several reporters have a particular reputation for accuracy. Lawrence Altman is an MD, and he knows MDs read his work. Gina Kolata is a PhD. Is Uptodate any more reliable than Altman or Kolata? Nbauman (talk) 03:01, 28 March 2009 (UTC)

All too frequent

In my book, "reliable" does not include sources that are "all too frequently...wrong or misleading". Nbauman continues to argue that because newspapers can sometimes be accurate, we should regard them as reliable. In addition, he continues to avoid the "suitability" argument that even if they were reliable sources of medical information, they aren't suitable as sources for such in an encyclopaedia (see archive, for example, the "Bell curve" discussion). As DGG says below "No responsible encyclopedia could admit non medical sources to the discussion, except to discuss the ethical or political implication of the data. Lay sources testify to the lay understanding of science, not to science." I find this determination to "not get the point" after nearly two years of discussion on these talk pages to be quite extraordinary, verging on disruptive. Colin°^Talk 22:54, 25 March 2009 (UTC)

A lay source is a lay source, accurate or otherwise. Because it is open to accusations of inaccuracy, even if an sich reliable, it should not be used. It is as simple as that, and I agree with Colin that after so much time it should be clear that most voices here are in favour of a restrictive policy on lay sources. JFW | T@lk 23:44, 25 March 2009 (UTC)

Colin, the major medical journals, such as NEJM, JAMA, BMJ and Lancet, have been wrong or misleading in the past. For example, NEJM published articles promoting Vioxx, Lancet published an article claiming that autism is caused by vaccines, etc.

In your book, are the NEJM, JAMA, BMJ Lancet all too frequently wrong or misleading? Or are they acceptably not too frequently wrong or misleading? Nbauman (talk) 23:50, 25 March 2009 (UTC)

Oh for heaven's sake, do you ever see a newspaper or TV show publishing corrections? In both your examples, the journals made a point to publish large corrections. With the popular press you'd never know. Simply a ridiculous analogy. Also, when the medical journals mess up this is not due to inaccuracy but due to errors in judgement (Lancet) or misrepresentation by the original authors (NEJM). JFW | T@lk 23:56, 25 March 2009 (UTC)

I agree with Colin, the popular press needs to be used with care and never as the sole source for a fact or figure. The argument that because experienced and scrupulous journalists often get their stories right, so all news stories should be acceptable as a sole source in medical articles does not convince me. Tim Vickers (talk) 00:22, 26 March 2009 (UTC)

JFW, the point I'm making is that it's meaningless to say that newspapers are "too frequently" wrong or misleading. Any frequency of being wrong or misleading is "too frequently." Even prestigeous medical journals are wrong or misleading sometimes, and that's "too frequently" too.

As for corrections, yes, every reliable newspaper and TV show does publish corrections. (I've always made a distinction between reliable and unreliable news media.) I see them every day. The New York Times has a corrections page that runs every day [31], like the WSJ, and every major newspaper. The New York Times has a reader's editor, National Public Radio has an ombudsman.

If you're saying that the medical journals always make corrections while the newspapers don't, that's wrong. A pharmacist, Jennifer Hrachovec, was challenging the NEJM for at least 5 years on its Vioxx-VIGOR study, and Drazen refused to correct it. The WSJ, not the NEJM, published her complaints, and of course she turned out to be right. Nbauman (talk) 00:28, 26 March 2009 (UTC)

Tim, the only reason you're saying that is because the editors here are distorting my position (straw man). I have never said that the popular press should be the sole source for anything.

The argument that because experienced and scrupulous journalists often get their stories right, so all news stories should be acceptable as a sole source in medical articles does not convince me.

That doesn't convince me either. I never said it. I said that because experienced and scrupulous journalists usually get their stories right, we can use their news stories as one source for medical articles. But we shouldn't use irresponsible news stories. I'm simply following WP:RS.

Where did you get the idea that I am advocating this nonsense about using news stories as the sole source? Nbauman (talk) 00:38, 26 March 2009 (UTC)

Oh I see, reading over the meandering "discussions" above it looks like the topic now has little no relation to the text we are editing. It would be much more useful if you were to offer specific and concrete suggestions on how to improve the text of this policy. Tim Vickers (talk) 00:50, 26 March 2009 (UTC)

The problem is that the text we are editing misrepresents the sources that it cites in its support. I believe the text of the policy would be improved if it did not misrepresent the sources. I've given specific and concrete examples of how it misrepresents the sources, and how it can be improved. Unfortunately I'm outnumbered. That's Wikipedia for you. Nbauman (talk) 02:43, 26 March 2009 (UTC)

Any summary necessarily omits some of the summarized text, but that doesn't mean that it's misrepresentation. Your proposed changes would make the text a less-accurate summary of Dentzer, and would not be improvements. Improvements to the text are possible, but this discussion has beaten the topic into the ground enough that it's probably time to move on. Eubulides (talk) 03:13, 26 March 2009 (UTC)

Eubulides, Dentzer said in her introduction, "Often these messages are delivered effectively by [experienced] reporters who perform thoughtfully even in the face of breaking news and tight deadlines. But all too frequently, what is conveyed about health by many other journalists is wrong or misleading."

I propose to leave all that in, or to summarize it, leaving in the parts about good and bad journalists. You think that would be less accurate.

You think it's more accurate to delete the first part about reporters performing effectively, which goes against your personal opinion, and leave in the second part about journalists being wrong or misleading, which supports your personal opinion.

I think that's amusing. Nbauman (talk) 22:01, 26 March 2009 (UTC)

Please see # Popular press accuracy below. Eubulides (talk) 18:47, 27 March 2009 (UTC)

The worst

I just now reverted a change of wording "the worst news articles too often convey wrong or misleading information about health care". This change is definitely not supported by the cited source, Dentzer 2009 (PMID 19118299), who says that wrong or misleading information is conveyed "all too frequently", and who does not confine her criticism to the "worst" articles (she does writes that lack of context in medical news is a "worsening" problem, but that's quite a different matter).

Regardless of our personal opinions on this topic, we do have an obligation to summarize our sources as accurately as possible within the space constraints here, and conjuring up a "worst" here does not fulfill that obligation.

For more on this topic please see # Popular press accuracy below. Eubulides (talk) 18:47, 27 March 2009 (UTC)

NPOV

I am tagging NPOV because this essay conflicts with our policies on reliable sources and neutrality. It was recently used in an attempt at removing a reliable source about a social issue [32]. From the start, this essay was an attempt by pov pushers with a conflict of interest to use an essay to further their pov pushing and silence the other side. Among the supporters, Jfdwolff, MastCell, Cyclonenim, Casliber, Mattisse and probably others admitted a conflict of interest. I was lied to when I was assured that this essay would not be used for removing information about social issues. --Mihai cartoaje (talk) 09:02, 10 March 2009 (UTC)

Mihai, many medical issues are social issues as well, but that doesn't alter their fundamental nature. Xasodfuih (talk) 11:26, 10 March 2009 (UTC)

The statement "there is evidence that the neuroleptic drugs themselves cause loss of brain tissue" is not a social issue but a medical statement that must be sourced per WP:MEDRS and WP:V. This is an interview in a newspaper, of a scientist discussing research they haven't yet published, and in which they are very careful not to state "cause" but just a correlation with increased exposure to "drugs". It only becomes a social issue (why are people being given harmful drugs) if the fundamental assumption (the drugs are harmful) has been adequately proven.

I am really puzzled why this is regarded as a neutrality issue. Nancy Coover Andreasen is just the sort of medical expert MEDRS would like to cite once she publishes her results in a respectable journal. She's not on "the other side". Colin°^Talk 12:18, 10 March 2009 (UTC)

To backtrack on the article history a little, I was for including this interview until I, with the help of others, discovered that four other studies in this area have already been published (two in humans, two in primates), and I've added them to the article, initially directly, later through reviews, once I've found which reviews mentioned them—I still find this aspect of WP:MEDRS unwieldy at times. Andreasen's interview doesn't give enough details on her findings; given that and the delicate nature of these studies, plus the inclusion of already published studies in the wiki article, I don't see why waiting for her latest work to get published is such a big deal; currently the article read "In first episode psychosis typical antipsychotics like haloperidol were associated with significant reductions in gray matter volume, whereas atypical antipsychotics like olanzapine were not.[93] Studies in non-human primates found gray and white matter reductions for both typical and atypical antipsychotics.[94]". Xasodfuih (talk) 12:43, 10 March 2009 (UTC)

Mihai cartoaje, as you rightly say newspapers per overall WP:RS can be used as in this example to confirm a social aspect, namely that a person holds a view (likewise if the President states he thinks the economy will be helped by an action) although I do not know enough about that person & the interview in question as to whether of sufficient "worth" to warrent inclusion or not in the article. However that is not the same as confirming that the view is correct or supported by data that the (real-world) majority can agree upon (i.e. interest rate or exchanges rate might change but that does not lessen that Obama had the view, nor conversely override the fact that the economy really might not been helped). Newspapers (in this regard) confirm the truth of a person making a statement, not the truth of the statement made. It is for that proof of fact (rather than social commentary of opinions being held) that WP:MEDRS helps clarify sources that may be helpful. Likewise any number of newspapers can confirm that Prince Charles holds great belief in homeopathy, but that is not taken as proof or otherwise that homeopathy actually works (for which need peer-reviewed scientifically undertaken controlled ...etc etc), his views on some modern building architecture being a monstrous carbuncle is highly notable being from who said it and the public debate this started - indeed I would tend to agree with him on this - yet buildings are not abscesses :-)

WP:MEDRS has been accepted as a guideline, it is not an essay. But note the top description "a generally accepted standard that editors should attempt to follow, though it is best treated with common sense and the occasional exception" - "generally" does not mean everyone, and I would agree even WP:RS should only seek to confirm the state of knowledge per WP:NPOV and not some abolute truth as we're not WP:SPOV. I fail to see a conflict therefore between the various WP help pages: none are written in stone (WP:IAR), all need sensible interpretation and on occassion discussion between editors as to how best to work on some specific aspect within an article :-) David Ruben ^Talk 13:20, 10 March 2009 (UTC)

(having edit conflicted with Xasodfuih) Yes Xasodfuih, I think one might use (not same as should include, for that is an editorial decission in writing a well rounded section) the interview to confirm her view, but it would need be clear then that a view of one person rather than an absolute fact (a subsequently submitted paper may be accepted, but could also be rejected by a journal). David Ruben ^Talk 13:25, 10 March 2009 (UTC)

It's like a controversy during the Olympics. For example, were the fireworks real or computer generated? The Chinese organisers have their view, the media have their view and bloggers have their view. A neutral article would cover all these views. --Mihai cartoaje (talk) 14:23, 22 March 2009 (UTC)

I resent Mihai's continued suggestion that I might have a conflict of interest. JFW | T@lk 16:39, 10 March 2009 (UTC)

I don't see anything in the above discussion that suggests that WP:MEDRS is in conflict with Wikipedia policy, that it is particularly relevant to social issues, or that it has been used to "removing a reliable source about a social issue". Surely this thread belongs on Talk:Schizophrenia, not here. Eubulides (talk) 16:46, 10 March 2009 (UTC)

I'm concerned about Mihai's 'story' here. The discussion on the article's talk page is reasonable and making progress. MEDRS was cited as one example of a policy that says the amount of attention stuff gets in Wikipedia is based on the amount of attention they get in print. It's a bit odd to turn that widely accepted principle into "removing a reliable source about a social issue".

It's also a bit odd to turn "volume of gray matter" into a "social issue". Mihai, please don't confuse "the reason I believe this" with "what I believe". A reduction in gray matter volume (or its absence) is a concrete, measurable, scientific fact. The conclusions that someone draws from this fact (for example, that there should be government policy changes) are social issues.

As for the specific topic: actual causation may be hard to prove in humans. You'd have to prove that changes in schizophrenic patients were due to drugs and not to disease processes. (I understand that just the opposite has been proven in bipolar patients: manic episodes kill brain cells.) WhatamIdoing (talk) 17:11, 10 March 2009 (UTC)

As I commented on the talk page, the question of efficacy or safety is not in any sense asocial issue. No responsible encyclopedia could admit non medical sources to the discussion, except to discuss the ethical or political implication of the data. Lay sources testify to the lay understanding of science, not to science. DGG (talk) 02:28, 11 March 2009 (UTC)

It's almost becoming a bit hard to take Mihai seriously - but then I am a "POV-pusher with an admitted conflict of interest", just like everyone else who disagrees with him. MastCell ^Talk 05:03, 11 March 2009 (UTC)

Applied kinesiology: confused use of research sources

In this section of the article, there is a confusing use of sources. This situation has occurred because of the edits made by science-based editors on one side, and a board member of the ICAK on the other side. He has attempted to present this pseudoscience as a scientifically legitimate discipline. The best research has consistenly shown it to be hogwash. Please take a look and you'll be rewarded with a fascinating glimpse into the 10th most used chiropractic technique. -- Fyslee (talk) 05:30, 17 March 2009 (UTC)

Mediation

A user has requested mediation on this issue. Jmh649 is here to help resolve your dispute. The case page for this mediation is located here.

I have not been involved and have added some comments similar to those above.--Doc James (talk · contribs · email) 21:51, 24 March 2009 (UTC)

Popular press accuracy

(This follows up on the #Medicine and the popular press thread above, which was getting a bit long.)

Above, TimVickers suggested inserting this text somewhere:

Although some articles in the popular press are accurate, many others are inadequate, misleading, or even completely wrong.

I have a few ideas for making the proposal more concrete and better-supported by the cited source, Dentzer 2009 (PMID 19118299). I just now reread the above thread and Dentzer, and with all this in mind, I suggest that in WP:MEDRS #Popular press, we make the following change (deletions struck out, insertions italicized, and changing the preceding comma to a period):

and Although medical news articles often deliver public health messages effectively, they too often convey wrong or misleading information about health care, partly when reporters do not know or cannot convey the results of clinical studies, and partly when they fail to supply reasonable context.[PMID 19118299]

This captures Dentzer's thrust more accurately (albeit at more length), and mentions a positive thing she has to say about the news, which seems to be the main point of contention here. Eubulides (talk) 18:47, 27 March 2009 (UTC)

I'm concerned we are placing too much emphasis on trying to work out and reproduce the opinions and words of one author in one paper, which is an opinion piece, not a product of research. The article is not just concerned with accurate reporting (though she does give examples of misleading or incorrect reporting) but whether the underlying health message is being conveyed successfully. The wrong message may be conveyed even if the reporting of the "new" story is accurate, because it is incompletely described, or is not placed in context with other research or health concerns. The statement that "Often these [public health] messages are delivered effectively" is actually the least convincing opinion in the article and she cites no research to back it up. Perhaps nobody questioned that the word "effectively" means "to have an effect". Public health messages are largely ineffective, and I'm sure there's lots of research on the subject. Indeed, newspapers are guilty of effectively spreading anti-health messages such as those from the anti-vaccination lobby, or over-simplistic messages such as the antioxidant theory encouraging the consumption of red wine, or waste-of-time-and-money messages such as the high coverage of many CAM "therapies". We must also remember that "medical news articles" with the highest readership are found in glossy women's magazines, tabloids, and television news. The author of that paper, along with the folk on this talk page, probably read a better quality of news than average. Colin°^Talk 20:06, 27 March 2009 (UTC)

Colin, I agree that we are placing too much emphasis on the Dentzer piece which you correctly point out is an opnion piece, not supported by data or research. If you want peer-reviewed research supported by data, you can follow the links on Schwitzer's web site. Nbauman (talk) 20:52, 27 March 2009 (UTC)

Eubulides, I read an article in the BMJ about doctors who murder patients, which cited several examples.

Would you agree that doctors all too frequently murder patients? Nbauman (talk) 20:54, 27 March 2009 (UTC)

I don't know about Eubulides, but I would not agree with the statement "doctors all too frequently murder patients". What a ridiculous example. Nbauman, you are seriously in danger of self-highlighting the sort of misunderstanding and misreporting of statistics that we are criticising journalists for. It is a cliché that "one avoidable death is a death too many", but fortunately the world doesn't get paralysed while we wrap it in cotton wool. Colin°^Talk 21:34, 27 March 2009 (UTC)

A recent change gave reporter inexperience as the reason for poor medical stories, claiming the cited article excluded "seasoned reporters". It does nothing of the sort. That is a logical error like saying that "weekdays are dry" based on a source that says "often weekends are ruined by rain". Colin°^Talk 21:18, 27 March 2009 (UTC)

Nbauman - stop edit warring. You were bold. It got reverted. Now discuss. Your reinsertion of text which is neither supported by the cited article nor supported by consensus of WP editors is extremely bad form. I strongly encourage you to have the decently to self-revert and engage in consensus seeking here before making contentious changes. Colin°^Talk 21:37, 27 March 2009 (UTC) WP:RS does not apply outside of article space, and you yourself regard it merely as an "opnion piece, not supported by data or research". Colin°^Talk 21:39, 27 March 2009 (UTC)

Colin, did you read Dentzer's article before making that change? She says:

Often these messages are delivered effectively by seasoned [experienced] reporters who perform thoughtfully even in the face of breaking news and tight deadlines. But all too frequently, what is conveyed about health news by many other journalists is wrong or misleading.

We agreed that "seasoned" means "experienced."

So Dentzer said that "other journalists" are wrong or misleading, not experienced journalists. Nbauman (talk) 21:44, 27 March 2009 (UTC)

Four statements: [1] There exist seasoned reporters who have delivered messages effectively. [2] There exist other journalists who have conveyed wrong or misleading information. [3] Wrong and misleading information is conveyed by inexperienced journalists. [4] Wrong and misleading information is not conveyed by experienced journalists. 1+2 implies neither 3 nor 4. Please go read up on logic before suggesting I didn't even read the article. Colin°^Talk 23:24, 27 March 2009 (UTC)

Colin, please go back and re-read the word "other." Nbauman (talk) 03:03, 28 March 2009 (UTC)

"Other" means "those journalists not in the first set". The first set is defined as "seasoned reporters who have delivered messages effectively". The first set is not defined as "all seasoned reporters". For that, the author would have had to write "Seasoned reporters deliver health messages effectively." You are seeing (because you want to) an implication that strictly isn't there. However, I suspect Dentzer didn't worry about whether her statement would be logically analysed or mined for implications, because as I pointed out earlier, her use of the word "effectively" is careless. Colin°^Talk 10:44, 28 March 2009 (UTC)

I strongly disagree with the recent change "news articles too often can convey wrong or misleading information about health care particularly when they are written by by inexperienced reporters". Not only is it ungrammatical, it focuses on what is a secondary topic of Dentzer's article (lack of training) and misses her main point (lack of context in news articles). Also, it removes a "too often" that is an important point in Dentzer, and should be a point here. I agree with both Colin and Nbauman that this thread is focusing too much on Dentzer and too little on what the guideline should say. I frankly am dismayed by Nbauman's debate tactics on this talk page; they are not conducive to collaboration. Let's work out a change on the talk page first, and not edit-war the project page. Eubulides (talk) 21:57, 27 March 2009 (UTC)

Perhaps more importantly, this is not a mainspace article on Role of popular press in health care. This page is advice from Wikipedia editors to other Wikipedia editors. We don't need sources to "prove" that our advice is to use high-quality secondary scientific sources for statements of scientific fact, and we don't need to split hairs over the ideally and comprehensively neutral way to represent one person's published opinions.

There is no consensus for change. Nbauman remains offended that we're not showing enough respect to his favorite colleagues, and will probably always remain offended by our supposed scientism. None of the other involved editors want to lower our standard to accommodate his POV. I think we need to just stop talking at this point. WhatamIdoing (talk) 22:56, 27 March 2009 (UTC)

Agree completely. Scientific journals are the best source. This type of question can be decided by a vote if needed.--Doc James (talk · contribs · email) 22:59, 27 March 2009 (UTC)

Vitamin E

• An interesting example of a recent popular press distortion comes in the form of NYTimes recent coverage of vitamin E. For example, they painted a misleading picture of the aftermath of the SELECT trial [33] (which our selenium article reflects fixed). Much better commentary appears in Gann's accompanying editorial on the SELECT trial [34] or in the article's discussion itself [35]. The most striking difference is that while earlier trials were focused on selenium-deficient people, the SELECT men were high in selenium. Selenium is toxic at higher doses. Big surprise. Similarly, NYTimes recently highlighted the "increased mortality" from vitamin E in an article titled "Extra Vitamin E: No Benefit, Maybe Harm". As I pointed out in my my comment, they hardly emphasize that the increased mortality occurred only at 400+ IU (27 times the RDA) and excluded the fact that at a meta-analysis found a non-statistically significant reduction in deaths at 100 IU. They also excluded the fact that meta-analysis was hesitant to generalize this to the non-elderly population. What they also don't mention is that, according to the latest NHANES, 90% of Americans don't get the RDA of 15 IU. From that perspective, basic vitamin E supplementation looks prudent, and indeed the ATBC trial found benefits for those with higher serum levels. Despite the issues with NYTimes coverage, it was added to the orthomolecular medicine article by experienced medicine editor MastCell [36]. Although I pointed out the problems in E's coverage at that article's talk page, no one seems interested in providing a neutral overview of poor E. II | (t - c) 23:35, 27 March 2009 (UTC)
  • I'd rather not discuss specific content issues here, especially when they're as historically intractable as vitamin supplementation. From the perspective of WP:MEDRS, you're actually dismissing the New York Times article far too hastily. In fact, it does note that the excess mortality was seen with high doses. An entire subsection of the Times article is entitled "Too Much of a Good Thing", and states:

    Recent studies have even suggested that at the high doses many people consume, vitamin E could be hazardous. In November 2004, the American Heart Association warned that while the small amounts of vitamin E found in multivitamins and foods were not harmful, taking 400 International Units a day or more could increase the risk of death. ([37])

    That's followed by a fairly detailed breakdown of recent studies on the subject. So I think the Times piece is actually an example of responsible mainstream-media reporting, which is one reason I cited it. Another reason is that the citation occurred in orthomolecular medicine, which specifically involves very large doses of vitamins and where the potential harms of such megadoses are relevant.

    I can't fault the Times for "excluding" a non-significant finding - I'd be more inclined to praise them for doing so. And the Times piece reaches the same conclusion as Gann's (excellent) editorial: Vitamin E pills aren't a magic bullet. MastCell ^Talk 00:40, 28 March 2009 (UTC)

• Dentzer's point is that "when journalists ignore complexities or fail to provide context, the public health messages they convey are inevitably inadequate or distorted". That's exactly what happened there, when the average vitamin E intake is around 7 mg/day [38]; the ATBC study (not covered) found benefits for moderate supplementation, and the meta-analysis found that "low-dose" vitamin E (still several times above the RDA) reduced deaths; the probability that it was due to chance was 20% -- not stat. significant, but not worth discarding either. Based on the literature, the article could also have reasonably been titled: "Extra vitamin E: Good in moderation". It was quite easy to come away from that article thinking that vitamin E is scary and dangerous. Further, the results of the meta-analysis were challenged [39]; anyway, to get back on topic, I actually think Dentzer is being somewhat selectively cited, but the section is fine overall. One thing to keep in mind is that papers can sometimes be extremely technical; in those cases a lay summary is necessary. As far as OMM, I'm disappointed you didn't notice the major problems in the vitamin E section (not the least the statement that "initial hopes for OMM were based on observational studies" -- implying that mainstream, rather than OMM fringe, was responsible for the RCTs), and hopefully you'll comment the next time I bring it up. By the way, looking at Gann again, I'm not so impressed by the way he attributes the results of earlier studies to chance when they were clearly different populations. II | (t - c) 07:10, 28 March 2009 (UTC)

• Agree with MastCell that this thread on Vitamin E doesn't belong here; presumably it belongs in Talk:Orthomolecular medicine. Eubulides (talk) 16:38, 28 March 2009 (UTC)

Scientific evidence for guideline

WhatamIdoing, you've got it exactly backwards. I have always been a strong critic of medical journalism, and I agree with most of Schwitzer's work because Schwitzer is based on evidence. Much medical reporting is terrible. I don't know if "most" of it is terrible, but I'll go with the evidence.

I'm flattered when I see articles like Dentzer's, because I'm one of those experienced journalists she talks about. I write for doctors, my work is edited and reviewed by doctors, and if I didn't get the story right in every respect, my work would be tossed out pretty fast.

My difference with you and the other editors here is that I believe that evaluations and criticisms of medical journalism should be based on the same kind of scientific evidence that we use in medicine and elsewhere in science. That's what I'm offended at -- your abuse of science and contempt for scientific method. You're not using high-quality secondary scientific sources at all.

You're making wild claims about journalism based not on scientific evidence, like Schwitzer's work, but on your own personal prejudices. I won't speculate on the psychological roots of those prejudices, but you can't support your broad overstatements with facts.

You cherry-pick articles like Dentzer's to find excerpts that you think support your position, and quote them selectively and misleadingly, even though you misunderstand the whole essay.

I've seen something similar, where alternative medicine advocates will pick out excerpts of medical articles to support their argument that all doctors and all drug companies are harming people.

Your basic fallacy is to condemn all journalism in the popular press, when the scientific data displays a much more interesting pattern: some of the popular press is accurate and some is not. So you're really missing the point.

It's paricularly ironic that you claim to be supporting "high-quality secondary scientific sources", when you're doing exactly the opposite. Except for Schwitzer, your entire guideline is based on personal opinion and personal prejudice and ignores those sources. Nbauman (talk) 23:45, 27 March 2009 (UTC)

• I dunno, are any of the other Wikipedia guidelines or policies rigorously supported by scientific evidence? Although the scientific method is quite useful for resolving scientific disputes, one can't use it for everything.
• Hmm, come to think of it, shouldn't any effort to document (and perhaps even scientifically support) medical journalism be put into mainspace? I just now created the stub Medical journalism to do that. I get the feeling that if even 10% of the effort expended on this thread had been put into Medical journalism we might have a useful article there by now. (And no, that's not a scientific estimate....)

Eubulides (talk) 16:38, 28 March 2009 (UTC)

A secondary source dilema

Here is a dilema that I have noticed particularly with books. Whilst it is for obvious reasons preferable to use secondary sources wherever possible, is this always the case? One scenario I saw was when reading a high quality secondary source in a book which is widely used especially by medical students where it stated that benzodiazepines are GABA reuptake inhibitors or increase GABA levels for their therapeutic effects which is clearly false as they work at benzodiazepine receptors enhancing the effect of GABA at the GABAa receptor via benzodiazepine receptors. This scenario didn't happen on wikipedia but using it as an example of inaccurate info in books, but something similar happened in the past few days where on the surface seemingly high quality books on psychiatry and psychopharmacology were cited but on closer inspection the authors were employed by literally over 20 pharmaceutical companies and they had cherry picked 2 or 3 primary sources and then distorted the results of the primary sources and reinterpreted them and falsely represented the primary sources without any evidence but their opinion. So what would happen in that scenario? To be clearer though the sections of the books were not doing an extensive review of the literature but were basically touching a little on every aspect of psychiatry and psychopharmacology. So that brings up another question what happens if the secondary source is not doing a meta-analsysis or extensive review of the literature but are simply touching on one aspect of a condition or drug or syndrome etc? Also is a peer reviewed meta-analysis or review superior to a non-peer reviewed book? Is a peer reviewed secondary source published by people with ties to a drug company or some other conflict of interest equal to an independent peer reviewed secondary source? I guess my point and dilema is this. The use of secondary sources are viewed higher on wikipedia because they are seen as being an independent outside opinion but what if the author(s) are not independent and/or they can be shown to be distorting data? My main concern is secondary sources which are not peer reviewed as this area seems to be most open to abuse by those with a conflict of interest. Currently the recommendations is books which are not necessarily peer reviewed and open to abuse, opinion, synthesis, reinterpreting results, distorting facts are declared as "excellent" (secondary) sources. I disagree and think that the problems of books needs to be addressed. There are some good medical books which are high quality don't get me wrong but you see the dilema that I see and problem for misinformation getting inserted into articles. I think peer reviewed secondary sources trump non peer reviewed secondary sources and that many (but not all) books in general are not excellent sources of reliable citeable data. Infact I think primary sources are quite often superior to many books depending on the subject matter being quoted for reasons that I have just explained.--Literaturegeek | T@1k? 23:32, 1 April 2009 (UTC)

I have striked out some of my text as on closer inspection I see that you differentiate between non-reliable medical books and reliable trusted publications, but still have some concerns mentioned above.--Literaturegeek | T@1k? 15:54, 2 April 2009 (UTC)

That's kind of a long question, and it sounds like this is really more of a question about a particular case, but in general the answer (I think) is that peer-reviewed sources are generally (although not always) preferred. Conflict of interest is of course an important issue, but in practice almost everybody has a conflict of interest of one form or another, and a review should not be disqualified simply because a reviewer has a conflict of interest. Eubulides (talk) 16:45, 2 April 2009 (UTC)

[Edit conflict with Colin, posting at same time] Sorry for length of questions. This was the book that I was talking about. Would this book be considered superior to an FDA review or Committee on Safety of Medicines review? This book is cited in an article and I have no intention of deleting it so I am not in a conflict with anyone per se. It was just when reading the book and comparing it to their sources I noted how they had distorted or even falsified the conclusions of the sources they were citing. Also they appeared to be giving best practice guidelines based on taking one or two primary sources out of context and also synthesising without any evidence in conflict. I guess I noticed a potential occasional serious problem with using books over more reliable peer reviewed secondary sources. Then of course the above example I mentioned where a trusted book source got the mechanism of benzodiazepines completely wrong is another example. I am not against using books and more often than not they do make good sources to quote and infact I used one today myself. I guess I am asking does a secondary peer reviewed meta-analysis or review source trump this example of a book? Or does FDA review or CSM review or department of health guidelines trump it?--Literaturegeek | T@1k? 17:56, 2 April 2009 (UTC)

Don't overplay the "independent" aspect of secondary sources. The aspects of secondary sources we like are that they are written by experts and they have undergone some form of expert review (peer or editorial). When wikipedians choose primary sources we end up with the selection & analysis of those primary sources done by nobodies, with no reputations to uphold, and (often) no serious review. So I find your concern about bias in the secondary-source authors to be a red-herring -- there is bias in the primary sources and there most certainly is often bias among wikipedians (which is undeclared, whereas conflicts-of-interest are declared in some sources). A medical book published by a serious publisher will have undergone some form of review. If you have found mistakes in a student textbook, perhaps you should contact the publishers. Colin°^Talk 17:51, 2 April 2009 (UTC)

Colin, I think perhaps you misinterpreted me or perhaps it is my fault in how I worded the issue. I was not in anyway shape or form saying that primary sources are superior to secondary sources. I was raising an occasional isolated problem of what happens basically if a secondary source says something which is nonsense. For example should we change the mechanism of action of benzos to say they are reuptake inhibitors because a good quality book made a mistake? Or in such a scenario would a peer reviewed secondary source trump it so we can add accurate data? What happens if the book is demonstratly misrepresenting their sources etc. I was talking about isolated examples. These examples didn't happen in real life on wikipedia but I just noticed a potential from spotting several inaccuracies and thought that I would raise it. I have noticed this is less problematic in the peer reviewed literature especially secondary sources which have been published in medical journals. The inaccuracies I have spotted in sources (particularly books which often briefly skim a topic) are not necessarily POV issues but just a matter of an inaccurate factoids. Another one I have noticed is flunitrazepam is short acting when it is long acting.--Literaturegeek | T@1k? 18:03, 2 April 2009 (UTC)

• One of the reasons that User:Paul gene strongly opposed the emphasis on reviews was that, in his words (if I recall correctly), they are often "written by pharmaceutical hacks". These days the tide has begun to turn and, if you look hard enough, you can find unbiased information, as evidenced by the recent Cochrane review on St. John's wort (PMID 18843608) and the recent review on the NEJM on publication bias in antidepressants [40]. Sources without conflicts of interest should be preferred in my mind, or at the least included, and we should probably have a section on it in this guideline. II | (t - c) 18:41, 2 April 2009 (UTC)

This guideline covers the whole of medicine. Drugs and the pro/anti-pharma controversy is really a small part. Folk get worked up about conflicts of interest and bias when much of what we should be writing about on WP, and pushing towards FA, is bread-and-butter non-controversial medicine. We must always bear this in mind when giving and suggesting changes to guidance on sources. Much of research is done and reviewed by honest folk earnestly trying to work out why we get ill and how to fix it. It is just a little wearying to keep hearing that secondary sources are full of bias, written by hacks in the pay of 20 pharmaceutical companies, and that if you'll only let me choose, weigh and interpret the primary sources instead, then I'd do a better job... Colin°^Talk 19:10, 2 April 2009 (UTC)

Speaking of misrepresenting primary sources, you appear again to be misrepresented me. I never said nor implied that I should resynthesise an authors opinion. I was asking what do we do in that scenario? Do we accept the demonstrately intentional or unintentional inaccurate factoids or opinions of a book which often briefly touched on one aspect of medicine,,, or does a peer reviewed secondary source such as a meta-anaylis or review or FDA, department of health etc etc trump it? I came here just to ask for guidance on what to do. Initially I misread the book guidelines on this project and striked out my text above but still wanted some guidance or discussion on what to do in these isolated scenarios I have mentioned. Instead I appear to be getting responses trivialising and mocking me as an idiot by distorting what I was saying and asking.--Literaturegeek | T@1k? 19:28, 2 April 2009 (UTC)

Secondary sources are usually more reliable than primary sources. The fact that a handful are not just means that you should, in these cases, use several secondary sources to give a broad view of the range of opinions on the matter. What is not appropriate is for us to use primary sources to cast doubt on the conclusions of secondary sources - that is original research. Tim Vickers (talk) 19:15, 2 April 2009 (UTC)

Thank you Tim this is what I was asking for, some guidance. Could you also give me your opinion on whether official national reviews by say the FDA or Department of Health, meta-analyis or review in a journal would be superior to a book? What happens if a book says for example SSRIs work by stimulating the release of serotonin when the truth is they block its reuptake or something like that? Do we delete the book ref and replace it with a more accurate secondary source? Or do we leave the mistake of the book in even though we know that it is false? I guess I would like this project to state whether reviews or meta-analysises (secondary sources) in medical journals are superior to a (secondary source) book if content is in dispute.--Literaturegeek | T@1k? 19:28, 2 April 2009 (UTC)

That's really a question for the RS noticeboard, this page is for discussing this policy, not specific sources or specific disputes. However, the advice given already at Wikipedia:Reliable_sources_(medicine-related_articles)#Choosing_sources seems to apply well to this instance. Tim Vickers (talk) 19:33, 2 April 2009 (UTC)

Okie dokie Tim, problems with specific sources, try and sort out on article talk page or else if that fails go to RS noticeboard is what you are saying I believe. So no need to alter guidance. You have answered my question, thank you.--Literaturegeek | T@1k? 19:47, 2 April 2009 (UTC)

Any source that is rather general and broad (such as a student textbook) is likely to be inferior to a more specific book or paper. It may well have attempted to condense a complex subject with a simple explanation that is in fact wrong. This may be the case for your benzo textbook. A specialist book aimed at medical professionals would be a better source, as would a relevant secondary article such as those you mention. Where you have several sources in disagreement then it may be relevant to note the disagreement among professionals within the article (as Tim suggests) or you could post a query on the article talk page or WP:MED talk page and ask for the opinions of others. You may find someone has access to a better definitive source. Colin°^Talk 19:51, 2 April 2009 (UTC)

Ok thank you Colin for your reply and good advice, makes sense.--Literaturegeek | T@1k? 20:00, 2 April 2009 (UTC)

Just to clarify it wasn't a "benzo book", but a clinical psychopharmacology text book (see above) and also another medical text book. Anyway like I said the inaccurate data wasn't cited on wiki so I was raising a potential issue I had noted with isolated books saying inaccurate things eg benzos are reuptake inhibitors or distorting sources. Anyway like Tim said discuss on article talk page to reach consensus, use better secondary sources or as a last resort go to RS noticeboard in such a scenario.--Literaturegeek | T@1k? 21:52, 2 April 2009 (UTC)

This has been bugging me and I would like to end this by saying that I apologise for the confusion generated by myself initially misreading (due to skim reading when half asleep) the policy regarding books and I now know why Colin reacted the way he did. He is only trying to defend wikipedia. Ironically I was trying to do the same in my posting and had nothng to gain and the policy actually reads the way I thought it should read but I misread it initially. Anyway I got my questions answered in the end so all is good. :) I would like to just end this conversation here by saying I support Colin's efforts in defending this project and wikipedia and I am glad we now understand each other's misinterpretation of each other. No need for anyone to reply.--Literaturegeek | T@1k? 02:15, 6 April 2009 (UTC)

Review status

I hope I can raise a query here without offending! But regarding the determination of 'review ' status for a paper, does listing as a review by the NIH generally differentiate it. To fit the criteria, Ideal sources for these articles include general or systematic reviews in reputable medical journals, Given A secondary source in medicine summarizes one or more primary or secondary sources and that not all reviews particularily narrative type necessarily label themselves as reviews, but the NIH might list them as such? Leaving aside of course arguments about various journals. Peerev (talk) 22:37, 20 May 2009 (UTC)

PubMed, in my experience, is not 100% reliable in labeling reviews. Occasionally it labels an article as a review when it is not; more often, it fails to label an article a review when it is indeed a review. (If you can't tell whether an article is a review after reading the article, then you probably shouldn't be citing that article; reliable sources are not supposed to be so confusing....) Eubulides (talk) 23:29, 20 May 2009 (UTC)

What the author is actually doing matters much more than how some person in the back office happened to code the database.

I have also found errors in the presentation of articles, including straightforward case reports labeled as "reviews" and things that I thought were good reviews being unlabeled (well, and basically all older reviews aren't tagged as reviews, but you generally shouldn't be using those anyway).

If you've got a question about a specific, then please feel free to share it with us. WhatamIdoing (talk) 05:38, 21 May 2009 (UTC)

Thanks multiple specifics, but in general I have also noticed that some papers say for instance an hypothesis, will on reading be largely a review of the relevant literature with a much smaller expansion of the hypothesis. Such papers are often listed as a review, but may not be tagged such, perhaps they are older as you suggest! I would assume it is safe to cite the review part generally but not necessarily the hypothesis part (unless under an Hypotheses heading). Other papers of course will also contain large reviews within them and I suspect that is why they are coded as review? When a pubmed search is done for a subject with 'and review' included, these can be the only 'reviews' found, regardless of date, hence the need for clarification. Peerev (talk) 21:25, 21 May 2009 (UTC)

I believe (don't ask for a source) that PubMed classifies "reviews" based on a handful of numerical criteria: reviews have few authors and many citations, and so forth. In other words, PubMed uses a computed algorithm to determine which articles are "reviews". I've written a couple which are not characterized as such by PubMed - probably I didn't use enough references :) Anyhow, I could be wrong since I can't find a source to support my statements, but I would suggest that editors may be as good or better position to determine which articles are "reviews" than PubMed. MastCell ^Talk 21:31, 21 May 2009 (UTC)

If there is a difficult case, there is always the idea of looking at the journal and seeing where in the issue the article is listed and how the journal describes it. Tim Vickers (talk) 21:53, 21 May 2009 (UTC)

Crazy talk! :P MastCell ^Talk 22:06, 21 May 2009 (UTC)

Thanks again, so it seems it is a matter of the editor reading such sources and deciding which parts can be considered 'review' and which parts should not be so cited ! Peerev (talk) 21:11, 22 May 2009 (UTC)

I think that what Tim is saying is that many journals explicitly publish reviews under a heading like "Review Article". Some journals (e.g. the Expert Opinion series, or Current Opinions) publish only reviews, and say so explicitly. It's not so much a matter of making something up as looking closely at the source yourself. MastCell ^Talk 21:40, 22 May 2009 (UTC)

Science by press release

Is it worth mentioning the {dangerous|idiotic|POV-pushing|your favorite adjective here} practice of citing popular media reports on studies that have not (yet) been published? This practice isn't very common, but it seems to be most common in things that turn out to be lucrative for the scientist but false later, i.e., Andrew Wakefield's long-delayed publications in the MMR hoax.

While I think relying on a news article instead of the proper paper is undesirable (and common), "science by unsubstantiated press release" is dramatically worse. WhatamIdoing (talk) 17:59, 23 May 2009 (UTC)

Isn't this already covered? WP:MEDRS #Popular press says:

"Newspapers and magazines may also publish articles about scientific results before those results have been published in a peer-reviewed journal, or reproduced by other experimenters. Such articles may be based uncritically on a press release, which can be a biased source."

If this doesn't state the point sufficiently clearly, what should be added? Eubulides (talk) 18:05, 23 May 2009 (UTC)

Apparently, what we need to add is more attention on my part; I didn't see it in my brief skim-over, and it did seem odd that it wasn't there. Thanks. WhatamIdoing (talk) 19:01, 23 May 2009 (UTC)

Second thought: one possible way to highlight this issue is to expand "Editors are encouraged to seek out the scholarly research behind the news story" to include "and to not use any source for which the scholarly research has not yet been published". Also, a link to our article, Science by press conference, might not be out of place. WhatamIdoing (talk) 19:04, 23 May 2009 (UTC)

Too far. We should certainly be very wary, especially with controversial subjects, non-mainstream findings, product promotion, etc, but not all medical information is achieved via the scientific method and not all is published in journals. As this is a guideline, we should give advice but not absolute rules. BTW: perhaps Science by press conference should pass WP:V before we link to it? Colin°^Talk 21:14, 23 May 2009 (UTC)

The statement is still just 'encouragement': "Editors are encouraged to seek out...and to not use..." WhatamIdoing (talk) 21:50, 23 May 2009 (UTC)

As a possible citation, I recently saw a journal article which evaluated the content, quality, and accuracy of the material contained in various university-sponsored press releases. Needless to say, it was abysmal. Maybe someone else has this article at hand; if not, I'll track it down. I do tend to agree with Colin; we should discourage the citation of such press releases, but not absolutely forbid it since there are always exceptions and instances where it may be reasonable. MastCell ^Talk 23:04, 23 May 2009 (UTC)

This must be Woloshin et al. 2009 (PMID 19414840). I took a modest stab at mentioning academic medical center press releases, and citing that source. Eubulides (talk) 05:42, 24 May 2009 (UTC)

Question about a journal

If anyone wants to comment or has some insight, I've a question about the Journal of Mental Health over at RSN. WLU (t) (c) Wikipedia's rules:^simple/_complex 14:33, 26 May 2009 (UTC)

I usually pester User:DGG about questions like that. WhatamIdoing (talk) 00:21, 27 May 2009 (UTC)

Publisher is T&F, a good but second level academic publisher. The editor in chief and about half the editors come from a single department, often a danger sign, but it's the Institute of Psychiatry at Kings College London, which is rather well known. . However, it's not in Journal citation Reports, although indexed by Web of Science, it is not in JCR, and has published an article trying to explain why its important anyway. This is a serious indication that there are problems. DGG (talk) 04:50, 29 May 2009 (UTC)

Talk:Aspartame controversy#Air Force alert needs verifiable & reliable source

I have posted this at the RS/N, but want to make sure the medical matters contained are addressed by readers here. I have concerns about the way a source is being used. The way it is being used is consistent with the way it has been used in internet conspiracy theories regarding the supposed myriad dangers of aspartame, but I believe it grossly misrepresents the nature of the original source. I would appreciate more eyes on the matter. This is a medical and scientific matter, so to some degree WP:MEDRS would apply to any interpretation of the scientific matters discussed in the source. The original source seems to bear evidence of being influenced by the fringe conspiracy theories, while fortunately noting the mainstream POV from governmental and medical sources. -- Brangifer (talk) 04:27, 28 May 2009 (UTC)

Medicinal clay

The very new Medicinal clay article is in need of a thorough checking. Lots of undoumented claims are made. -- Brangifer (talk) 04:47, 1 June 2009 (UTC)

Expansion: CDC's website?

Under WP:MEDRS#Websites, could we mention some of the major government websites, like the CDC? We have ongoing problems (e.g., this) with people insisting that only peer-reviewed journals are acceptable. Perhaps a mention of major (international?) charities would also be appropriate: they often have decent statistics and can be accurate sources for statements about whether a treatment is common or only used for refractory cases, etc. WhatamIdoing (talk) 15:49, 4 June 2009 (UTC)

Clearly the CDC website is a reliable source, so I was bold and added that to WP:MEDRS#Websites. Charities are a bit iffier, as some have axes to grind. Perhaps wording about them could be proposed here? Eubulides (talk) 16:56, 4 June 2009 (UTC)

I was hoping someone else would have a brilliant idea about how to address that issue. I've also been thinking about the advantages of "permitting" these websites (as if this really trumped WP:RS), but still "recommending" medical textbooks or high-quality reviews. The hardy folks that deal with fringe topics are not going to be happy to have anything that suggests that www.QuacksRUs.org is a good source for medical information. Perhaps a restatement of not using websites to debunk the literature would be in order. WhatamIdoing (talk) 21:57, 4 June 2009 (UTC)

Statements and information from reputable major medical bodies (CDC, NIH, WHO, Institute of Medicine, etc) should obviously be considered valuable encyclopedic sources. One would hope that common sense would be sufficient to enable appreciation of that point, but apparently it needs to be codified.

Sadly, there is no language that will completely prevent some of our fellow editors from trying to sneak www.associationforsnakeoilstudies.com or rife.org into the encyclopedia under the same aegis as the CDC. I would suggest wording along the lines of "information from reputable major medical and scientific organizations may be a valuable encyclopedic source." If we reach the point where the words "reputable major scientific organization" have lost their meaning, then it's time to abandon this ship anyway. MastCell ^Talk 22:55, 4 June 2009 (UTC)

You mean like Australasian_Journal_of_Bone_and_Joint_Medicine? J/K ;) Unomi (talk) 02:59, 5 June 2009 (UTC)

I don't see the relevance to this discussion (assuming your comment had some intent beyond scoring cheap points). That journal was not a "reputable major scientific organization". It was a journal, and an obscure, low-profile one at that. Hopefully our existing guidelines would have led editors to properly weight material from that journal. On the other hand, sometimes shit happens. Wikipedia policies can't prevent real-world misdeeds. If someone had cited articles by Jayson Blair from the Times before his fabrications were revealed, it wouldn't make them a bad editor, just someone who was taken in like everyone else. MastCell ^Talk 03:34, 5 June 2009 (UTC)

I suspect that "J/K ;)" means "Just Kidding ".

Perhaps we should list the NIH, WHO, IOM (any others?) as examples. Would anyone object? WhatamIdoing (talk) 00:34, 12 June 2009 (UTC)

That sounds reasonable; I've proposed that in #Proposed rewording for Websites section below. Eubulides (talk) 07:25, 12 June 2009 (UTC)

Proposed rewording for Websites section

Looking at it another way, the word "Websites" in the section header WP:MEDRS #Websites is problematic. Whether info is on a website is reasonably independent of how reliable it is. After all, JAMA is a website, and it's generally reliable. Even medical books are now commonly accessed via websites. So I propose the following change, along the lines suggested above (and with some wording shamelessly stolen from MastCell's comments):

• After Books, insert a new section Medical and scientific organizations with the following contents:

"Statements and information from reputable major medical and scientific bodies may be valuable encyclopedic sources. These bodies include the U.S. National Academies (including the Institute of Medicine and the National Academy of Sciences), the British National Health Service, the U.S. National Institutes of Health (including the Centers for Disease Control and Prevention), and the World Health Organization. The reliability of these sources range from formal scientific reports, which can be the equal of the best reviews published in medical journals, through public guides and service announcements, which have the advantage of being freely readable but are generally less authoritative than the underlying medical literature."

• Change the title of Websites to be Other sources, and make the following changes to its contents:

"Internet websites Press releases, blogs, newsletters, advocacy and self-help publications, and other sources contain a wide range of biomedical information ranging from factual to fraudulent, with a high percentage being of low quality. Peer-reviewed medical websites information resources such as WebMD and UpToDate, along with websites of government health organizations such as the CDC, can be useful guides about the relevant medical literature and how much weight to give different sources; however, as much as possible Wikipedia articles should cite the literature directly."

Eubulides (talk) 07:25, 12 June 2009 (UTC)

I would specifically add both the professional [41] edition of the Merck Manual,DGG (talk) 04:17, 16 June 2009 (UTC)

Seems a very good change.--Garrondo (talk) 07:15, 16 June 2009 (UTC)

Looks good to me. Tim Vickers (talk) 16:06, 16 June 2009 (UTC)

I'm not sure we need to add this: ideally, if those sources have useful info, they are already included as sources for the article (which I see has been suggested in Eubulides' wording). (And the Merck Manual has long been woefully inaccurate on Tourette syndrome). SandyGeorgia (Talk) 16:05, 16 June 2009 (UTC)

Are the 3 comments above specifically about The Merck Manuals, or about the other proposed changes? Anyway, surely any mention of The Merck Manuals belongs in Books, not in the proposed Medical and scientific organizations or Other sources sections. Also, Books should be changed anyway, to make it clear that it talks about both online and printed medical texts. How about if I make a separate proposal for Books, which mentions The Merck Manuals and also mentions that many books are online? Eubulides (talk) 17:56, 16 June 2009 (UTC)

(Change proposed in #Proposed rewording for Books and Formatting citations below.) Eubulides (talk) 21:02, 16 June 2009 (UTC)

I really dislike the idea of endorsing the Merck Manual ... does anyone have the most recent version to see if it's still inaccurate on TS? And we shouldn't be encouraging adding anything to external links; ideally the sources are cited (can Eubulides' wording go further to state that?). SandyGeorgia (Talk) 18:00, 16 June 2009 (UTC)

The Merck Manual's entry on Tourette's is online and free. I haven't read that entry carefully, but it hasn't been revised since 2005 and anyway it is pretty terse, so it wouldn't be a good source for the Tourette syndrome article regardless of any factual errors. I just now read the Merck Manual's autism entry, which is pretty good: it has a few minor errors and is a bit dated despite its 2009 revision, but overall it would be suitable as a reference for the Autism article (if Autism weren't already citing better sources). Overall The Merck Manuals seem to belong in the Books section, which already comments on such such books' terseness and incompleteness, and their problems with being up to date. Eubulides (talk) 21:02, 16 June 2009 (UTC)

Just to clarify: this is about what is considered acceptable as a proper <ref>erence, not as an ==External link==. We do occasionally run into people that have a problem with major websites, including, e.g., a long conversation about the symptoms of swine flu, because they aren't peer-reviewed journals. WhatamIdoing (talk) 20:07, 16 June 2009 (UTC)

oopsie :) SandyGeorgia (Talk) 20:09, 16 June 2009 (UTC)

Proposed rewording for Books and for Formatting citations

How about this change to Books, to mention online books and The Merck Manuals?

"Many medical books are published online, and some, such as The Merck Manuals, are updated regularly. Ensure the book is up-to-date, unless a historical perspective is required."

While we're at it, Formatting citations should probably give an example of a book. Something like this, perhaps?

"A book citation should mention which chapter or entry of the book is being sourced. It is good practice to mention the date, authors, and (if available) DOI for the individual book component, along with the ISBN or OCLC number for the overall book. For example:

• Dawson M, Mottron L, Gernsbacher MA (2008). "Learning in autism" (PDF). In Byrne JH (ed.-in-chief), Roediger HL III (vol. ed.) (ed.). Learning and Memory: A Comprehensive Reference. Vol. 2. Academic Press. pp. 759–72. doi:10.1016/B978-012370509-9.00152-2. ISBN 978-0-12-370504-4. Retrieved 2008-07-26.{{cite book}}: CS1 maint: multiple names: authors list (link)"

Eubulides (talk) 21:02, 16 June 2009 (UTC)

No further comment, so I installed the proposed changes. Eubulides (talk) 19:09, 22 June 2009 (UTC)

Casting it in stone

When considering studies for inclusion in clinical articles, I have usually kept a rule of thumb that phase II trials need to be absolutely earth-shattering to deserve inclusion. Even phase III trials need to be included with caution, because their findings don't necessarily mean that a procedure or substance will ever be used in a widespread fashion.

I was wondering if there was any scope for anchoring this rule of thumb into this guideline. After all, this applies to all medicine articles. JFW | T@lk 18:35, 21 June 2009 (UTC)

It seems you not only keep that rule of thumb, but have already generalized it to all other medical editors in the encyclopedia, as your recent edit summary shows ("there is longstanding consensus not to include phase II trials unless their results are absolutely earth-shattering"). You probably won't be surprised that I don't share this "consensus". Considering that consensus means general agreement within a group, and considering how small our group is, even a single disagreement rules out the word consensus. It's interesting that you provided no argument. I think most medical editors would disagree with your opinion. Clinical trials are not common for a fair amount of things. I imagine that many interventions don't even get to Phase III clinical trials. Should we make a rule where we keep out all negative evidence? Suggestive epidemiological and animal evidence can be, and often is, encyclopedic -- so why would actual controlled human data be non-encyclopedic? II | (t - c) 19:24, 21 June 2009 (UTC)

Could you please take my post at face value rather than trawling my edit history? Also, please let other editors speak for themselves rather than putting words in anyone's mouth.

A phase II trial is, by its very definition, a small trial in diseased subjects and the first to evaluate for effectiveness. Very large numbers of treatments don't even make it out of the laboratory, and even then a large proportion is not developed further if there are no encouraging results at phase II-III level. It is therefore impossible to say that a treatment is "promising" even if phase II trials are positive, because they may still die before being marketed, let alone implemented widely.

I was referring specifically to clinical articles. I think it would be utterly dangerous if we started listing all agents being trialled currently in Alzheimer's disease, for instance, and create the completely erroneous impression that dementia is a treatable disease. Wikipedia should not be in the business of promoting treatments that have no certain future. I cannot understand how you could actually disagree with this premise. JFW | T@lk 04:17, 22 June 2009 (UTC)

Generally speaking, I think that such a rule could be created and would be widely supported, although I think it would require more than a single sentence to properly explain the limits. For example, are we talking about this standard as applied to a disease article, or in an article about an investigational new drug? Is this information under ==Treatment== or under ==Research directions==? Are the rules different for rare diseases? For diseases that currently have no adequate treatments?

Additionally, are we sure that this the right guideline for such a recommendation?

As for the timing: I think that the vast majority of improvements to Wikipedia's guidelines stem from someone seeing an actual problem. I'd be unhappy to discover that someone sat around thinking up hypothetical problems to "solve" in the guidelines. That path leads to needless WP:CREEP and to guidelines that do not address editors' real needs. WhatamIdoing (talk) 04:45, 22 June 2009 (UTC)]]

Wikipedia already documents the substances undergoing clinical trials for Alzheimer's disease. The article is called Alzheimer's disease clinical research. In a highly-developed article like Alzheimer's disease, yeah, I can see why you wouldn't mention Phase II clinical trials. In less highly-developed articles like pancreatic cancer, the argument is less convincing. My rule of thumb is that most information can be presented on Wikipedia, although much of it should be presented in subarticles. II | (t - c) 05:38, 22 June 2009 (UTC)

Wouldn't it be better to try to educate readers what the ramifications/limitations of the different phases are? Then clearly state that it is a Phase II clinical trial. I think that our goal should be to enlighten readers. There are no space limitations, and it would be a mistake to write an article with a 'message' in mind. Unomi (talk) 07:19, 22 June 2009 (UTC)

This is an issue of WP:WEIGHT rather than an in-or-out global rule. As WhatamIdoing says, rare diseases with no current effective treatment might well warrant the mention of any promising, solid research. I don't see what the pancreatic cancer article being less "highly-developed" has to do with it. WEIGHT is more of a content than a sourcing issue. I'm sure there is more we can say about balancing content wrt medical aspects. WP:MEDWEIGHT anyone? Colin°^Talk 08:18, 22 June 2009 (UTC)

I think ImperfectlyInformed's reaction illustrates the need for guidance on the inclusion of phase I/II trials. This is not a current major issue but it happens constantly in numerous articles, sometimes driven by media reports and sometimes driven by editors who have read something in a journal and find it highly fascinating and surely worth including. More is not always better; this applies to huge topics such as Alzheimer's but I can't see why pancreatic cancer (which receives a fair bit of research attention) doesn't fall under such guidance. Again, just because a very small trial has found some marginally interesting results for "some scary illness" doesn't automatically mean we are including it, because in the larger subsequent studies such effects may well regress to the mean or shown to be marred by toxicity. I can say with confidence that many of the drugs in Alzheimer's disease clinical research will never hit the shelves. What was then the purpose of giving them so much airtime? JFW | T@lk 14:15, 22 June 2009 (UTC)

Ideally, I'd like to see this handled flexibly. In general, I do think we give too much weight to the latest abstract from ASCO, but that is a failing of the popular press as well, so it's difficult to break free of it. On the one hand, whenever a new study piques interest and gets written up in the popular press, someone will reliably add a lengthy segment about that study to Wikipedia. We should probably discourage this, because of WP:NOT#NEWS, recentism, etc.

On the other hand, the evidence base for specific medical practices varies widely. In some areas (pancreatic cancer being an example), interventions are often adopted wholesale on the basis of very limited data, and Phase III trials may or may not ever be done (certainly the impetus for a drug company to fund such a trial evaporates rather quickly once their drug becomes de facto standard-of-care).

Basically, our articles should be in line with respected, expert medical thought and opinion, since our goal is to produce a serious and respectable reference work. If such opinion is based on Phase II trials, then it would be OK to cite them (hence I can't really get behind a cast-in-stone prohibition). On the other hand, we shouldn't be emphasizing the latest, greatest preliminary result at the expense of a balanced treatment of a topic. It might be OK to have a section in an article like pancreatic cancer where we discuss areas of active investigation - preclinical work and early Phase I/II results - with the clarity that this is the cutting edge and may or may not pan out. That would probably do readers a service - by informing them of current avenues of research - without misleading them into thinking that a single-institution, uncontrolled Phase II abstract definitively represents the wave of the future. MastCell ^Talk 18:31, 22 June 2009 (UTC)

• It's definitely not a black-and-white issue. For a well-researched area like pancreatic cancer, though, it should not be hard to find reliable sources that review experimental and research therapies, and Wikipedia articles should rely on these secondary sources instead of citing primary sources directly. There are exceptions for groundbreaking primary sources that have not had time to be reviewed yet, but these exceptions should be quite rare (and shouldn't last longer than a few months).
• This particular case does not seem to be an exception to the rule. Dhillon et al. 2008 (PMID 18628464) is a reasonable primary source on one relatively narrow topic, but it has been reviewed multiple times (notably by Fryer et al. 2009, PMID 19442075), and Wikipedia should be citing these reviews, not the primary source.
• For rarer diseases such as (say) wandering spleen, primary sources may be the best we can do, and in that case they're OK.
• The Assess evidence quality section already comments on evidence-quality rubrics, and perhaps advice about phase I/II trials could be folded into there. However, that section is already problematic, as it simply gives rubrics for assessing evidence quality, without making it sufficiently clear what the rubrics are for. When I first read that section, I misunderstood it to mean that Wikipedia editors should be using phrases like "Class IIIb evidence" in articles to summarize primary studies that articles cite, which (I hope!) was not the intent.
• I'm dubious about encouraging jargon like "Phase II clinical trial" in articles on diseases or therapies. Medical articles should avoid jargon like that. The main text of articles should concentrate on what's known, not on how we know it.

Eubulides (talk) 19:09, 22 June 2009 (UTC)

Have we just confused early clinical trials with whether the reports about them are primary or secondary sources? Many Phase II trials are described in secondary sources. WhatamIdoing (talk) 20:33, 22 June 2009 (UTC)

Surely it's OK to cite a reliable review such as Fryer et al. 2009 (PMID 19442075) that discusses phase II trials, presumably along with other evidence. However, the original example given (Dhillon et al. 2008, PMID 18628464) is a primary source about a single phase II trial, and as I understand it the underlying dispute here is over whether it's OK to cite such sources. Eubulides (talk) 21:20, 22 June 2009 (UTC)

Even early and inconclusive trials can have major implications for the companies. If even a phase I/II trial is reported outside the scientific journals, we may need to cover it. The terms need to be used in individual articles. They're the exact descriptors, and people know them--they're now widely used in newspapers, all the patients lobbies know them, and so does the financial world. What would help is to add a short descriptor each time they are first used in a article (for safety) (for possible effectiveness) etc. That will be enough to orient the person who first comes across these terms in an article. Our articles should be in line with serious respected opinion, yes, but they also extend into areas where there is not yet defined medical opinion. If the article is written properly the status will be clear. (My basic rule is if it is well enough known that people will come to look for it in a general encyclopedia like this, it belongs here, which is approximately equivalent to the General notability guideline for things that the general or specialized newsmedia cover--such as new drugs. I would include the publications of the major patient lobbies as responsible media. DGG (talk) 04:45, 27 June 2009 (UTC)

I dunno, I just now searched Google News for "phase II clinical trial" and found that almost all news sources that used that phrase were press releases, stock tip sheets, and other similarly unreliable sources. Wikipedia medical articles are not the right place to pass along this kind of material. Sure, there are exceptions, truly notable phase II results that cry out to be mentioned even though no reliable reviews have been published yet; but these should be rare, and they should be removed in short order as reviews of these oh-so-notable results come in. Eubulides (talk) 05:19, 27 June 2009 (UTC)

I could imagine good reasons to mention a Phase II trial in an article about a biopharma company, and occasionally in an article about a drug, but I'm unconvinced that it should be mentioned in an article about a disease -- especially under "Treatment". WhatamIdoing (talk) 17:15, 27 June 2009 (UTC)

Absolutely. My main point was with regards to discussing treatments in clinical articles. I think we need to think really carefully about whether we go along with the mindless hype of newspapers to write hysterically about every in vitro study (in the UK, the Daily Mail is especially good at this) or try to educate by providing balanced high-quality context-driven content. JFW | T@lk 07:24, 28 June 2009 (UTC)

So perhaps a new section might be in order. What do you think of something like this:

Summarize relevant, currently accepted knowledge
Research is underway in nearly all areas of medicine. Although the results of a recent study might interest many people, from patients to investors, Wikipedia is not a newspaper and includes only encyclopedic information, in due proportion to its long-term importance. Medicine-related articles should present the current state of medical knowledge. Editors should normally not include speculative proposals and should avoid presenting early-stage research in ways that suggest it is widely accepted.
For example, the results of an early-stage clinical trial are unlikely to be appropriate for inclusion in the "Treatment" section of an article about a disease, because a possible future treatment has no bearing on current treatment practices. However, the results might, in some cases, be appropriate for inclusion in an article dedicated to the drug or treatment in question, or to the researchers or businesses involved in it. Such information might also be appropriate for a well-documented section on "Research directions" in an article about a disease. If including this information in this situation, it is frequently helpful to the reader to clearly identify the level of research (e.g., "first-in-human safety testing," "Phase II clinical trial") to prevent misunderstandings.

On the other hand: perhaps MEDRS is the wrong place for this. Perhaps instead it needs to be at WP:MEDMOS. WhatamIdoing (talk) 22:59, 28 June 2009 (UTC)

Peer-review as MEDRS

I'd like some clarification on an issue that I'm not entirely sure is addressed by the main page (or perhaps I just didn't understand it). Let's say Dr. Watson writes an article in a highly-reliable source like The Lancet, and this article is cited in a Wikipedia article. His peer, Dr. Seuss, later writes a critical response to the original article, which also gets published in The Lancet. The response is, presumably, considered editorial in nature being only one person's opinion. Is it appropriate and/or desirable to include that response within the body of an article, assuming it's relevant? Something along the lines of "In an article in The Lancet, Dr. Watson wrote that all bunnies are fluffy, but this claim was countered by Dr. Seuss who dismissed Dr. Watson's claims." This is basically the point of peer review is it not?

Just as a point of note, I'm not a doctor, researcher, or anything else of that nature, so I don't have a good understanding of the amount/quality of review that would go into any given article or response in order for it to actually be published in a reliable source like The Lancet. Us computer nerds tend to just blog everything. :) Thanks in advance for any clarification on this question. --RobinHood70 (talk) 21:03, 12 July 2009 (UTC)

Letters to the editor are not exactly peer review but an extension of it. What is meant by peer review is that an article is reviewed by independent reviewers before it is published. Reviewers may advise the editor not to carry an article if it has flaws or doesn't meet the aims of the journal. Often, changes are suggested to make the article better once it is accepted.

Letters are usually not peer-reviewed, but the editor of the journal employs some form of discretion as to what gets published. It is very unusual for a letter writer to completely uproot the premise of a publication, and if that is the case then the editor will not have any alternatives but suggest to the original author to retract his/her article. If that doesn't happen then obviously the letter writer might have a dissenting opinion but is not necessarily more right or wrong that the original author. Truth told, if you are talking about reviews, the review writers are usually chosen on merit, while letter-writers are self-appointed.

For the purposes of WP:MEDRS, I think very good secondary sources are needed to give any weight whatsoever to letters to the editor. If a journal like The Lancet carries a review, they will have done their homework in deciding whether the article is worthy of that status. JFW | T@lk 22:41, 12 July 2009 (UTC)

Thanks for the clarification, JFW. The issue came up recently in an article I was editing (though I wasn't involved), and I since I wasn't sure, I thought I'd clarify it for future reference. Thanks again! --RobinHood70 (talk) 23:26, 12 July 2009 (UTC)

Such letters work a lot like the letter to the editor pages at a regular newspaper: even if the editors think the writer is a complete idiot, they don't want to be accused of "silencing dissent" or "ignoring evidence" (etc.). So they publish letters that they disagree with, on grounds of keeping the dialogue open or keeping the readers interested.

Even when I agree with the letter (e.g., it clarifies a point that was lost in a larger article) or it is written by people I respect, I don't think that they're high-quality sources for our purposes.

Additionally, the letters most interesting to POV-pushers on Wikipedia are frequently written by people that are notorious for their fringe-y views. A well-respected journal will expect its readers to recognize, and discount, the views of the "Rush Limbaughs" in their field -- but they'll still publish them for exactly the same reasons that a left-wing magazine might print a rant from a right-wing celebrity. WhatamIdoing (talk) 01:18, 13 July 2009 (UTC)

Medically unexplained symptoms

Question about when MEDRS is not MEDRS. I have interest in conditions with medically unexplained symptoms, i edit articles on Morgellons, chronic Lyme, Fibromyalgia, Chronic fatigue syndrome. The reviews on CFS and FM say, the cause is not known, but i use the phrase "medically unexplained symptoms" and other editors do not like. I think some editors think MUS is the same as "functional somatic syndrome" so they do not like that, but i do not think it is, it means medically unexplained.

Editors say I need to proove MUS is used in the fibromyalgia literature, i am ok with it, and they say no article on fibromyalgia uses MUS, i am not ok with that! A week now, i am giving reference and more reference like at my sandbox User:RetroS1mone/MUSreferences and the articles Talk:Fibromyalgia#Primary_sources, Talk:Fibromyalgia#MUPS and user talks, also with quotes, many from the reviews and articles I am giving have "medically unexplained" in abstract or title. Reviews on FM and articles use MUS or MUPS, reviews in the MUPS literature say FM and CFS, textbooks also. Reviews in the psych literature say MUPS and FSS and CFS and FM. There is more support for CFS and FM "medically unexplained" then most of the things in these articles. The editors are saying, these MEDRS are not good enough bc the articles they find in there searches have only a few citations. but the articles i am giving, some have in hundreds of citations, i do not know why they do not look at those.

The question, is there a total citation number before a MEDRS can be used, and how many MEDRS reviews does it take, for supporting a statement in article?? Thx RetroS1mone talk 11:53, 20 July 2009 (UTC)

Sorry, there is no magic number. One needs to look at all the literature on a topic, assess its reliability, and attempt to reflect it in a balanced way. (I haven't looked at your particular case.) Eubulides (talk) 18:46, 20 July 2009 (UTC)

The sources you list are, on their faces, good sources. The list includes books, reviews, a meta-analysis, and other high-quality publications. Of course, it's possible to misuse a good source, so you'll want to be careful about whether or not the source, taken altogether, really supports a specific given statement. WhatamIdoing (talk) 20:07, 20 July 2009 (UTC)

It looks like this was resolved at Talk:Fibromyalgia#MUPS today. WhatamIdoing (talk) 20:14, 20 July 2009 (UTC)

DrGreene.com

See Linksearch: [42]. Should these links be removed as not up to WP:MEDRS? Seems all very worthy in intention, but it's ultimately a celebrity doctor site. Gordonofcartoon (talk) 20:49, 21 July 2009 (UTC)

I searched that site for "autism" and wasn't impressed. Compare this in Autism therapies #Prescription medication:

"A 1998 study of the hormone secretin reported improved symptoms and generated tremendous interest, but several controlled studies since have found no benefit."

to this in Dr Greene's page on autism and secretin:

"While much research remains to be done, secretin may be a promising treatment and major step through a new doorway into the treatment of chemical problems underlying autism."

The problem here may be simply that Dr Greene's page was written in 1998 and last updated in 2000, and is simply obsolete. However, I found similar obsolete-or-sensationalistic material on mercury and autism. I'd be leery of citing that website for any but the most routine medical information. Eubulides (talk) 21:43, 21 July 2009 (UTC)

I wasn't impressed with Preventing Down syndrome? which just cites one study about folic acid to conclude "It may well be that this simple measure can also dramatically reduce the risk of Down syndrome"; hardly an authoritative basis for a secondary source... Gordonofcartoon (talk) 22:04, 21 July 2009 (UTC)

These should be removed; they don't seem like appropriate encyclopedic links or sources. MastCell ^Talk 23:53, 21 July 2009 (UTC)

The House of Hype. Shoot on sight please. JFW | T@lk 00:02, 22 July 2009 (UTC)

Piece of sh*t, in the article on malaria it states "In some areas of the world, mosquitoes that carry malaria have developed resistance to insecticides , while the parasites have developed resistance to antibiotics." since Plasmodium spp are eukaryotes they show little susceptibility to antibiotics, and the few that are active have never been used in treatments since they are not useful antimalarial drugs. I might expect simple mistakes like that in an undergraduate essay, but not a reliable secondary source. Remove links. Tim Vickers (talk) 00:11, 22 July 2009 (UTC)

Assess evidence quality

Here is the first cut at a replacement for the Assess evidence quality section, for discussion. It's too long and wordy, but I thought I'd get it out the door now and we can trim it later. The basic idea is to incorporate some of the discussion mentioned above, and to trim away some of the unnecessary detail that's in the section now.

---

Several systems exist for assessing the quality of available evidence on medical subjects, and these should be kept in mind while assessing whether a particular viewpoint is a majority or minority one, and in deciding what constitutes evidence-based medicine. The best evidence comes from meta-analyses of randomised controlled trials (RCTs), and from systematic reviews of bodies of literature of overall good quality and consistency addressing the specific recommendation. Narrative reviews can help establish the context of evidence quality. Roughly in descending order of quality, lower-quality evidence in medical research comes from individual RCTs, other controlled studies, quasi-experimental studies, and non-experimental studies such as comparative, correlation, and case control studies. Although expert committee reports or opinions, along with clinical experience of respected authorities, are weaker evidence than the scientific studies themselves, they often provide helpful overviews of evidence quality. Case reports, whether in the popular press or a peer-reviewed medical journal, are a form of anecdote and generally fall below the minimum requirements of reliable medical sources.

If an notable important scientific result is so new that no reliable reviews have been published on it, it may be helpful to cite the primary source that reported the result. Although popular-press news articles and press releases often tout the latest phase II clinical trial, such trials are rarely notable important enough to mention in an encyclopedia. Any such results should be described as being from a single study, for example:

"A 2009 U.S. study found the average age of formal autism spectrum diagnosis was 5.7 years." (citing PMID 19318992)

After enough time has passed for a review to be published in the area, the review should be cited in preference to the primary study. If no review is published in a reasonable amount of time, the primary source should be removed as not reporting an notable important result. When in doubt, omit mention of the primary study, as per WP:RECENTISM.

Speculative proposals and early-stage research should not be cited in ways that suggest wide acceptance. For example, the results of an early-stage clinical trial are unlikely to be appropriate for inclusion in the Treatment section of an article about a disease, because a possible future treatment has little bearing on current treatment practice. However, the results might, in some cases, be appropriate for inclusion in an article dedicated to the treatment in question, or to the researchers or businesses involved in it. Such information, particularly if citing a secondary source, might also be appropriate for a well-documented section on "Research directions" in an article about a disease. To prevent misunderstandings, the text should clearly identify the level of research cited, e.g., "first-in-human safety testing".

---

End of draft replacement. Eubulides (talk) 16:57, 2 July 2009 (UTC)

I've made a minor change to remove the word "notable". The use of this 'term of the art' on Wikipedia in contexts that relate to due weight issues instead of article inclusion criteria frequently leads to confusion. (I'm not committed to my word choice, just to the removal of the word "notable".) WhatamIdoing (talk) 17:38, 2 July 2009 (UTC)

No further comment, so I replaced the text in question with the above draft. I copied the previous version to #Previous version of "Address evidence quality" below, as it looks like it could well be useful in an article somewhere. Eubulides (talk) 06:42, 23 July 2009 (UTC)

Previous version of "Address evidence quality"

Several systems exist for assessing the quality of available evidence on medical subjects, and these should be kept in mind while assessing whether a particular viewpoint is a majority or minority one, and in deciding what constitutes evidence-based medicine.

Two of the most commonly used schemes are the US Agency for Healthcare Research and Quality one:

Class	Requirements
Ia	Evidence from meta-analysis of randomised controlled trials (RCTs)
Ib	Evidence from at least one RCT
IIa	Evidence from at least one well-designed controlled study without randomisation
IIb	Evidence from at least one other type of well-designed quasi-experimental study
III	Evidence from well-designed non-experimental descriptive studies, such as comparative studies, correlation studies, and case control studies
IV	Evidence from expert committee reports or opinions and/or clinical experience of respected authorities

And the slightly simpler NHS one:

Grade	Evidence	Description
A	Ia, Ib	Requires at least one RCT as part of the body of literature of overall good quality and consistency addressing the specific recommendation
B	IIa, IIb, III	Requires availability of well-conducted clinical studies but no RCTs on the topic of recommendation
C	IV	Requires evidence from expert committee reports or opinions and/or clinical experience of respected authorities. Indicates absence of directly applicable studies of good quality.

Case reports, whether in the popular press or a peer-reviewed medical journal, are a form of anecdote and generally fall below the minimum requirements of either scheme.

Sources cited within primary research

Primary research often, if not usually, includes introduction and discussion sections which cite other sources. Can these be then considered as "secondary" sources? - Tekaphor (TALK) 14:26, 20 July 2009 (UTC)

Only according to ImperfectlyInformed (talk · contribs). According to everyone else, those sources are often chosen selectively to remain relevant to the line of research in the article, and therefore less useful. JFW | T@lk 16:31, 20 July 2009 (UTC)

WP:MEDRS #Biomedical journals says "Research papers are primary sources; although they normally contain previous-work sections that are secondary sources, these sections are typically less reliable than reviews." It's better to use a real review, as a primary source often has an axe to grind and its previous-work section is typically not reviewed nearly as carefully as a real review would be. Eubulides (talk) 18:45, 20 July 2009 (UTC)

Might as well also add that review articles can be primary sources. I recall Eubulides telling me that a review I found was a primary source because the author of the review cited two papers, including one which he had coauthored (this was a discussion on water fluoridation and environmental impact). II | (t - c) 19:36, 10 August 2009 (UTC)

On a related note, Tekaphor, I'd like to thank you for edits like this one. A pilot study like this is not a sufficient source for an entire section. WhatamIdoing (talk) 20:20, 20 July 2009 (UTC)

Thanks all. - Tekaphor (TALK) 13:33, 21 July 2009 (UTC)

Medical Hypotheses again

Hi everyone, i cannot believe, but Medical Hypotheses journal is discussed again as MEDRS Talk:Chronic_fatigue_syndrome#Herpes_and_CFS, a single purpose account editor thinks it has a MEDRS review about Herpes causes CFS. I tried explain them, it is not MEDRS, but editor has very good arguments for MEDRS by calling me uninformed and a conservative, ha ha if they knew!! I am sorry for bugging every body about it, pls can we say the journal is, is not MEDRS. Thx RetroS1mone talk 13:51, 30 July 2009 (UTC)

In fairness to Jagra, his contribution history shows that he is not an SPA. His history over the last year has been almost exclusively CFS-related, but then again, there's been a lot of controversy on that page over the last year, resulting in one banned user and several formal and informal disputes.

In terms of the current debate over Medical Hypotheses, while I'm currently in dispute resolution with Retro on other issues, I consider myself to be neutral on this particular topic in that I am not involved in the addition or reversion of the material, nor do I have an opinion on its validity. Retro has pointed out some relevant topics in the regular RS archives, and I think they speak fairly clearly to the issue (though I haven't examined them or the surrounding discussion in detail), but I think it would be reasonable to address the issue here as well, as editors here are likely to have better knowledge of medical journals and I do not see any previous discussion of this journal in the MEDRS archives, where I think many people would expect to find it. --RobinHood70 (talk) 20:21, 30 July 2009 (UTC)

OK, I gave it a whirl with an edit that also mentioned Journal of Medical Biography as a different kind of specialization in medical journals. Further comments welcome. Eubulides (talk) 21:24, 30 July 2009 (UTC)

You can only cite Medical Hypotheses as a source supporting the fact that somebody has a belief. It has no use as a source of factual statements about reality and should be avoided in medical articles. Tim Vickers (talk) 19:20, 10 August 2009 (UTC)

Yeah, that was my understanding as well from the discussion linked to in RetroS1mone's post (and elsewhere as mentioned there). Good to hear confirmation of it, though. Thanks, Tim! --RobinHood70 (talk) 19:41, 10 August 2009 (UTC)

Thank you!! RetroS1mone talk 12:57, 11 August 2009 (UTC)

Proposal

We need some wording to disambiguate between different sources regarding alternative medicine. I propose something like the following:

Complementary and Alternative Medicine (CAM)

Generally, much less reliable research has been conducted about the efficacy, safety, and legitimacy of complementary and alternative medicine (CAM) than has been done for mainstream medicine. For this reason, reliable sources may be much more difficult to find and unreliable sources are often readily available. Whenever writing about a claim about CAM practices, it is vital that third-party, independent sources be used. Sources written and reviewed by the advocates of CAM are excellent for describing their own opinions, but extreme care should be taken when using such sources lest the more controversial aspects of their opinions be taken at face-value or, worse, asserted as fact.

ScienceApologist (talk) 19:14, 10 August 2009 (UTC)

This is a tricky area and obviously the simplest thing, which is what WP:MEDRS currently does, is to say nothing, in effect holding CAM up to the same standards as everybody else. I see some problems with the proposal above:

• CAM is a broad area, and ranges from therapies like acupuncture (for which there is scientific evidence of effectiveness, for some conditions) to therapies like radionics (which is clearly pseudoscience and quackery). Any advice we give should not throw out whatever babies are in the bathwater.
• Many areas of medical controversy in medicine are not commonly characterized as CAM, and yet they still would benefit from advice along the lines of the above. These include sublingual immunotherapy, pluripotent stem-cell therapy, androgen replacement therapy, vaccination, and many other areas.

It would be better, I think, to rephrase the advice above so that it does not mention CAM, but does acknowledge and make suggestions for the problem of axe-grinding in poorly-supported areas. Eubulides (talk) 23:07, 10 August 2009 (UTC)

Perhaps a section discussing the interactions of WP:MEDRS and WP:FRINGE? - 2/0 (cont.) 03:03, 11 August 2009 (UTC)

I am deeply concerned that there is a tendency among some to disparage secondary references which cover mainstream thought while cherry-picking positive references and using them as sources for either outright assertions of fact which are disputed or as a means to inappropriately "balance" the general consensus of medical experts. It is no secret, for example, that certain therapies, supplements, and treatments do not provide the efficacy, safety, nor legitimacy that they claim. For example, today I went to the pharmacy and found that fully one-third of the products that were marketed as sleep aids were either homeopathic, naturopathic without claims being evaluated by a regulatory authority (sold as a "supplement"), or were downright lunacy (including one product which sold for $29.95 that consisted of a magnet attached to a strip of gauze that you were supposed to wrap around your wrist). I know that Quackwatch has articles on all of these products and names many of them by name. However, there are editors in good-standing here at Wikipedia who would make the claim that Quackwatch is not a "good source" per MEDRS. The argument they give is for particular attribution to Stephen Barrett. This type of argumentation seems entrenched and we need to address it Head-On (pardon the pun). WP:PARITY does not seem to sink in, and often people seem to think that PARITY means something it does not. To wit, parity of sources is supposed to allow for leniency in sourcing mainstream opinion about fringe ideas. Synergistically with WEIGHT, the idea is to be able to write NPOV with proper sourcing. Too often, certain editors at Wikipedia think PARITY means something like BALANCE. "Oh, the homeopath says this and this medical doctor writing for an on-line magazine that's not peer-reviewed says this. Obviously these are equivalent sources and should be treated as he-said/she-said controversy." No, PARITY just gives editors permission to find sources outside the normal channels when the claims themselves are made outside the normal channels (e.g. not necessarily referenced by PubMed). In such cases, we rely more on the reputation of the sources and evaluate their potential biases before using them as authorities or asserting facts reported by such sources. This point does not seem to be elucidated on this page.

To be clear, most of the products I saw at the pharmacy do not have Wikipedia articles and are not directly mentioned anywhere in the encyclopedia. However, we need to think about what happens as these articles do get written: and eventually, unless Wikipedia goes belly-up, they will get written. Having a clause or two in this guideline that lets editors know that mainstream sources which directly address controversial or problematic medical-related subjects can be trusted would be very helpful. The basic issue right now is that we go into great depth here discussing topics about which there exist a moderate to extraordinary amount of research. We need to accommodate those subjects which are more marginal -- dare I say -- FRINGE.

ScienceApologist (talk) 04:01, 11 August 2009 (UTC)

I also am concerned with the all-too-common insertion of unreliable material or sources. For example, I just now removed a bunch of unsourced material claiming that water fluoridation is bad for you; see Talk:Water fluoridation #As a public policy decision. I regularly have to remove similar fringe material from other articles I help maintain, ranging from Autism to Zostavax. In my experience CAM, while a big part of the problem, is not the whole problem; and furthermore, part of CAM is scientifically OK. With this in mind, it'd be better to adjust the phraseology of any proposal so that it does not focus on CAM, but instead focuses on antiscience, pseudoscience, fraud, axe-grinding, etc. Phrasing it in this more-general way will (1) apply more generally to problems in medical articles, and (2) avoid pointless arguments about whether this-or-that treatment is CAM. (Most chiropractors don't agree that chiropractic is CAM, for example; and aren't they the experts on what chiropractic is? That sort of argument.) Eubulides (talk) 05:17, 11 August 2009 (UTC)

I see your point, Eubulides. In the cases you and I are citing there seems to be two common denominators: 1) a lack of reliable sourcing and 2) an agenda of declaring medical "facts" that are not supported by reliable sourcing. The issue is that occasionally such advocacy is notable enough to be written about in this encyclopedia. How do we handle this? ScienceApologist (talk) 12:54, 11 August 2009 (UTC)

Here is a new proposal:

Poorly sourced claims

While many medical subjects are covered by a plethora of reliable sources, there is a sizable number of notable though marginal medical claims where little to no reliable research has been conducted about the efficacy, safety, or legitimacy of the statements made by proponents. In such cases, reliable sources may be much more difficult to find and unreliable sources can often be more readily available. Whenever writing about medical claims not supported by mainstream research, it is vital that third-party, independent sources be used. Sources written and reviewed by the advocates of such marginal ideas can be used to describe notable personal opinions, but extreme care should be taken when using such sources lest the more controversial aspects of their opinions be taken at face-value or, worse, asserted as fact.

@ScienceApologist, re above proposal for poorly sourced claims: that sounds fine to me as long as we're not construing it to say that Quackwatch can be cited as fact rather than attributed with an inline citation, or that it supersedes other MEDRS's. Quackwatch is certainly fine to use, per WP:PARITY, but you go too far with your argument here when you hold Quackwatch as a superior source to the Institute of Medicine, viz:

"But there are times that sources may contradict the mainstream science that Barrett references, or may hedge where Barrett is plain, or might avoid issues Barrett handles directly. In such instances, Barrett is clearly the superior source, and particular attribution to Barrett as though his opinion is singular would clearly be in violation of our guideline against particular attribution."[43]

Quackwatch can be a valuable source, but it's not the best MEDRS out there, and certainly isn't as good as sources like the Cochrane Collaboration and other mainstream, peer-reviewed sources. I doubt that I'd put it in the same tier as sources mentioned under Wikipedia:MEDRS#Medical_and_scientific_organizations, either, because (its advisory board notwithstanding) there's no apparent mechanism of peer review.

When a topic is not discussed by superior MEDRS's, Quackwatch can be an ideal counter to self-published, WP:VANITY claims; but when a topic is discussed by superior MEDRS's, Quackwatch can still be cited, but with less weight. In that light, I agree with Eubulides' caveat above about throwing out the baby with the bathwater. My position is simply that we should give Quackwatch no more or less than its due weight per WP:PARITY, and always use inline attribution with a wikilink (which is the default, safe, wise approach on WP anyway). --Middle 8 (talk) 19:59, 11 August 2009 (UTC)

I'm not sure that "poorly sourced claims" is the right title here. A widely accepted fact could be poorly sourced (e.g, to someone's blog, because it was handier than a medical textbook that says the same thing), and this paragraph doesn't seem to address that. This seems to deal more with controversial claims (everything from "no data exists" to "a tiny minority thinks the Earth is flat"). WhatamIdoing (talk) 20:24, 11 August 2009 (UTC)

@WhatamIdoing: agreed (also ditto 2/0 above[44]). --Middle 8 (talk) 23:14, 11 August 2009 (UTC)

Okay. I have tried to take all the comments and considerations into play. Middle 8, we're going to have to agree to disagree about what this particular paragraph can be construed as -- but actually I think that's okay for now. ScienceApologist (talk) 17:16, 14 August 2009 (UTC)

Yes, we can take that up later, when we have a concrete example. regards, Middle 8 (talk) 07:11, 15 August 2009 (UTC)

I disagree, if the source is not reliable we should not list the information. Defining what's "widely believable" with no reliable source to support that information is WP:OR--Nutriveg (talk) 17:41, 14 August 2009 (UTC)

I agree that "if the source is not reliable we should not list the information". I also agree that defining what is "widely believable" with no reliable source is WP:OR. However, I do not think that the wording even came close to implying either of those points. Can you explain how you came to this conclusion and perhaps offer an alternative wording? ScienceApologist (talk) 20:11, 14 August 2009 (UTC)

"Sources written and reviewed by the advocates of such marginal ideas can be used to describe notable personal opinions". Who says what is notable and what is not?--Nutriveg (talk) 20:43, 14 August 2009 (UTC)

We have guidelines which give us a good clue: WP:N is the general notability guideline, but here I think WP:FRINGE certainly applies. Do you disagree? Would wikilinking to either of those two pages help? ScienceApologist (talk) 21:05, 14 August 2009 (UTC)

WP:N doesn't apply, since it "refers to whether or not a topic merits its own article". WP:FRINGE still "demands reliable (...) sources" where "sources written and reviewed by the advocates" are not.--Nutriveg (talk) 21:18, 14 August 2009 (UTC)

So, WP:N applies if we are talking about particular articles, which it seems to me we may sometimes (but by no means always) be doing. WP:FRINGE#Independent sources says, explicitly, "While fringe theory proponents are excellent sources for describing what they believe, the best sources to use when determining the notability and prominence of fringe theories are independent sources." Seems to me to be pretty much the same thing. Would a direct quote help? ScienceApologist (talk) 21:22, 14 August 2009 (UTC)

A direct quote would help. But I wonder how many "notable personal opinions" are there, how contradictory they can be (between each other and between itself) and how big is the mess all that can make.--Nutriveg (talk) 21:42, 14 August 2009 (UTC)

Believe me, I'd prefer to exclude all the personal opinions and just stick to asserting facts. But the problem with this is that there are some personal opinions which are notable and encyclopedic. I'll give you one example: the Atkins diet. Many, many problems with the proposals made by its advocate, but still a notable opinion if indeed fringe. Below is my addition of the quote. Please tell me what you would like to tweak, change, etc. ScienceApologist (talk) 21:55, 14 August 2009 (UTC)

Controversial claims

While many medical subjects are covered by a plethora of reliable sources, there is a sizable number of notable though marginal medical claims where little to no reliable research has been conducted about the efficacy, safety, or legitimacy of the statements made by proponents. In such cases, reliable sources may be much more difficult to find and unreliable sources can often be more readily available. Whenever writing about medical claims not supported by mainstream research, it is vital that third-party, independent sources be used. As described at our guidelines for fringe theory notability: "While... proponents are excellent sources for describing what they believe, the best sources to use when determining the notability and prominence of fringe theories are independent sources." Sources written and reviewed by the advocates of such marginal ideas can be used to describe notable personal opinions, but extreme care should be taken when using such sources lest the more controversial aspects of their opinions be taken at face-value or, worse, asserted as fact.

← I think there is a real need for language like this. If anyone doesn't believe this is a problem, I'll point them to a series of articles demonstrating the issue. Combine the following:

• One poor-quality, obscure, but marginally useable source on a fringe medical topic.
• Complete lack of actual reliable sources on the topic as it is well below the fringe/notability horizon in the real world.
• One determined single-purpose account.

The result is a low-quality article which can never be anything other than a) credulous snake-oil promotion or b) original-synthesis deconstruction of the nonsensical claim. And with one dedicated single-purpose editor (and a few of the usual suspects to chime in) the article can never be deleted through AfD. This is a real barrier to the goal of creating a serious, respectable reference work, so I would support some language clarifying the ways in which Wikipedia's core principles apply to such situations. MastCell ^Talk 04:11, 15 August 2009 (UTC)

Along these lines, I've been thinking about Protandim, which owes its AfD survival to one (major) media article and a couple of PubMed-listed journal articles. But, as is typical for new dietary supplements, the scientific articles are all written by proponents (e.g., people employed by the manufacturer). Peer review does not turn a proponent into a third-party, independent source.

On the other hand -- thinking about WP:BEANS, and the unsupportable claims made by an editor at Endoscopic thoracic sympathectomy that any research that is even slightly favorable is unacceptable because only money-grubbing surgeons that perform ETS write papers about it -- the fact that a person favors something, or gets paid for it, does not mean that the person is necessarily unreliable. I'm not sure whether we can (or should) address this issue here.

More helpfully, given the number of people that ask at WP:V about whether "third-party" means tertiary source, it might be worth linking the Wiktionary page in this text. WhatamIdoing (talk) 04:25, 15 August 2009 (UTC)

WhatamIdoing, these are fascinating issues you bring up. I was interested yesterday in thinking about how to write an article about Procera AVH which is essentially a dietary supplement hawked by self-described "brain fitness guru of our time", Joshua Reynolds and an Volunteer Clinical Professor of Medicine at UCLA Medical Center (one of 3300 medical doctors who volunteer a certain number of hours a month to be associated with the university), Robert Heller, MD. The company selling the product has been able to gain endorsements by a number of medical doctors and masters of science(!) and the supplement has been subject to clinical trials by three PhD research scientists who they explicitly call "objective third party scientists". The infomercial describes the supplement as "The Mental Edge" and offers a guarantee that Procera AVH will "eliminate brain fog, mental fatigue and forgetfulness." Looking for sources on this particular product I find mostly promotional stuff, a reference in the Skeptdic Newsletter, and, ironically, perhaps the most neutral source is a Yahoo Answers site: [45]. How we would go about writing an article is a touchy subject indeed considering the lack of independent MEDRS-quality sources. WP:PARITY is somewhat deceptive here considering the fact that this particular company has carefully collected the endorsements of credentialed scientists and medical doctors who they explicitly describe as "third party" while the truly "independent sources" are all anonymous internet commentators. It's almost as if this company had anticipated Wikipedia's sourcing policy and has done their best to essentially prevent us from being able to write a truly neutral article on the subject. I must admit to being at an almost complete loss for even knowing where to begin. One possibility is that Procera AVH is not notable enough for an article, but I have a sneaking suspicion that if someone started an article on the subject it would not be deleted. Perhaps it is time to consider a Wikipedia: Notability (medical claims) policy to address these issues directly. My feeling is that if the independent sources are of low quality, then it is likely that the subject itself is not notable enough for inclusion. I suspect the same thing applies to protandim. ScienceApologist (talk) 16:50, 17 August 2009 (UTC)

Yeah, I think that's a pretty good idea -- just don't have an article on stuff like that, and let's clarify the intersection of WP:N and MEDRS. That would probably be the easiest way to go. In fact I bet I could find several "masters of objective science", or whatever, right here on WP who would agree that such an approach is an excellent, holistic approach to headache-prevention (not notable enough for an article, of course, but probably dandy as part of a guideline ;-). regards, Middle 8 (talk) 02:42, 18 August 2009 (UTC)

Okay, I added a sentence to that effect. It mentions the possibility that poor independent sources can be used as an argument against the notability of the subject. This has potential revolutionary implications, but I am more-or-less confident it is the right mood since Middle 8 and I both agree broadly with its ideal. ScienceApologist (talk) 12:44, 18 August 2009 (UTC)

Broadly speaking, there seems to be agreement here that something along these lines is a positive addition to the guideline, yes? Does anyone mind if we go ahead and drop it in? - 2/0 (cont.) 06:12, 18 August 2009 (UTC)

I took the silence here and below to be assent. If I am wrong, WP:BRD should take care of it. ScienceApologist (talk) 02:15, 21 August 2009 (UTC)

Did this discussion ever go anywhere? I ask because of my involvement in this discussion: Wikipedia:Articles_for_deletion/Color_light_acupuncture. Derek Andrews (talk) 12:39, 18 September 2009 (UTC)

Yes, it was installed, and after further edits it is in WP:MEDRS #Use independent sources. Eubulides (talk) 17:39, 18 September 2009 (UTC)

Workshopping a proposal

Controversial claims

While many medical subjects are covered by a plethora of reliable sources, there is a sizable number of notable though marginal medical claims where little to no reliable research has been conducted about the efficacy, safety, or legitimacy of the statements made by proponents. In such cases, reliable sources may be much more difficult to find and unreliable sources can often be more readily available. Whenever writing about medical claims not supported by mainstream research, it is vital that third-party, independent sources be used. As described at our guidelines for fringe theory notability: "[P]roponents are excellent sources for describing what they believe[;] the best sources to use when determining the notability and prominence of fringe theories are independent sources." Sources written and reviewed by the advocates of such marginal ideas can be used to describe notable personal opinions, but extreme care should be taken when using such sources lest the more controversial aspects of their opinions be taken at face-value or, worse, asserted as fact. Alternatively, if the independent sources discussing a medical subject are of low quality, then it is likely that the subject itself is not notable enough for inclusion.

i can't say I understand the intent of the last sentence. AIs it saying we have two equally valid choices, 1/ to use such sources as there are in a way that makes the limited reliability clear OR, at our option, 2/not cover the topics entirely? IUf so, I don't see it as giving much guidance about what course to follow. My own view is that RSs for the purpose of notability can include even the most absurd material, as long as they have sufficient prominence. Otherwise we are in the unfortunate position of calling quackery what it is, but only if it's moderately respectable, and not covering the worst of it., which I do not see as NPOV treatment of the subject in general. DGG ( talk ) 23:15, 18 August 2009 (UTC)

Well, give us an alternative. We've outlined scenarios where the independent and most reliable sources are clearly not going to look superficially as reliable as the ones that Wikipedia would identify as such. I remarked myself that I'm not sure what to do with such claims and pointed out that a new Notability guideline would be appropriate. Middle 8, who often does not see eye-to-eye with me seemed to think that the solution I had of simply saying that subjects which have poor sourcing are not notable is agreeable to him. So, I went ahead and added the last statement to illustrate that point. I'm not sure what else to do. I could remove the sentence but that will not remove the problem. ScienceApologist (talk) 03:11, 19 August 2009 (UTC)

Perhaps too much is being read into the last sentence? If, for example, the New York Times has a story on the latest unsupported medical fad, then I'm sure we'd all agree that it is a high-quality source (even if the fad is unsupported). Conversely, if the only 3rd-party sources on a medical fad are anonymous blogs, I'd be leery of covering the fad at all. In between, there are judgment calls, but I don't see how the current wording precludes judgment calls. I did see several opportunities for tightening up the text without (I think) affecting its gist or intent, which I installed. The most major changes are to the title of the subsection (which I reworded to make it fit better with its sibling subsection titles), and to the lead sentence (which I trimmed heavily for focus), and to remove a footnote citation to Wikipedia itself (which was a bit odd; that's what wikilinks are for; and anyway would be a pain to maintain the self-quote). Eubulides (talk) 22:18, 21 August 2009 (UTC)

Use of inconclusive commentary as a source.

Hi, there is an ongoing discussion in the Wikipedia:Reliable sources/Noticeboard#Sources appropriate for medical and health-related articles about the use of an inconclusive commentary as a source for a medical and health-related article, that relates with WP:MEDRS. So your opinion is welcome.--Nutriveg (talk) 20:17, 16 August 2009 (UTC)

Since that discussion was mentioned in this page I'll take its conclusions as discussed here.--Nutriveg (talk) 14:20, 17 August 2009 (UTC)

Sorry, but I can't interpret the previous comment. I don't know what it means. Eubulides (talk) 16:26, 17 August 2009 (UTC)

This isn't the first time this question has been raised. I've boldly taken a stab at it with this text:

Most journals also publish pieces that are not scientific articles, such as editorials, op-ed pieces, commentaries, and letters to the editor. These sources may provide interesting ideas, but they are never peer-reviewed and are rarely even fact-checked, and therefore they should be treated with caution. In general, such pieces must comply with Wikipedia's standard rules for self-published works, including rejecting claims outside the author's expertise and requiring direct attribution of the claims to the source.

My short summary is "mere assignment of a PMID does not make it a peer-reviewed article." If anyone has any improvements, please be bold. WhatamIdoing (talk) 16:39, 17 August 2009 (UTC)

One issue is with the wording, "they are never peer-reviewed". This is not technically true in the sense of informal peer review. I know of a number of journals which will essentially not publish letters-to-the-editor which are blatant mischaracterizations, for example. The peer-review may not be as rigorous as for a journal article, but there is a level of editorial control. I do not think that these sources necessarily correspond to self-published works. They are somewhere in between. This issue extends well-beyond MEDRS. ScienceApologist (talk) 17:01, 17 August 2009 (UTC)

That definition of 'informal peer-review' only works if you assume that the editor of your local newspaper does 'informal peer-review' when selecting letters from readers to publish on the letters-to-the-editor page. Editorial selection is not the same as either editorial control or peer review. Wikipedia has a long-standing consensus that letters to the editor in The New York Times should be seen as essentially self-published; I see no particular reason to say that a letter to the editor in an obscure medical journal should be given more standing than NYT. WhatamIdoing (talk) 18:14, 17 August 2009 (UTC)

I'm not so sure it's that black-and-white. There are some powerful editorials written in Science (journal) and Nature (journal) that are subject to more editorial control than many papers published in more obscure journals. ScienceApologist (talk) 19:06, 17 August 2009 (UTC)

I'm willing to remove proper editorials from this paragraph: anything actually written by the editor meets every possible definition of editorial control (although they're still not peer-reviewed). I think what I've written is still true about op-eds, commentaries, and letters. WhatamIdoing (talk) 19:32, 17 August 2009 (UTC)

How about this?

Some reputable journals also publish pieces that are not strictly journal papers, such as editorials, op-ed pieces, commentaries, and letters to the editor. These sources may provide interesting ideas, but they should be treated with caution as the peer-review status and accuracy of the text can be nebulous. In general, such pieces must comply with Wikipedia's standard rules for self-published works, including rejecting claims outside the author's expertise and requiring direct attribution of the opinions of the author to the source.

ScienceApologist (talk) 20:15, 17 August 2009 (UTC)

This edit inserted a paragraph saying that editorials etc. must have direct attribution of claims to the source. If I understand that correctly, it goes too far: if (say) a NEJM editorial says that the use of Lupron to treat autism is controversial, we need not have the article text say "Mr. Schmoe wrote that the use of Lupron to treat autism is controversial". Also, if a letter to the editor by an article's author corrects information in that article, surely a citation to the letter suffices; we don't need intext attribution. Furthermore, the change caused the project page to be incoherent: in one paragraph it says that editorials are articles, and in the next paragraph it says they aren't. I'm not sure what this change is motivated by, but I suggest that it be discussed here further before installing. Eubulides (talk) 16:47, 17 August 2009 (UTC)

I really, really hate the direction that people are taking with attribution. The countervailing arguments are twofold: WP:ASF and WP:FRINGE#Particular attribution. If a letter to the editor is written by an expert and the fact asserted is in concord with standard texts/reviews/etc. on the subject, then attribution of the fact to the letter-writer is, in my estimation, a violation of WP:ASF. For example, imagine a commentator writing something like, "This particular supplement relies on claims that contradict basic physiology." We can verify whether the author of the editorial is correct in his or her analysis and if the statement is a fact, simply referencing the commentary is perfectly fine - especially since many people write more clearly in commentaries and editorials than they do in academic articles. Likewise, I think that the "particular attribution" game is obviously being done as a way to weasel out of uncomfortable facts. Perhaps we should relocate particular attribution to the WP:NPOV page since it is sometimes done amongst legitimate controversies (not just FRINGE issues)? ScienceApologist (talk) 16:57, 17 August 2009 (UTC)

SA, I think you and I would do the same thing in practice. My concern is primarily about citing commentaries and letters as "facts" when you can't find a better source because no better source exists, not when you're thinking that any source is probably good enough. For example: I'm worried about using these kinds of sources for "balancing" statements against a generally accepted claim made in a peer-reviewed article, and the only source you can find is a letter to the editor.

For a live example, efforts to promote the POV of certain trans activists have recently resumed at The Man Who Would Be Queen; among other things, a publish-all-responses call for commentaries on a peer-reviewed history-of-the-scandal article has resulted in several trips to RSN ([46][47][48][49]) over whether you can selectively quote the opinion of a non-academic physician about whether, e.g., having your children's names and school photos published with captions like "Kate: a cock-starved exhibitionist, or a paraphiliac who just gets off on the idea of it?" as retaliation for "insulting" the trans community (by publishing your current scientific views about transsexuality) has a chilling effect on academic freedom. WhatamIdoing (talk) 18:38, 17 August 2009 (UTC)

Good example. The ridiculous text that is passing for analysis at the article on Quackwatch might also be a good object lesson. In general, I think that any time people refer to "balancing" they are usually trying to controvert NPOV. I spent a good number of months trying to excise the word "balance" from NPOV for this very reason and now it doesn't appear. The real key is to remember WP:WEIGHT which seems to be the thorn in the side of most POV-advocates. ScienceApologist (talk) 19:06, 17 August 2009 (UTC)

Eubulides, his text says "scientific articles", where the current text says "journal articles", you may change it to "scientific studies" if that definition sounds better. The motivation is on the link on the top of this section.--Nutriveg (talk) 17:40, 17 August 2009 (UTC)

(Unindenting, as the above is too hard to follow.) I have some concerns about ScienceApologist's above proposal (dated 20:15, 17 August 2009). I am mostly concerned about Letters to the Editor. Since these are the standard method by which errors in published work are discussed, I think it would be a mistake to craft a rule that permits an erroneous paper to be cited but not a Letter to the Editor describing the error. Consequently, I think care is needed. In particular, I don't think that we should classify this material as self-published. It's considerably more reliable than that. At the very least, it will have received the attention of the Editor, who will have made the decision to publish (based, presumably, on merit). And some journals (for example the International Journal of STD and AIDS) apply a full peer-review process for Letters to the Editor. I suggest simply stating that such material should be treated with caution, that it may be worth determining what review process (if any) applies, and that extra care should be taken to watch for dubious or surprising claims that may require stronger sourcing. Jakew (talk) 13:55, 18 August 2009 (UTC)

Great point. The "in general" may not be a sufficient enough caveat to cover the examples you give. This proposal needs more work. ScienceApologist (talk) 14:07, 18 August 2009 (UTC)

Critiques on possible problems of an specific study are also usually presented as letters to the editor; and as such I believe that are of specific relevance when presented in conjunction with the commented article; since the letter addresses possible specific problems of the first. Maybe it would be a good idea to specifically say something like When they have an specific article as its topic; although they can not be used to debunk its conclusions they can give some indication on its possible strong points or limitations. (My wording is probably not very good; feel free to make alternative proposals if agreed with content). Bests.--Garrondo (talk) 16:23, 18 August 2009 (UTC)

"How to Critically Appraise an Article: Selection and Critical Appraisal of Research Literature"

Review in Nat Clin Pract Gastroenterol Hepatol. Free full text at Medscape, could be a useful source. Fvasconcellos (t·c) 16:46, 22 August 2009 (UTC)

Colin already added that here in March. Medscape is pretty slow, compared to our better editors. Eubulides (talk) 22:42, 22 August 2009 (UTC)

Wow. *facepalm* Fvasconcellos (t·c) 23:33, 22 August 2009 (UTC)

In the presence of Colin, all bow and be humbled! SandyGeorgia (Talk) 00:59, 23 August 2009 (UTC)

Colin's less than brilliant prose recently featured as an example in one of Tony's exercises. Fvasconcellos, respected admin whose brilliant prose and scholarship have won the unanimous praise of his peers, has nothing to feel humbled about. Sandy, what on earth does "take your tomato" mean? Colin°^Talk 10:07, 23 August 2009 (UTC)

Guidelines should not cover medical-physiological articles

There are good reasons for the existence of specific medicine-related reliability criteria. But the guidelines fail to explain them with the result that its application is too wide.

• Medical articles can potentially affect people's medical treatment and health decisions. It is therefore important that all treatment related information in Wikipedia is not only reliable but also has medical consensus. Thus they should be based upon not only upon primary sources but also secondary sources from recognized publications that filter and discuss them. The medical community recognizes this need and so creates many reviews and textbooks that discuss such primary sources.
• But many articles that are "medically-related" concern physiological processes and lack implications for any treatment or health related decisions. These need to be reliable but the issue of consensus is not relevant and reflecting this journals publish many fewer secondary reviews upon them. Moreover in many areas of physiology research progress is so fast that field reviewing has shifted from review articles to the review sections that make up the introductory sections of research papers. In addition since the issue of consensus is not so important as advancing a field of research, where reviews are written, editors often allow what might be called position reviews in which a particular viewpoint is presented in the context of a general advocacy. This makes many secondary sources in physiology less reliable than many primary ones which only report research findings.
• It is therefore inappropriate that physiological articles should ideally be sourced upon reviews rather than reliable primary research journal ones.
• The medicine-related guidelines should reflect this and limit the need for secondary sources only for treatment or health related articles with physiological ones having the same reliability requirements as other scientific Wikipedia articles.--LittleHow (talk) 00:57, 23 August 2009 (UTC)

WP:MEDRS currently says in its title that it's about "medicine-related articles", in its nutshell that it's about "biomedical articles", and in its lead that it's about "medical articles" which are a "source of health information", and later that it's about "medical and health-related articles". The intent here, as I see it, was to focus on medicine and articles closely related to medicine, not on articles that are about biology in general. There are of course gray areas, and physiology is one of them. At one extreme, WP:MEDRS is certainly not intended to apply to the Plant physiology article; at the other, WP:MEDRS certainly would have something to say about the Physical exercise article, which Template:Musculoskeletal physiology links to. I sense that the previous comment, though written in general terms, is motivated by one or more specific examples; if so, it would be helpful to know what those examples are. Certainly it is not unique to medicine that secondary sources are preferred; see WP:PSTS. Eubulides (talk) 05:49, 23 August 2009 (UTC)

I disagree with both the assumptions and the conclusion here. It's true that basic physiological processes aren't usually the subject of recent literature reviews, but there are other excellent secondary sources available, such as college and medical school textbooks.

It's also worth remembering that the preference for secondary sources isn't something dreamed up on this page: it is a Wikipedia-wide standard that all articles should comply with, regardless of subject matter. WhatamIdoing (talk) 23:50, 23 August 2009 (UTC)

Pharmanoia

Are there any essays, guidelines or policies that talk about pharmanoia, and the idea that sources shouldn't be trusted because of Big Pharma having sticky fingers in things? I've been treating such arguments as spurious without a specific source criticizing a specific paper. I'm also interested in whether there's something similar for when alt med advocates cite the pharmanoia argument to support the idea that vitamins are a panacea and preventive wonder for all things (also would be nice to have something to point to that states the alt med advocates often have their own conflict of interest, and are not immune merely because they aren't Pfizer). WLU (t) (c) Wikipedia's rules:^simple/_complex 20:47, 3 September 2009 (UTC)

There's obviously a borderline between healthy skepticism and abusive paranoia. Unfortunately, that dividing line is grounded in commonsense, which seems largely absent from many quarters of Wikipedia. Leaving aside the more loony and self-serving invocations of pharmanoia, WP:WEIGHT is a tricky issue. Journal editors obviously feel that such potential conflicts of interest are relevant (as do most journal readers) - hence the disclosure statements accompanying articles. However, if we translate that onto Wikipedia, it's grossly disproportionate to give one sentence to a study's findings and one sentence to the study's funding - that does not accurately reflect the weighting in the source, and it gives undue weight to the funding issue. I'm not sure what the right answer is, other than to punt it and handle it "on a case-by-case basis". MastCell ^Talk 21:03, 3 September 2009 (UTC)

I believe editorial judgment is the best guideline here :) Now, as for essays... MastCell, doesn't that vaguely resemble a challenge? Fvasconcellos (t·c) 21:22, 3 September 2009 (UTC)

I'd prefer to call it an "opportunity". :) MastCell ^Talk 21:33, 3 September 2009 (UTC)

If I don't see an essay in the near future, I'm switching to a different histamine-themed favourite admin. Ben Goldacre might have some comments on his blog. WLU (t) (c) Wikipedia's rules:^simple/_complex 22:54, 3 September 2009 (UTC)

Not to sidetrack the discussion, but someone should really tell Ben to submit a better picture for his article. :P MastCell ^Talk 23:57, 3 September 2009 (UTC)

Aspartame controversy

I am in doubt about a reference that has been added. To me it sounds like dubious woo-woo advice by pushers of dietary supplements that isn't backed up with good evidence. The ref is being used to justify mentioning that aspartame can produce migraines. While there is a subset of individuals that are sensitive to it, the ref is being used to imply that it causes migraines, without qualifying the subset it applies to:

• The diff

• The referenced article

The edit seems to be pushing an unjustified and POV use of the source. Brangifer (talk) 04:43, 11 November 2009 (UTC)

Sheesh. Why not simply reword the statement to read

In some people, aspartame may act as a migraine trigger, as may many other food additives and naturally occurring substances.

—Fvasconcellos (t·c) 13:12, 11 November 2009 (UTC)

Thanks, I added wording along those lines, and followed up at Talk:Aspartame controversy #Association with migraine. Eubulides (talk) 19:03, 11 November 2009 (UTC)

Excellent, glad I could help :) Fvasconcellos (t·c) 19:42, 13 November 2009 (UTC)

Talk:Thiomersal controversy#Introduction Neutrality Dispute

An IP editor (131.215.40.141 and 131.215.6.110) seems to need advice regarding the importance of not depending on primary sources, on the difference between in vitro and in vivo, and on how the dose makes the poison. -- Brangifer (talk) 04:51, 17 November 2009 (UTC)

You might want to move this thread to WT:MED, which is watched by about four times the number of editors. WhatamIdoing (talk) 06:02, 17 November 2009 (UTC)

OK. -- Brangifer (talk) 06:29, 17 November 2009 (UTC)

Medicinal clay

This article needs to be gone through. I suspect there are many false claims backed up by sources that appear to be official medical journals, but are really fringe journals not even listed in PubMed. -- Brangifer (talk) 15:23, 22 November 2009 (UTC)

This is the talk page for discussing improvements to the WP:Reliable sources (medicine-related articles) page. When you want to let people know that specific articles are in need of responses, you will get more responses at WT:MED.

I would be surprised if this Alternative medicine topic didn't have some non-mainstream sources. AltMed topics are like that. WhatamIdoing (talk) 17:12, 22 November 2009 (UTC)

They don't seem to appreciate me coming there instead of here. It seems that this is the closest thing to an "WP:RS/N" for MEDRS topics, so that's why I'm starting here. (We really should have a section of WP:RS/N for such matters: WP:RS/MEDRS) I expect alt med topics to contain non-mainstream sources, but they shouldn't have undue weight and shouldn't be misleading. If they stand alone, then they mislead readers. MEDRS guidelines are supposed to correct such imbalances. -- Brangifer (talk) 22:55, 22 November 2009 (UTC)

Hair analysis (alternative medicine)

This section seems to be one huge special pleading section with peacock language and undue weight given to alt med advocates like Haley and Blaxil from SafeMinds, and ancient claims made by Bowen. (Yes, believe it or not, Bowen does rant about Satan, Zionism, and the Illuminati, and in his letter to Martini explains why his medical license was revoked.) While the fringe position should be stated clearly, it shouldn't be allowed to deceive the public by making their claims seem plausible through a presentation marred by undue weight. -- Brangifer (talk) 00:10, 23 November 2009 (UTC)

What "assess the evidence" means

I've been looking through some conversations from earlier this year, and I have run across an area that WP:MEDRS#Assess_evidence_quality may need to provide a little more clarity. It seems that this section is being occasionally (mis)understood as requiring editors to decide whether a study (or review) has sufficient data to support the conclusions that it draws.

So what we mean -- and what I thought we said -- was that if you're faced with contradictory papers, or trying to decide whether something's important enough to include, then you prefer reviews and meta-analyses to original RCT reports, and that you prefer RCTs to case studies, and so forth.

I'm seeing this invoked as a justification for WP:OR: I didn't think that their data sample was big enough to justify that conclusion... and the review cites Smith's trial for this fact, so that's just a single original paper... and Smith's paper is a little old, anyway, even though the review is recent... and probably the reviewer is twisting Smith's conclusions... -- even though we're citing the review instead of Smith.

I think that the problem might actually be in the section's title. Could we consider renaming it to something less open-ended, like "Prefer well-designed studies"? WhatamIdoing (talk) 19:58, 19 October 2009 (UTC)

Wikipedia should be a tertiary resource and so if it comes down to determining whether a single paper is reasonable or not or which of a pair of contradictory papers Wikipedia should be citing, it is always best to go with secondary sources. In general, we should be encouraging editors to use WP:PSTS and avoid primary sources completely. The purpose of the encyclopedia is not to stay on the cutting-edge of medical wisdom but to provide a dispassionate and removed summary of the most conventional and staid wisdom about notable subjects. To that end, I'd say what we should retitle the section Use consensus summaries or something of that nature. ScienceApologist (talk) 21:52, 19 October 2009 (UTC)

I very much agree with ScienceApologist. Regards, —Mattisse (Talk) 22:20, 19 October 2009 (UTC)

Yes, of course. But that point is fully covered in WP:MEDRS#Respect_secondary_sources, and is not the point that I'm talking about. My question involves the section that compares meta-analyses to individual trials, and individual trials to case studies, and case studies to pure speculation. The title of this section seems to be misunderstood as encouraging editors to pass judgment on whether a secondary source has accurately represented the primary works that it discusses. WhatamIdoing (talk) 22:30, 19 October 2009 (UTC)

Not sure that the point is fully covered there, actually. Essentially all journal articles can be considered to be primary sources whether they are meta-analyses, RCTs, case studies, etc. A secondary source would review the state of the topic and report the consensus opinion. Those are the best sources to use. The issue I have with this section is that it does not go far enough in making sure that primary sources are avoided whenever possible. I guess you can say that I'm a fan of, "When in doubt, ask for an expert summary." ScienceApologist (talk) 22:47, 19 October 2009 (UTC)

Again, I agree with ScienceApologistm having dealt with many medical/psychiatric articles recently where primary sources are considered ok. Not only primary sources, but old primary sources. Regards, —Mattisse (Talk) 22:56, 19 October 2009 (UTC)

Again, encouraging the use of secondary sources is not the point of this subsection. No matter how fervently we all agree that secondary sources are wonderful -- and I believe that we do agree on that point -- our agreement does not change the content of this particular subsection to be about secondary sources instead of about study designs. WhatamIdoing (talk) 23:35, 19 October 2009 (UTC)

Why not just rename the section to "Use high-quality sources"? That matches the section's contents better anyway, and would help avoid the possible misunderstandings noted above. Eubulides (talk) 23:25, 19 October 2009 (UTC)

Because an elderly self-published source is not a high-quality source even if it is a meta-analysis. Study design is not the only factor that determines high-quality sources. It is, however, the only factor addressed in this particular subsection. WhatamIdoing (talk) 23:32, 19 October 2009 (UTC)

But the section is about more than study design. It also emphasizes reviews and other secondary sources that are not research studies in the usual sense. I'd rather not have the section header limit itself to "studies", which sounds too much like primary research studies. Eubulides (talk) 23:44, 19 October 2009 (UTC)

Agree. We should not be int the business of "evaluating" primary research studies. Regards, —Mattisse (Talk) 00:06, 20 October 2009 (UTC)

Eubulides, I take your point about "studies", although the major point of this section is clearly just that. I'm open to alternative titles.

Mattisse, with respect, we should be in the business of evaluating primary studies, because this guideline instructs us to do that. If you do not stop to figure out what kind of paper you're looking at, you'll treat an outdated narrative review of three case studies as being far more reliable than the a recent very large randomized controlled trial on the same point. WP:PSTS is not the only valid consideration, and this guideline has always encouraged appropriate use of primary sources. WhatamIdoing (talk) 00:19, 20 October 2009 (UTC)

How about Consider study design? I think asking editors to conduct assessments of studies is wildly inappropriate, and maybe even insisting on "preferences" can be construed problematically, but exhorting people to consider the designs of studies when writing prose is okay in my book. ScienceApologist (talk) 03:35, 20 October 2009 (UTC)

The "Consider" is fine, but as mentioned above the section is about more than just study design. How about "Consider the type of source"? Also, in the 1st sentence the phrase "assessing whether a particular viewpoint" should be changed to "considering whether a particular viewpoint". Eubulides (talk) 16:33, 20 October 2009 (UTC)

"Consider the type of source" will be misunderstood as "Classify it according to WP:PSTS," which would make this redundant with the previous section (which is all about PSTS issues).

I don't think that there's really very much in this section that isn't directly or indirectly about study design. WhatamIdoing (talk) 02:53, 1 November 2009 (UTC)

A lot of it is about study design, but some of it is about other kinds of sources as well (meta-analyses, reviews). How about "Critically appraise sources"? This title was inspired by the cited source Young & Solomon 2009 (PMID 19153565), "How to critically appraise an article". The section's lead sentence should be altered to match the new title: "Several systems exist for assessing the quality of available evidence on medical subjects, and these Sources for biomedical topics often vary widely in quality, and this should be kept in mind ...". Eubulides (talk) 02:42, 3 November 2009 (UTC)

Too vague, and misleading. Source has a technical definition on Wikipedia (see the footnotes at WP:V), and what this section is about has nothing to do with that. The main point of this section is: A meta-analysis trumps a case study. I want a title that specifically deals with the main point. WhatamIdoing (talk) 04:35, 3 November 2009 (UTC)

That's an important point of the Assess evidence quality section, but it's not that much more important than the section's other points. The section also mentions systematic reviews (at the same level as meta-analyses), narrative reviews, and clinical experience, and none of these fit into the narrow rubric of "study". As for the technical definition of source in WP:V, although the section focuses on articles, it also briefly talks about publishers and authors, so the word "sources" is not out of place here. Eubulides (talk) 04:55, 3 November 2009 (UTC)

Perhaps I'm missing something obvious, but I can't find the words "publisher" or "author" anywhere in this section. I see nothing in these two paragraphs that is unrelated to figuring out whether or not a given claim is based on high-quality data -- from "meta-analyses" at the top of the scale to "speculative proposals" at the very bottom.

I agree that a narrative review isn't technically a 'study'. However, a systematic review generally is taken to be a study (of the previously published literature). (It's not an "experiment," but neither is a case report.) WhatamIdoing (talk) 19:29, 11 November 2009 (UTC)

The words "publisher" and "author" are not in the section, but the topic of publishers is mentioned when the section talks about "popular press or a peer-reviewed medical journal", and the topic of authors is raised when the section talks about "expert committee" and "respected authorities". I take your point that the term "study" sometimes includes reviews; but often it doesn't, and it'd be better to avoid ambiguity here. How about "Consider reviews and study design"? Eubulides (talk) 21:16, 11 November 2009 (UTC)

That phrase means two unrelated things to me.

When we tell an editor to "consider reviews", we mean "consider relying on them -- they're usually pretty good sources for all sorts of purposes". When we tell an editor to "consider study design", we mean "This paper is just a short case series, and that's pretty weak evidence" (or "That's a meta-analysis, which is strong evidence").

IMO, the point of this section is the second, not the first. WhatamIdoing (talk) 22:27, 11 November 2009 (UTC)

New template

The new template is interesting, on the one hand, but based on its sole use to date,[50] I suspect that it may be no more useful than Template:Verify credibility, and may be misused. MEDRS doesn't presume to provide standards for strictly social facts, such as whether scientists "should" publicly oppose quackery, or whether a country devotes sufficient resources to health care for a given marginalized population, or what the polite term is for a given condition, or that sort of thing. Instead, it cares mostly about physical facts, of the type that you can determine with a well-designed trial or three. The other RS-related guidelines are quite sufficient for these non-scientific statements.

Does anyone else have any thoughts about this new template? WhatamIdoing (talk) 02:52, 1 November 2009 (UTC)

I think I'll use it a lot, and it will help shorten edit summaries and explanations ! SandyGeorgia (Talk) 02:54, 1 November 2009 (UTC)

Its contents "MEDRS violation?" were too cryptic; almost nobody outside our little circle would know what that means. I changed it to "Reliable medical source?". Like SandyGeorgia, I think it could be useful; this is true regardless of whether it was used well or poorly in its first use. Also, I noticed that {{fix}} says "please do not use this template to create a new sub-template without discussing and justifying the need for it at Wikipedia talk:WikiProject Inline Templates", so I just now created a thread Wikipedia talk:WikiProject Inline Templates #New template MEDRS to give that project a heads-up. Eubulides (talk) 06:19, 1 November 2009 (UTC)

Scope of this guideline

This guideline implicitly defines its scope as all medical articles. While reasonable in principle this is probably not precisely what is intended, or at least should not be. I believe the situation is parallel to WP:BLP, which generally talks about biographic articles but really means biographic information in any article.

Here are two real examples of the problem.

• Thalidomide doesn't have a section "Thalidomide in popular culture", but it does have something that comes close: Thalidomide#Notable children affected.
• List of common misconceptions#Human body and health contains medical advice about sugar/hyperactivity, wet weather/common cold, sleepwalking, vitamin A/eyesight and duct tape/warts.

The first problem is addressed by a paragraph in WP:MEDRS#Popular press. It makes clear that for non-medical sections in medical articles there is a relaxed standard. It does not try to define this standard; clearly such an attempt would be out of proportion in this guideline. It would be more efficient to delegate this to WP:RS.

I can't see that the more serious second problem is addressed anywhere: Medical information in List of common misconceptions seems to be outside the scope of this guideline! Therefore I propose adding a new section at the end of the lead:

"For simplicity this guideline refers to medical articles. This is not meant to restrict or define its scope: This guideline applies to medical information in all articles.

Hans Adler 11:12, 2 November 2009 (UTC)

You're absolutely right that the guideline is intended to apply to all medical facts and figures, regardless of whether the overall article topic is exclusively medical; this is already indicated by the guideline's title of medicine-related articles). But rather than introduce that disclaimer, I think it'd be better to systematically fix the language in the guideline to make this clearer; that way readers won't have to know about the disclaimer in order to understand the rest of the article. I did that (it was a surprisingly small change). Eubulides (talk) 18:10, 2 November 2009 (UTC)

Not sure if this adds anything, but the WP:FA category is called "Health and medicine", and there's also the "Psychology" and "Biology" categories. SandyGeorgia (Talk) 18:27, 2 November 2009 (UTC)

Hans, is there an active dispute that centers around this distinction, or is this purely hypothetical? WhatamIdoing (talk) 03:27, 3 November 2009 (UTC)

Neither. I am afraid there may be a dispute at List of common misconceptions sooner or later. I feel rather strongly about us not distributing medical misinformation based on random newspaper reports. And as you know people tried to argue that MEDRS must be applied to our reporting of a political controversy in the past, at Aspartame controversy. Hans Adler 07:53, 16 November 2009 (UTC)

Excellent thread Hans, and thanks to Eubulides for the simple fix. (I notice a couple places where more of this type of fix can be done. I'll take a whack at it.) MEDRS doesn't replace RS, but supplements it: "These guidelines supplement the general guidelines at Wikipedia:Reliable sources..." In any given medical article or article that contains biomedical information, MEDRS applies to the scientific nitty gritty details, not to the political, controversial, historical and cultural details. That is still covered by RS. That which is falsifiable will often come under the scope of this guideline. -- Brangifer (talk) 14:59, 16 November 2009 (UTC)

Need for WP:MEDRS/N noticeboard

I have started a thread and hope readers here will chime in there:

We need a parallel noticeboard to deal with medical sourcing: WP:MEDRS/N. Currently such matters end up at Wikipedia talk:Reliable sources (medicine-related articles) and Wikipedia talk:WikiProject Medicine, but that's unsatisfactory. We need a separate noticeboard which is watched by medical professionals who understand the complexities of medical matters and medical research, understand our RS policy, and understand the MEDRS guideline. What think ye?

Please discuss it there, not here. -- Brangifer (talk) 00:50, 23 November 2009 (UTC)

That discussion seems to have arrived at a consensus not to create a new noticeboard. There is longstanding consensus on this talk page that specific issues should be discussed on Wikipedia:Reliable sources/Noticeboard, WT:MED or WT:PHARM, so I suggest moving the next two threads off this page (where they are getting ignored) and onto one of the other pages. Eubulides (talk) 07:47, 2 December 2009 (UTC)

Support. WT:MED seems appropriate as there isn't one specific reliable-source issue for the RS noticeboard to discuss. Colin°^Talk 08:58, 2 December 2009 (UTC)

At least we all agree that this talk page shouldn't be used for that purpose, but when the other places say to come here, and the RS noticeboard is often useless for this purpose, we're left with the need for a WP:MEDRS/N. What should be done? It's frustrating when an obvious need is being ignored. The very fact that this MEDRS variation of the RS guideline exists at all reveals the need for a special place for MEDRS issues to be discussed. The closest thing that sometimes works is the Wikipedia:Fringe theories/Noticeboard, but it's not limited to medical subjects, and it doesn't have a cadre of editors who watch it who are specifically interested in RS issues. I agree that this talk page isn't really the proper venue, but until we establish a MEDRS/N, it's the closest thing available. When this door is closed, at least open another one. -- Brangifer (talk) 15:13, 2 December 2009 (UTC)

I agree that it would be very nice to have a WP:MEDRS/N noticeboard, watched by experts; but the consensus in the other discussion seemed to be that we lack resources to do it. In the meantime, for occasions where Wikipedia:Reliable sources/Noticeboard is useless, this talk page suggests using WT:MED or WT:PHARM. Can you give an example of trying (say) WT:MED, why the result was unsatisfactory, and why the result would be different with a WP:MEDRS/N (assuming that we don't have any more resources than we do now)? Also, what "other places say to come here"? Eubulides (talk) 19:20, 2 December 2009 (UTC)

Yes, what other places say to come here (to the talk page for a discussion, not to the guideline for information) with these kinds of issues? That needs to be fixed. WhatamIdoing (talk) 20:22, 2 December 2009 (UTC)

I do have a problem with the Wikipedia:Reliable sources/Noticeboard, usually the first people to answer are not used with WP:MEDRS and those who should know it and join the discussion usually do so to support their POV. I cite an example.--Nutriveg (talk) 20:39, 2 December 2009 (UTC)

The commenters in that example seemed fairly well versed in WP:MEDRS. The commenters all had opinions, but I don't see how that discussion would have played out differently if it had occurred on the proposed WP:MEDRS/N noticeboard. Eubulides (talk) 20:49, 2 December 2009 (UTC)

In the end the editorial (commentary) was included and got same weight as the meta-analysis/review (secondary source).

ProtonK was unwilling to discuss the issue "My first suggestion is to ask at WT:MEDRS", Irbisgreif didn't help at all, ScienceApologist was more interested about truth than verifiability, Garrondo was supporting his POV, "2/0" didn't found appropriate to discuss how it should be used "Whether the source should be used is beyond the remit of this board". Too much noise.--Nutriveg (talk) 21:22, 2 December 2009 (UTC)

???: Frankly I do not see relationship with present discussion... Since most people involved in that discussion are heavily involved in the medicine project, and more opinions were asked in the med project I am quite sure that the discussion result would have been the same.--Garrondo (talk) 23:13, 2 December 2009 (UTC)

Conference abstracts

Is there any guidance anywhere on the use of conference abstracts and symposia? My find function doesn't turn anything up, and who wants to read all that WLU (t) (c) Wikipedia's rules:^simple/_complex 13:38, 3 December 2009 (UTC)

See Wikipedia talk:Reliable sources (medicine-related articles)/Archive 3#Journal supplement for a related discussion. I don't think it resulted in any changes to the guideline. What is your opinion on the issues? Colin°^Talk 15:39, 3 December 2009 (UTC)

Interesting, but no real consensus on a change... My understanding is conference presentations are essentially preliminary results that are "reviewed" by the presenters - you don't want to look stupid if you're wrong but you want to get your ideas out there (particularly if they are interesting). I would be inclined to use such documents with extreme caution, particularly if it's recent. If you're presenting a poster or abstract on 6 patients, but your ultimate N is 60, it's very easy to get skewed results. I'd basically be inclined to use it somewhere between the preliminary results discussed in respect secondary sources and a news story and as soon as secondary sources were produced, replace it. It's somewhat of a moot point, since we're supposed to use mainly review articles, but what do you do when the first publication was in 2008? Chronic cerebrospinal venous insufficiency is the reason I ask the question incidentally, and a good test case since it's reporting results from a symposium. WLU (t) (c) Wikipedia's rules:^simple/_complex 16:18, 3 December 2009 (UTC)

I'd deal with that by stating "John Doe, of The University of Exurbs, presented some preliminary research at the 2009 ASCB meeting in San Diego which suggested that K-Ras controls p53 ubiquitination." I'd only include it if this was vital to the topic and was a real breakthrough (you'd need secondary sources for that conclusion eg newspapers or journal news articles). I would never use a source like that to state "K-Ras controls p53 ubiquitination". Tim Vickers (talk) 17:42, 3 December 2009 (UTC)

<OR> Abstracts are subjected to much more minimal peer review than publications. Generally, the authors are given a great deal of benefit-of-the-doubt that wouldn't be the case if they were submitting a manuscript for publication. This is true at the major meetings where I've submitted and reviewed abstracts, so I suspect it to be even more the case for smaller meetings. The idea behind an abstract is that the authors will present their actual data in detail at the meeting and this will stimulate interest/discussion from the meeting audience. They aren't really intended to be a final record of scientific truth. In particular, any abstract which is more than a couple of years old and has not been turned into a full-fledged publication should probably be viewed with at least gentle skepticism.</OR> Here is some interesting reading on the subject - in abstract form, no less. MastCell ^Talk 19:02, 3 December 2009 (UTC)

I'll raise your OR with some anecdotal evidence. I misspelled the organism I was working on in my first conference abstract. It was published with that glaring error uncorrected. Tim Vickers (talk) 19:29, 3 December 2009 (UTC)

I'm expecting DGG (talk · contribs) to come along any minute with some original research on abstract reliability, only his will have been published in an academic journal. (see Wikipedia talk:Reliable sources (medicine-related articles)/Archive 3#PubMed Central manuscripts vs final versions for the last time this happened) :-) Colin°^Talk 20:00, 3 December 2009 (UTC)

So there's no guidance on the subject that can help me with a current question, but is it worth drafting some? Though I'm sure we can discuss on talk pages that "X source is a conference abstract and therefore preliminary", it's always better/easier to point to something like WP:CONFERENCE. Personally I'd be quite reluctant to use an abstract, and replace it ASAP with the actual article once published. WLU (t) (c) Wikipedia's rules:^simple/_complex 21:06, 3 December 2009 (UTC)

I've added 2 sentences on this at Wikipedia:Reliable_sources_(medicine-related_articles)#Other_sources. Tim Vickers (talk) 21:42, 3 December 2009 (UTC)

FWIW, I proposed a long time ago that conference proceedings and low-impact journals be considered primary source documents that have not undergone strict review for the intents and purposes of Wikipedia. My concern was outlined here: Wikipedia:Reliable_sources/Noticeboard/Archive_9#Low impact journals used to POV-push. ScienceApologist (talk) 22:47, 3 December 2009 (UTC)

I would agree that they're primary sources, and (relative to the standards we normally expect of medical pages) fairly poor ones at that. Not that they don't have a limited place... Changes look good, even a mention to illustrate they aren't a "first choice" source is good. WLU (t) (c) Wikipedia's rules:^simple/_complex 23:42, 3 December 2009 (UTC)

I think that they ought to be considered WP:SELFPUB. Different conferences have different standards -- even multiple standards, depending on whether it's a simple poster or a keynote lecture -- but as a general rule, there's no editorial control. WhatamIdoing (talk) 23:55, 3 December 2009 (UTC)

Agree with this approach. WP:SELFPUB as a guidelines seems close to what is appropriate for the evaluation of conference abstracts and symposia, from my point of view. —Mattisse (Talk) 00:29, 4 December 2009 (UTC)

OK, I've added that link. Tim Vickers (talk) 18:03, 4 December 2009 (UTC)

Sorry to raise this thread from its deathbed, but it might be worth looking at how the Center for Integrative Medicine at M.D. Anderson Cancer Center approaches conference abstracts:

The problem with these [conference] abstracts is that they do not contain all the available details of the study methodology and data, and are not reviewed by experts in the same field (peer-reviewed) for how well the data supports the conclusions. In fact, it is not uncommon for such abstracts to be subsequently published in greater detail in peer-reviewed scientific and clinical journals with modified conclusions from the original abstract, or to be rejected for publication because the methodology or data was judged to not support the conclusions. Because of this problem with abstracts, the CIMER Web site has a policy for its content of only drawing conclusions from the most carefully reviewed information available in complete articles published in peer-reviewed journals. [51]

This mirrors the approach (and reasoning) we should be using fairly closely, and might be a useful addition (or not). Anyhow... MastCell ^Talk 21:32, 15 January 2010 (UTC)

Thanks, I gave it a shot with this edit: "; often they have no review at all and are : they are often unreviewed self-published sources, and their conclusions are often significantly changed or rejected for peer-reviewed publication." Eubulides (talk) 22:53, 15 January 2010 (UTC)

Independent eyes, please

Would someone independent please have a look at [this edit]? Prior discussion on the article's talk page pertains. Thank you,LeadSongDog come howl 20:08, 7 December 2009 (UTC)

... and also at Medicinal mushrooms. I've refrained from making any comments due to potential COI, but it would be appreciated if someone else might be able to comment on the sources and conclusions in this article. Sasata (talk) 20:32, 7 December 2009 (UTC)

You might want to move your requests to WT:MED or WP:RSN. This page is for efforts to improve the guideline itself. WhatamIdoing (talk) 20:33, 7 December 2009 (UTC)

Doing so now.LeadSongDog come howl 21:17, 7 December 2009 (UTC)

Question about MEDRS, definitions and science articles

Hello all,

I'm more and more convinced that the definitions used in MEDRS are unjustifiably limited to medical articles. The definitions found at WP:PSTS gives the impression that a peer-reviewed article on a scientific subject is a secondary source, when really if there is any debate the appropriate secondary source would be a review article. Is there any discussion elsewhere that points to this? For a celebutard's personal life, a news article is appropriate, but for a discussion of the impact of different drivers on human bipedalism, a review article would seem a better source than a journal article - where it is easy to cherry-pick sources to provide the illusion of being a respected hypothesis. Any wikilinks to policies or guidelines that would be helpful to me? WLU (t) (c) Wikipedia's rules:^simple/_complex 13:14, 21 January 2010 (UTC)

I agree that we should seek to extend the MEDRS definitions (or something similar) to all science articles, perhaps as WP:SCIRS. As this is certainly going to be controversial amongst certain crowds, care should be taken to broach the topic in an unimpeachable way in order to avoid being interminably bound up in wikilawyering. I'd suggest that WP:Centralized discussion is the place to discuss it and that Wikipedia talk:WikiProject Science, Wikipedia talk:Reliable sources, Wikipedia talk:PSTS and Wikipedia:Village pump (policy) should all be notified of the discussion. Before doing so, you may want to draft a strawman in your userspace. You may find it helpful to read WP:Scientific_citation_guidelines, WP:Fringe, the mainspace article Scientific consensus, and their talkpages. LeadSongDog come howl 19:08, 21 January 2010 (UTC)

Wikipedia:Scientific standards was attempted at one point. Please feel free to revive it. ScienceApologist (talk) 20:02, 21 January 2010 (UTC)

WLU, the sort of review articles that you're talking about are journal articles: they're just not the only possible kind of journal articles. Have you asked the folks at PSTS to address this? WhatamIdoing (talk) 20:19, 21 January 2010 (UTC)

I have tried to follow up at this by making edits to WP:PSTS that explicitly mention original research papers as examples of primary sources, and review articles as secondary. I've also followed up with discussion at WT:NOR #Deletion of examples of primary sources from PSTS. I don't know if these changes will stick (I'm no expert on the arcane points of Wikipedia policy). Eubulides (talk) 21:02, 21 January 2010 (UTC)

You'd be surprised how inexperienced and wrong many contributors to policy talk page discussions can be. Please, the more of our brightest and most experienced medical editors who offer help tweak and keep an eye on those pages, the better. It can be a hard battle but if we let folk undermine our policies upon which our guidelines are built, then everything falls apart.

I would like to add one point: PST should be judged not just by what type of source it is, but by what facts are being drawn from it. For example, any source can be a primary source for some piece of information (like the fact that so-and-so published such-and-such and when). Any discussion on PSTS that fails to acknowledge that is being too simplistic IMO. Colin°^Talk 22:31, 21 January 2010 (UTC)

Thanks for the many replies and wikilinks - I'll read through them. My apologies for not responding to all the posts on this one, I got caught up on editing a couple articles and this completely slipped my mind.

WAID, I agree that review articles are journal articles, but the guidance provided here about review articles, textbook chapters and summaries by official bodies taking precedence over single pieces of research was a powerful revelation to me as an editor and struck me as being the best way to ensure DUE weight. I haven't asked anyone at PST to respond, and I should have, thanks for the idea.

Colin, I also agree. However, there's a vast gulf between "X said Y about Z" and "There is general agreement that A is the cause of B". Like reporters "balancing" views on vaccination with notorious deniers and celebrities, by reporting "X said Y about Z" it gives significant weight to the idea that X is an expert on Z and that Y is true. For new editors (and particularly) for POV-pushers, cherry picking (particularly in fringe areas) can easily lead to Medical Hypotheses being cited as a source about AIDS denialism. More explicit guidance on what a secondary source is and why it takes precedence would be quite helpful. This whole thing was brought about by a civil POV-pusher on bipedalism trying to force the idea that the 'wading hypothesis' (a fringe with the fringe theory of the aquatic ape hypothesis) is a serious contender to explain bipedalism. Single sources were cited to support this across multiple years, but realistically the theory was not considered serious or useful by paleoanthropologists (though possibly changing PMID 19890871). So, is citing single sources saying "X said Y about Z" a good idea in this case, or an example of undue weight on an unproven, unaccepted hypothesis? In my mind an emphasis on secondary (i.e. review) sources in this case would neatly solve this problem. In this case I would say the facts were accurate, but still disproportionate. Perhaps this is a point to be made at WP:UNDUE rather than WP:RS... WLU (t) (c) Wikipedia's rules:^simple/_complex 13:55, 25 January 2010 (UTC)

I am likewise concerned about the application of MEDRS but for the converse reason, and that is the editing policy of some editors to classify article X as a "medical article" and therefore treat any information as falling under MEDRS at their convenience. As far as I can see the scope definition for MEDRS is given in the opening two sentences: "Wikipedia's articles, while not intended to provide medical advice, are nonetheless an important and widely used source of health information. Therefore, it is vital that biomedical information in articles be based on reliable published sources and accurately reflect current medical knowledge." To me, a reasonable interpretation of this would be:

1. It applies to any Wikipedia article which includes biomedical information, however
2. It only applies to the accuracy of that biomedical information. Elsewhere straight WP:RS applies.

So for example the statement that "Actor X is HIV positive" is making no medical claims about HIV, so MEDRS does not apply here (though there is a host of other policies and guidelines which do govern the inclusion of such content). If this is not the case then we need to tighten up the guideline wording. I am not a medical expert, but I do expect that MEDRS be sufficiently clear in its scope that even any competent editor can sensibly interpret it. -- TerryE (talk) 00:09, 26 February 2010 (UTC)

Yes, the intent of the opening sentence is to apply to the biomedical information in an article, independently of whether the article's topic is itself biomedical. Eubulides (talk) 00:27, 26 February 2010 (UTC)

Moreover, even if the the article's topic is itself broadly biomedical, it shouldn't be applied to remove information that is not MEDRS but is valid for the article. For example one might argue that a specialist subject such HIV-1 is entirely MEDRS in scope, but the AIDS covers history and social aspects which are clearly outside MEDRS. One the other hand in the case of Chronic Fatigue Syndrome which is also both an medical subject and also has social and other aspects, editors in the past have successfully argued to exclude RS material because it falls outside MEDRS. Of course we should exclude biomedical on MEDRS criteria, but for non-biomedical content RS and not MEDRS should apply. -- TerryE (talk) 01:46, 26 February 2010 (UTC)

Sorry, but I'm afraid that things are not as black-and-white as the previous comment suggests, as it's not always the case that "material" is clearly biomedical or clearly non-biomedical. Furthermore, in controversial areas like CFS and AIDS it's even more important than usual to employ high-quality sources, and the advice given in WP:MEDRS is particularly good advice to use in such areas. For the history of AIDS, for example, the question of where AIDS started and how it spread is clearly biomedical, as are many social aspects of AIDS involving transmission of disease. Eubulides (talk) 04:15, 26 February 2010 (UTC)

Clearly we don't live in a world of black and white, but the whole purpose of having policies and guidelines is to help us move towards more objective discriminants. For example, WP:V states that in the case of submitting content "The burden of evidence lies with the editor who adds or restores material". What I hate is when some editor justifies that his revert or whatever is "because of MEDRS", but when you ask a perfectly reasonable Q such as "what aspect of this content breaches what specific aspect of MEDRS", the editor doesn't give a direct reply and instead just repeats "it's MEDRS silly" or words to that effect. This whole idea of having a trump card up your sleeve to drag out when convenient without explanation doesn't work for me. In such grey areas, the editor citing MEDRS must be willing to define what elements of the content fall under MEDRS and also which specific aspects of MEDRS are breached. Otherwise it isn't a sound edit and it therefore doesn't deserve to stand. -- TerryE (talk) 15:32, 26 February 2010 (UTC)

We expect editors to use their best judgment, and to apply guidelines like this one with 'common sense and the occasional exception.'

Have you read the last half of the section on popular press? It gives several examples of information that are often not best sourced to the peer-reviewed scientific literature. WhatamIdoing (talk) 20:59, 26 February 2010 (UTC)

Medicine or medical?

This guideline appears to be about medical articles, but it's titled "medicine". A Quest For Knowledge (talk) 17:05, 21 March 2010 (UTC)

No, it's not. It's title is "medicine-related articles". There is a difference between a 'medicine article' and a medicine-related article'. WhatamIdoing (talk) 22:30, 21 March 2010 (UTC)

Assessing the evidence

We talked about this a while ago and never got very far. My bold change rearranges a bit, but it's the first and last paragraphs that are (mostly) new.

I didn't get very far with wordsmithing, so if you've got any ideas about how to improve it, please go ahead. WhatamIdoing (talk) 00:32, 27 March 2010 (UTC)

MoS naming style

There is currently an ongoing discussion about the future of this and others MoS naming style. Please consider the issues raised in the discussion and vote if you wish GnevinAWB (talk) 21:00, 25 April 2010 (UTC)