Wikipedia talk:Reliable sources/Archive 24

This is an archive of past discussions on Wikipedia:Reliable sources. Do not edit the contents of this page. If you wish to start a new discussion or revive an old one, please do so on the current talk page.

Archive 20

←

Archive 22

→

Different matter relating to news

The project page says, "Some news organizations have used Wikipedia articles as the sole source for their work." This statement is not sufficiently attributed, and I am going to change it. Maurreen (talk) 04:41, 9 March 2010 (UTC)

To what? Gigs (talk) 04:42, 9 March 2010 (UTC)

I changed it from "... as the sole source ..." to "... as a source ..." Disclosure: I am a journalist. Maurreen (talk) 04:44, 9 March 2010 (UTC)

A reasonable change. Gigs (talk) 04:46, 9 March 2010 (UTC)

This is not an article, and WP:V does not apply. See WP:POLICY#Not_part_of_the_encyclopedia.

It is certainly true that some journalists have used Wikipedia as their sole source for specific facts. Sometimes they even attribute Wikipedia. I've seen sentences like "According to Wikipedia, ____ is a disease that..." in dead-tree newspaper articles. WhatamIdoing (talk) 02:30, 13 March 2010 (UTC)

Regardless of what applies, we should not mislead.

If you want it to say, "sole source for specific facts" that is different from "sole source for their work." Maurreen (talk) 02:45, 13 March 2010 (UTC)

Scholarship section (2.1) - does sources = journals?

Based on my reading of the following para

The scholarly acceptance of a source can be verified by confirming that the source has entered mainstream academic discourse, for example by checking the number of scholarly citations it has received in citation indexes. A corollary is that journals not included in a citation index, especially in fields well covered by such indexes, should be used with caution, though whether it is appropriate to use will depend on the context.

the 6th word in the first line, "source" refers to sources of papers and not the peer reviewed papers that journals publish. This is under debate over at Talk:Global Warming with a FAQ (F22) proposed to rely on the understanding that source can mean peer reviewed paper. The larger issue under discussion is the treatment of papers, when they can be included and when they should be excluded. My interpretation serves the inclusionist side and the source=papers interpretation serves the exclusionist side. It would be nice to get a wider perspective that does not necessarily even care about the parochial issues but rather the integrity of WP:RS itself.

Help. TMLutas (talk) 20:01, 6 January 2010 (UTC)

I tried to look at the discussion, but there is no explanation of what "FAQ (F22)" is. Please provide a link.

As for your question, I would think that any academic journal that is frequently cited in other recognized journals is reliable in general, and any peer-reviewed paper published in such a journal is, in the absence of other evidence, a reliable source. However, that does not mean it is the best source to use in any particular article. Also, other publications, such as books, videos, and computer databases, could be vetted as reliable sources through frequent citation in recognized journals. --Jc3s5h (talk) 21:24, 6 January 2010 (UTC)

Relevant links and timeline - On 22 December I started a discussion on Talk:Global cooling, proposing that we finally have a section covering advocacy for the theory in the 2000s. That section is here. A persistent problem developed that people would argue against the inclusion of a paper supported by various rules references that on examination did not support what the editor asserted. A number of editors refused to justify their opposition at all tying it to a rule, policy, or guideline. Finally on 17:35, 1 January 2010 I was referred to Talk:Global warming/FAQ Q22. This FAQ, it turned out, was just naked assertion with no reference back to any sort of policy and had been added on 30 December with no preceding discussion in Talk:Global Warming. I attempted to apply various improvements and found that it needed a talk to consensus so I created an appropriate section. It's clear that a local consensus exists among many regulars on the global warming page but, again, nobody can seem to justify the local practices as conforming to general rules. At the top of the section, I am keeping a running list of policies, guidelines, and essays cited as relevant to the question

policy

WP:NOTNEWS

WP:NOTTEXTBOOK

WP:WEIGHT

WP:NPOV

content guideline

WP:RS

WP:FRINGE

WP:TOOLONG

essay

WP:DEADLINE

WP:RECENT

This page, specifically the section cited above is the latest attempt to reconcile local practice with actual rules. The local practice is to exclude peer reviewed, published articles for a vague and indeterminate amount of time (I've never gotten a specific time period) until their "impact" can be determined. Studies with "impact" can be cited. Those that do not cannot be cited irrespective of the balance rules as specified in WP:WEIGHT. The nail they are trying to hang their hat on rests on "source" being a paper in 2.1(4) and not a journal. Given the context of section 2.1 I think that it's simply a mistaken interpretation but thought the WP:RS community should have an opportunity to weigh in on what has become a pretty contentious issue. TMLutas (talk) 22:48, 6 January 2010 (UTC)

But we cannot and should not cite very study in an active field like climate change. The problem is that climate change skeptics cherry-pick studies that support their position, and cite studies from dodgy journals like E&E. Reviews should be preferred to primary research studies. Fences&Windows 02:39, 7 January 2010 (UTC)

This behavior is covered by WP:WEIGHT. Take the global warming article.It's 98k and with the addition of just 3k more it would hit mandatory split territory. Under the source=journal interpretation (barring some other undiscovered policy) you could put in a cherry picked study but you would need to put in enough balancing text to make clear exactly how cherry picked it is so nobody is deceived. To satisfy WP:TOOLONG you would also have to pull out so much text in order to do that balancing that in the vast number of cases of skeptic cherry picking the article would lose more than it would gain and could be excluded on those grounds. But there are related pages where the study might be profitably put to use. Let's say there's a paper asserting global cooling (there's one in press right now as it happens). You could try to get it in the global warming article but it would be rejected for the reasons I outlined above. But global cooling both is much shorter (32k) and has zero coverage of global cooling assertions post 2001 so it isn't like there's no room to add it. And the weighting requirements would be different under WP:WEIGHT for a minority view page like global cooling. The study could go into Wikipedia in the global cooling page and a reasonable one or two line reference to global cooling could be in the global warming article. This would resolve the issue of unfair exclusion from Wikipedia without any special pleading that climatology articles are somehow different than any other topic out there. Yet the exact same arguments under the source=papers interpretation are used currently in both global warming and global cooling talk pages to keep the study out. This leads to conclusions of conspiracy and a great temptation to sockpuppet against the conspiracy to "get the truth out". Nobody needs that.

It's a straw man to raise the idea of citing every study. For every idea, multiple studies should be winnowed down to the best representation of the idea. But if you misunderstand how big the idea bucket should be, conflict is inevitable. There is no one idea bucket called global warming. There is one for AGW caused by CO2, AGW caused by ozone holes, GW by cosmic rays, GW by end stage funkiness of our current interglacial, etc. If you pick the best 1-5 papers representing each idea, you won't suffer overload and if you weight correctly, nobody's going to get the wrong idea that a minority opinion has greater support than it does. As pages get too large, split out ideas into their own pages and have a small summary on the main page. This is standard Wikipedia editing. No need for special pleading.

The question that I have for *this* page is whether it's special pleading. Does source=paper or source=journal? The current formulation seems as troublesome as the US' 2nd amendment. Fortunately for us, fixing the ambiguity is easier. TMLutas (talk) 15:21, 7 January 2010 (UTC)

Source = Paper and Journal, and even Author... when examining a source for reliability, all three need to be examined. Blueboar (talk) 15:39, 7 January 2010 (UTC)

I agree that an examination of all factors should enter into the general concept of reliable source. The practice discussed in section 2.1(4) is a bit more specific then examining papers, journals and even authors for reliability. It seems to be an abbreviated restatement of impact factor. Now that's fine as far as it goes but the impact factor article seems to specifically caution against using impact factor to judge anything other than journals. The relevant section is title Misuse:

The impact factor is often misused to evaluate the importance of an individual publication or evaluate an individual researcher.^[1] This does not work well since a small number of publications are cited much more than the majority - for example, about 90% of Nature's 2004 impact factor was based on only a quarter of its publications, and thus the importance of any one publication will be different from, and in most cases less than, the overall number.^[2] The impact factor, however, averages over all articles and thus underestimates the citations of the most cited articles while exaggerating the number of citations of the majority of articles. Consequently, the Higher Education Funding Council for England was urged by the House of Commons Science and Technology Select Committee to remind Research Assessment Exercise panels that they are obliged to assess the quality of the content of individual articles, not the reputation of the journal in which they are published.^[3]

There are scholarly examinations out there also labeling impact factor in individual papers and authors as misuse. Apparently the system can be manipulated and has been once it became clear that promotions and journalistic reputation depended on impact factor ratings.

I'm still left with questions, though things are coming into sharper focus. What is the stance of WP:RS regarding impact factor rating individual papers and authors as a method of deciding whether they shall be included in Wikipedia. Is impact factor examination to be limited to journals (source=journal) or have wider application (source=paper)? Is this well settled ground or is it something that the editors concentrating on WP:RS really haven't thought much about? TMLutas (talk) 01:30, 8 January 2010 (UTC)

This section seems to have drifted a bit as I took my wiki break. As I see it, there is enough information to answer the question definitively. Is a paper published in a reliable source journal considered reliable by virtue of its peer review in that publication or does it need to have a certain number of citations itself in the literature before it is considered reliable? TMLutas (talk) 20:10, 15 January 2010 (UTC)

By default I'd accept that as a RS, but that can depend on circumstances - if the accuracy of that paper is later challenged by another suitably credible source, its status as an RS needs to be re-evaluated. --GenericBob (talk) 23:23, 15 January 2010 (UTC)

I accept that as being good common sense. I don't think that the text of 2.1 explicitly supports that position at present and some editors do disagree (I raised the question specifically because there was a conflict elsewhere). Would the following be acceptable as a restatement of this position that could go into the article? "A scientific paper published in a peer reviewed journal that is considered reliable is, by default, considered reliable. This default reliability is weak and may be successfully challenged by subsequent peer reviewed papers." Or should this proposal be broken out and put into its own section so that we're more likely to have a full airing out of the issues and a better overall text? TMLutas (talk) 04:11, 24 January 2010 (UTC)

The default is not weak. A scientific paper published in a peer reviewed journal is comparable with, maybe better than, stories in newspapers and magazines, and books in the trade press. --Jc3s5h (talk) 04:18, 24 January 2010 (UTC)

I included the word weak in deference to others who are asserting elsewhere that there should be no finding of reliability until a separate hurdle of "impact" or "impact factor" is passed. I've no objection to dropping it but I'm a bit unclear as to how to reconcile your position with the dominant crowd over on pages like global warming. TMLutas (talk) 03:46, 28 January 2010 (UTC)

I should make it clear that my personal opinion is that they're just wrong and misapplying WP:RS. I just don't know how to go through the process to get them to stop misapplying it. 3-4 people talking things out on WP:RS is not going to cut it. I don't know what would short of significant sanctions. TMLutas (talk) 04:23, 28 January 2010 (UTC)

If we have a topic, like global warming, that has vast numbers of sources to choose from, it might be best to use concepts like impact factor to choose among the sources. But if we say that any article on any topic must have the same degree of acceptance as the sources we pick for global warming, there would be many topics we just couldn't write about at all, because there just are not as many sources to choose from. --Jc3s5h (talk) 04:42, 28 January 2010 (UTC)

I think I might not have been clear. What's going on isn't a winnowing of, let's say 100 ice studies all saying similar things to the 1 best representative one using impact factor. Such a use would be, and is, uncontroversial. What impact factor is controversially being used for is to delay and deny publication of minority viewpoints which might have 1-5 papers supporting them. You end up having the russian solar scientists who are going around claiming global cooling is here or coming (including two of whom have a public bet running on the subject) entirely locked out of the global cooling article. There's a new theory linking some novel discovery on ozone depletion in the polar zones to our recent bout of warming and claiming that we're likely to have a few decades of cooling due to more ozone at the poles. There are legitimate weighting issues to address with minority views but that's not the trouble. The cures outlined in WP:WEIGHT are available.

These papers' reliability is being questioned regardless of their publication status in an RS peer reviewed journal based on this impact factor even though it is a viewpoint that's getting knocked, not just a particular paper expressing the viewpoint. It's sort of like knocking out punctuated equilibrium evolution on the grounds that so many more papers were steady progress style darwinian (or vice versa). I'd have little problem with minority papers being relegated to lesser read support articles (like global cooling is for global warming) due to weight issues but to entirely exclude viewpoints in any article is problematic.

It's also a problem in a workflow sense. If you read about a new paper in this "impact factor" style of doing things, you try to get it in, get it knocked down by the impact factor guardians, and have to come back 6-9 months later to fight the whole battle over again. Unless you are *very* committed to the edit, it's just not going to happen. In a non-impact factor world, you get your paper in and if it's debunked, the paper gets removed a couple of months later as it's successfully shown to be bad science. That flows better, you get more edits and your article stays up to date on developments without acrimonious arguments about the judgments of scientists in their citation behavior. Wikipedia also gets to stay clear of the whole mess about conspiracies run out of East Anglia to game the impact factor ratings to punish those not pushing the AGW viewpoint. TMLutas (talk) 19:19, 29 January 2010 (UTC)

I happened to notice this discussion because it pinged my watchlist when Jc3s5h linked to this discussion from Talk:Global_warming/FAQ Q22. Thanks.

The specific problem in global warming is that there is an immense pressure to inflate the scientific significance of the very few papers that point in a direction opposite to the mainstream. The FAQ Q22, of which I wrote the original wording, just explains why we don't add every such new paper to the article on global warming. There's absolutely nothing wrong with inclusion of material questioning global warming; it's just that we shouldn't give undue attention to such material. There is a problem of recentism, really. Global warming is such a politically important field that it has been subject to a "refutation of the month" syndrome in the popular press. Allowing a paper to mellow for a few months helps us to sort out the wheat from the chaff.

I think the current version is reasonable. It says:

Q22: What about this really interesting recent peer reviewed paper I read or read about, that says...?

A22: There are many peer-reviewed papers published every month in scientific journals such as Geophysical Research Letters,[31] the Journal of Climate[32] and others. We can't include all of them, so we wait to see if papers have significant impact. Wikipedia's guideline on reliable sources states The scholarly acceptance of a source can be verified by confirming that the source has entered mainstream academic discourse, for example by checking the number of scholarly citations it has received in citation indexes. Brand-new papers will have accumulated few if any such citations, so we don't ordinarily base our writing on very recent works.

By counting how many academic sources cite a paper after some time we have concrete evidence as to the significance of the paper. Prior to that we really don't know. Adding references randomly on publication would lead to a very messy article. --TS 21:41, 3 February 2010 (UTC)

I don't have any opinion on any particular global warming paper. But this Reliable Sources guideline is quoted as if it serves to justify excluding a paper because the individual paper has not been cited often enough. The undisputed consensus in this talk page discussion is that, by default, a paper may be considered reliable if it is printed in a journal, and the journal has been frequently cited by other journals, books, etc.

Now, if the FAQ were to say something like "The process used to judge the reliability of a journal in WP:RS, that is, the frequency with which the journal is sited in other sources, must be made stricter in the case of global warming because..." that would be OK with me. --Jc3s5h (talk) 22:46, 3 February 2010 (UTC)

Obviously an academic paper that isn't widely cited may not make the cut. Am I missing some conceivable reading of this guideline that would mandate the inclusion of a paper that was, essentially, ignored by the academic community?

Anybody who claims that a single peer reviewed academic paper is intrinsically "reliable" is making a very controversial statement. If you claim, extraordinarily and to be honest a little shockingly, that there is any "undisputed consensus" to the contrary in academia or on Wikipedia or in this discussion I say "thus I refute you." A single scientific paper is not equal to a fact. --TS 00:00, 4 February 2010 (UTC)

In the context of the Reliable sources guideline, "reliable" means good enough to cite in an article. Sources that are not reliable are so bad they shouldn't even be mentioned in articles. Publication in a respected journal makes it acceptable to cite it on Wikipedia, but the way the information from the article is presented would depend on all the available information. Also, not all subject matter gets as much attention as global warming. Obscure subject matter will not receive as many citations, and the literature on obscure subjects might not be well-enough indexed to find any cites that do exist.

If one were to compare it to a trial, a reliable source would be analogous to admissible evidence, and other sources would be analogous to inadmissible evidence. Naturally one piece of evidence does not automatically determine the verdict. --Jc3s5h (talk) 00:10, 4 February 2010 (UTC)

I agree with the above. I think this is more a matter of due weight. There's a vast preponderance of scientific opinion that global warming is a real problem, and this opinion is based on the weight of evidence. There are occasional outliers in the academic press, which gain news value, but to be honest mostly blog value because of their exciting, grassy knoll attraction. --TS 00:32, 4 February 2010 (UTC)

I proposed due weight as an alternative to relying on WP:RS a position which was rejected on global warming in the discussion. The 'problem' with my alternative is that it would allow for inclusion of these papers but shifted to specialty pages, such as global cooling where there is space to properly weight them and where WP:WEIGHT specifically changes the amount of weighting necessary. This doesn't sit right with some who would prefer to exclude such papers altogether. And, in fact, the whole conversation started with exactly this sort of exclusion on global cooling, a page that weighs in @ ~32k and could easily accommodate an extensive discussion of current global cooling advocates (they exist out there) without confusing the issue that global cooling itself is only believed by a small minority. In order to avoid WP:WEIGHT and its inclusive solution, the majority is trying to fit WP:RS to do the job but the square peg won't fit in the round hole. I prefer to fix this by changing 2.1 to make explicit the consensus referred to by Jc3s5h instead of the alternative, which is to call for sanctions for persistently misapplying WP:RS. I think the clarification on this page route will be less emotional, more likely to lead to a smoother road going ahead. TMLutas (talk) 04:20, 7 February 2010 (UTC)

WP:UNDUE is never an alternative to relying on WP:RS... both must be adheared to at the same time. 04:24, 7 February 2010 (UTC)

BlueBoar, you mistake my meaning. The problem set out in the FAQ is that a new paper comes out and someone wants to include it while their is resistance. This happens a lot, especially with papers that don't quite fit the AGW narrative. In a huge article like global warming you have to prune at least as much as you include or the whole article goes over size. What should be done with these papers and, when they are excluded, by what policy/guideline/rule should they be excluded from that particular article? In that sense, WP:RS and WP:UNDUE are very much competitors because it is possible to pass one without the other. What ended up happening in that debate is that a sort of special pleading went on that being published in a reliable source journal was no longer sufficient to be considered reliable. WP:RS was being modified in practice without any change on the page by adopting a strained interpretation of what a "source" meant as per WP:RS section 2.1 para 4. Thus, my attempt to gain consensus for clarifying language so that this strained interpretation can no longer be viably held. Alternatively, I can seek to sanction the lot of them but I think that is excessive. What's your opinion on a proper resolution to the situation? Keep in mind that we're not just talking about global warming but a whole raft of climate papers because if WP:UNDUE is selected the papers can be included in specialist pages elsewhere (as per WP:UNDUE itself) while if WP:RS is the rationale chosen they cannot. TMLutas (talk) 21:20, 7 February 2010 (UTC)

I think that we are at a consensus here. I'm going to make an edit based on it. TMLutas (talk) 21:42, 20 February 2010 (UTC)

done TMLutas (talk) 21:46, 20 February 2010 (UTC)

Your edit did not reflect the consensus above. I suggest you file a formal rfc. Hipocrite (talk) 17:59, 13 March 2010 (UTC)

I actually don't know how to do that. I don't think I've ever had to. I'm leaving the version you changed on 20:09, 11 March 2010 up for now. You did not make the most basic attempt to find consensus before you modified the text and you started an edit war after I reverted you and sent you to discuss to consensus before you modify this page. One rule for me (a month and a half of talk is not enough) and another rule for thee (don't even bother with the talk page before changing a widely referred to rule) is not appropriate behavior. TMLutas (talk) 05:58, 14 March 2010 (UTC)

What makes a review site a reliable source?

Review sites such as Ain't It Cool, are often used as a reliable source. How is this determined though? If your website is mentioned in a newspaper, magazine, or some other source, does that make it notable enough to be used for reviewing movies, games, comics, or whatever? Many reliable sources do not meet the notability standards of Wikipedia necessary to have an article for them, but are still used, so being notable is not a requirement to be considered a reliable source.

I'm thinking just as a newspaper that had a hundred thousand subscribers would be considered a reliable source for reviews, while some local newspaper in a small town that had only a few hundred readers would not be, so it should be with websites. Its about the number of readers. If you have a significant number of people going to a review site, that exists specifically to reviews one genre or another, then it should be considered a reliable source and notable enough to be quoted in articles about whatever it is reviewing. http://www.alexa.com/ is the only site I know of that can gauge a website's traffic, but surely there are others. If there is no doubt that a significant number(exact number to be determined by consensus), go to a website for reviews, should it be considered? Obviously you can't trust a website to truthfully tell you their hits, so other sources would have to be found.

Also, can you judge a review site for accuracy, when all reviewers are just giving their own opinions on something? If all you were doing was quoting the review source in an article, then would accuracy be a legitimate argument against it at all? Reliable sources are judged by accuracy, and that an argument that came up in a recent discussion[1], so I was curious about that. The review is accurately giving someone's opinion of what they are reviewing, not listing anything in doubt. Dream Focus 21:39, 12 January 2010 (UTC)

Alexa is a remarkably iffy source for much. The criterion has been that if there is "editorial control" over the reviews (that is, the site is not in any sense an open blog for reviewers) that where the reviewer is otherwise notable as a reviewer, that their opinions in the reveiw could be used. In other words, the strange fellow who came up with great blurbs for every movies known to man would not count <g>. There is, moreover, no such thing as "accuracy" when dealing with reviews. Collect (talk) 22:02, 12 January 2010 (UTC)

Note: See Wikipedia talk:WikiProject Anime and manga‎#Reliable source Animetric and Wikipedia:Articles for deletion/Blood royale (hentai) for previous discussions on this subject, specifically relating to using reviews from the self-published website Animetric.com as a bases to determine notability. Essentially, Dream Focus is forum shopping this topic since he is not getting the results he is wanting. —Farix (t | c) 22:43, 12 January 2010 (UTC)

I already had a link in my post to the Wikiproject discussion. I am not forum shopping. In that discussion, I was pointed here, and told that is where the decision is made, so I came here. And this has nothing to do with that one article. In many AFDs, horror films only get reviewed on certain review sites, which some argue are not a reliable source. The same argument comes up time and again, so it needs to be dealt with here. Dream Focus 23:57, 12 January 2010 (UTC)

What is far too often ignored is that WP:RS instructs that sorces be considered for what they are in context to what is being sourced... mostly because RS shares (not mandates) a "rule of thumb" that offers "As a rule of thumb, the more people engaged in checking facts, analyzing legal issues, and scrutinizing the writing, the more reliable the publication." But that would be for facts... not opinions. Opinions may be based on facts but they are POV.

This rule of thumb suggests ways to determine reliabilty but does not mandate that all sources must positively meet that "rule of thumb". Further, it stipulates "Reliable sources may therefore be published materials with a reliable publication process; they may be authors who are regarded as authoritative in relation to the subject in question; or they may be both." That's "may"... not MUST. The most clarifying portion of that guideline, the very sentence preceding that "rule of thumb" and the portion most often ignored, is "How reliable a source is, and the basis of its reliability, depends on the context". (my emphasis)

This should properly be seen as indicative that sources should be considered on a "sliding scale of reliability" dependent upon context and just what is being sourced. For politics, I might give the Washington Post a 10 and Fangoria a zero. For mainstream films I might give Variety a 10 and New York Times a 9. For horror films, I'd give Fangoria or DVD Talk or Film Threat or Bloody-Disgusting or FEARnet or Rotten Tomatoes or Rue Morgue (magazine) a 10 and Washington Post a -5. This is all as guideline specifically instructs and allows... a source's reliability should be determined in context to what is being sourced... and WP:RS's suggested "rule of thumb" shows how some sources may be more reliable than others for different reasons and for different topics. All indicative that even guideline understands that different sources must be determined in different ways for sourcing different information.

What should perhaps be considered through the "common sense" caveat that heads each guideline is to include here a section that better defines this sliding scale of suitability for various topics. Schmidt, ^{MICHAEL Q.} 03:58, 13 January 2010 (UTC)

just as a generic aside (which I'm sure some will disagree with, but which I still think is useful) 'reliability' is a measure of confidence - a source is reliable when people generally have confidence that what it says is true. That has specific meanings in scientific studies, but in more real-world situations it's looser. number of subscribers is one measure of how confident readers are in pronouncements of a website, but I'm not sure it's the best - number of hits might wrk better. --Ludwigs2 07:23, 13 January 2010 (UTC)

I disagree. For one, hit count has a lot more to do with entertainment value than perceived accuracy. For another, even perceived accuracy is influenced by a lot of things besides the actual reliability of a source. A lot of people trust Michael Moore and (not the same people) Rush Limbaugh, not because either of them make a habit of careful fact-checking, but because they support the things people want to believe. --GenericBob (talk) 08:49, 13 January 2010 (UTC)

One problem is the circularity of the situation: Many readers want to know popular opinion: to get that opinion they go to popular sites; at that point they tend to adopt the popular opinions themselves, which in turn makes the popular opinion and the site more popular. It's analogous to the TV celebs who are "famous for being famous". But those TV celebs are riding a "bubble" of popularity. Travel back 40 years ... many of those "famous" people are largely unknown, today. I was watching a film from 70 years ago ... there was something peculiar about an actress' performance ... I checked who she was ... a well-known socialite of the day, playing herself! The "Wikipedia of 1930" would have a big article on her, with many references explaining how important she was. But would "Wikipedia circa 1950" use the same references? Which years would be the "reliable" source? Therein lies the problem: Wikipedia is intended to be an unbiased reference based on solid information. Fans and general readers want the latest perceptions. How is this resolved? I'm not sure it can be. Personally, I think Wikipedia should get out of the "review business" for anything that's released in the last 10 years. Leave it to Metacritic and the like to document each of the dozens of expressed opinions. Regards, Piano non troppo (talk) 12:02, 13 January 2010 (UTC)

Unbiased from Wikipedi'a view, naturally... as even established reviewers such as Roger Ebert offer only their own personal POV colored with lots of POV hyperbole. Being personal POV does not make their opinions right or wrong... only opinion... and as such does not matter much, as the threshold for inclusion in Wikipedia is verifiability, not truth... IE: what IS required ubner WP:V is "did a person or website being quoted actually state what is being asserted, and can that assertion be verified back to that source?" If popular websites offer opinions and those opinions are sought by the public, what is required here is proper attribution. Wikipedia certainly hopes and expects that readers can weigh sources for what they are and what they offer. For films being considered for notability, reviews may offer the coverage suggested by WP:N, and the confirmation mandated by WP:V. As I wrote above, the level of "reliability" has to do with context. And of course, Wikipedia can be and offer much more than its paper predecessors, and so should aspire to not just match them... but to exceed them in both scope and utility. Schmidt, ^{MICHAEL Q.} 00:04, 14 January 2010 (UTC)

Ok, but ... this just hit me yesterday ... How many reviewers does an article need? Supposing there are 100 reliable reviewers? What's the frigging purpose of quoting reviews? And ... I have an answer: It's a way of avoiding Wikipedia's restriction on original research. I.e., we're warding off every Tom, Dick and Jane expressing their personal opinion, but at the expense of pandering to professional critics. Guess I'm gravitating toward removing all reviews as subjective.... Piano non troppo (talk) 02:42, 14 January 2010 (UTC)

See WP:DUE for the answer to that question. Wtmitchell (talk) (earlier Boracay Bill) 03:15, 14 January 2010 (UTC)

<--I can't help but think that some issues are being conflated here. If, say, a movie review website is deemed a reliable source, then that means that we believe that the critic's opinion, as it is uttered on the site, is indeed the critic's opinion. But in many ways, that's trivial, since we're asking about someone's opinion, not about what the President said in a private conversation with their chief of staff, and I don't think we would doubt that person A said "Witness is the greatest movie ever", unless person A is not the reviewer writing the review. If some site reports that Neil Gaiman said Witness is the greatest movie ever," then we have cause to ask about reliability.

But it seems to me that what's at stake here is notability: if a couple of sites which are deemed reliable write a review on some relatively unknown movie, then those reviews confer notability--isn't that really what this is about? This is what MQS explicated above, if I read him correctly. My personal opinion, if anyone cares, is that the criteria for reliability ought to be stringent, and if that means that some sites are out and therefore some reviews cannot confer notability and therefore some articles have to be deleted...well, I may well be that kind of deletionist. And I think that MQS has a good point, in regards to the contextual determination of reliability--but a reliable source, in my opinion, doesn't have to be notable, and can be reliable without conferring notability. If all the fanzines in the world agree that Witness is the greatest movie ever, but none of the "bigger" publications hold such an opinion, then it's not a very valuable or notable opinion.

BTW, this conversation began with a misleading statement about bigger and smaller newspapers: small-town newspapers aren't unreliable (I think most here would agree), but they simply don't confer notability the way the NYT does. If my book is reviewed by the Times, I am pretty much automatically notable. If my book is reviewed by the Tuscaloosa News, I am not. DreamFocus, that's a different kettle of fish: that's conflating reliability and notability. Drmies (talk) 16:31, 14 January 2010 (UTC)

Is there consensus to change the page, to state that review sites are reliable sources and can be used, if they have conformable number of hits? Many articles are deleted if there is not a "Reception" section, listing what reviewers thought about it, many not considering it notable without one, and sending it to AFD. So we need to determine which sites are reliable sources by a clear set standard. Alexa shows ranking, but not actual numbers. Quantcast.com shows actual page hits a website gets. http://www.quantcast.com/themanime.org If it has at least 20,000 hits for any month, can we declare it a reliable source for reviews? Any objections towards adding that somewhere on the page? Dream Focus 16:21, 14 January 2010 (UTC)

No, there is no such consensus. Hits are not an indication of reliability at all, they are the most ridiculously easily to inflate numbers in the world with no effort at all, as any web developer can tell you. Further, sites like Encyclopedia Dramatica get a ton of hits, but they certainly is not a reliable source for anything, including opinions. -- AnmaFinotera (talk · contribs) 16:24, 14 January 2010 (UTC)

Dream Focus, you've been told several times that the number of people reading self-published reviews do not make the review reliable. It also doesn't make something notable either, despite your repeated claims to the contrary. Reliability of self-published sources are based on the criteria outlined at WP:SPS. You have to show that a reviewer is an established expert on the topic whose work in the relevant field has previously been published by reliable third-party publications. This is the criteria that WP:ANIME has used to determine which reviewers to use in articles and which to avoid. —Farix (t | c) 16:36, 14 January 2010 (UTC)

I have been told repeatedly by the two of you, but your opinions are not policy. This issue should be discussed here, with as many people participating in it, and giving their opinions, as possible. Please read SPS in its entirety. "Similarly, some self-published sources may be acceptable if substantial independent evidence for their reliability is found." And this independent evidence for their reliability, is discussed here. A review site is automatically reliable, since a review is by definition someone's opinion. Dream Focus 16:41, 14 January 2010 (UTC)

The second part is irrelevant as it is impossible to apply. Simply because some random kid blogs his or her opinion about a number of anime or manga doesn't make the kid's blog a "reliable source", nor does that opinion counted as a reliable source to determine if an anime or manga is notable. —Farix (t | c) 16:46, 14 January 2010 (UTC)

DreamFocus, that doesn't help you--since notability is an issue also. Not every reliable sourced statement is relevant, and many reviews are trivial or unimportant. If a reviewer is deemed (in independent sourced) to be reliable then they become relevant, no? That does not apply to the type of site you are referring to, and webhits don't help. If a community exists (say, a fan community) which deems a certain site reliable (and this can be proven) that's a different matter, and that is I think what MQS might subscribe to also. But such decisions would have to be made on a case-by-case basis, on different evidence than webhits. Drmies (talk) 16:49, 14 January 2010 (UTC)

In consideration of webhits, and I think I see where Dream may be going (I hope he'll correct me if I am incorrect), I'd like to use your own examples in way of comparison. You offer that Times and Tuscaloosa News may both represent reliable sources, but that an opinion (review of a book) by Times is more conducive of notability than a review by Tuscaloosa News (ah... that sliding scale). Would that be because of circulation? If they are both RS because of their editorial staff and reputations for fact-checking and accuracy, would this mean that because the Times has a far wider readership and serves a larger demographic that it is more reliable and more able to confer notability? That would seem to show that Wikipedia gives greater credence toward opinion from sources that have the bigger readership. If in then comparing hardcopy with electronic, it might be seen that web hits could then be compared to readership and distribution. But this still does not address that in all cases, a review is an opinion... and no matter who makes it or about what subject, it is by its nature POV and not subject to the same criteria as "reliability", nor can such be judged for accuracy or truth... but then, those two considerations are not part of WP:V's threshold for inclusion. Schmidt, ^{MICHAEL Q.} 19:19, 14 January 2010 (UTC)

I tend to go by the reviewer-- has s/he had work published in the field? Is his/her opinion cited by published sources covering the field? Has s/he done a professional DVD commentary/liner notes/ booklet? Is the site cited by reliable sources covering the field? Questions like that. I think doing subscription-counts does a disservice to niche fields, which, as MQS points out, have their own set of reliable sources/reviews, but which may have a far smaller audience than works generally covered by, say, the NY Times reviews. Just my 2-cents. Dekkappai (talk) 19:36, 14 January 2010 (UTC)

Just a request ... could we not use the terms "notable" and "notability" here unless we are discussing WP:NOTABILITY issues. We do not want to confuse people. What I think is beind discussed here is more along the lines of "note-worthiness". This is more a matter of editorial judgement, and consensus... and should be guided by WP:Undue weight more than WP:RS. Blueboar (talk) 21:35, 14 January 2010 (UTC)

Sure--if you will tell me what word to substitute for "notability" when discussing notability. Thanks, Drmies (talk) 23:52, 14 January 2010 (UTC)

The problem with not referring to "notability" is that the very reason that Dream Focus started this topic was because of WP:NOTE. Specifically, whether Animetric.com is a reliable self-published source which can be used to determine notability of certain a genre of anime because almost all other established anime critics steer clear of the genre. —Farix (t | c) 01:25, 15 January 2010 (UTC)

My view on this is that critique is almost like acadaemia in that its worth is decided by peer review. The notable opinions are those opinions that are held in esteem by other reviewers, journalists and news outlets. If a certain reviewer or review source is often reported or cited in other works then that source becomes a notable source for critique. I'm generally against the inclusion of Metacritic and Rotten Tomatoes scores since these are not notable unto themselves. These sites can be good for finding reviews and providing the correct balance of criticism - for instance, the reception section for Avatar should perhaps have 7/8 'good' reviews and a couple of negative ones to correllate with the scores, but the actual score of 84% hasn't had its notability established. There does seem to be a gradual erosion of notability as a criteria for including material on Wikipedia and personally I find that sloppy and bad practice. Betty Logan (talk) 03:17, 15 January 2010 (UTC)

Side note - metacritic's aggravate scores are considered "reliable" by those industry "peers" you talk about.

As for Animetric, there have been a couple press releases by RightStuf quoting some of their reviews and a listing on animeFridge's Top 25 sites陣内 _Jinnai 04:45, 15 January 2010 (UTC)

May we assume that "aggravate" here is intended to be "aggregate"? - Jmabel | Talk 21:30, 13 March 2010 (UTC)

A review website obviously isn't a reliable source. Reviews by well known critics are reliable for the opinions of those critics, which will have some significance in their field of competence. Certain aggregation sites that have a good reputation may be regarded as reliable measures of an item's critical reception. But a review on a website is worth very, very little. Of much more value are critiques from reputable news media that have a track record of influential reviews. --TS 21:38, 13 March 2010 (UTC)

Reverted change on section 2.1 bullet point 4

I reverted Hipocrite when he attempted to do a change without discussion regarding the use of index scores (also known as impact and impact factor). He went against consensus as established in the discussion above[2]. This section is created as a courtesy for Hipocrite to lay out his case for a new consensus or any of the other people protesting the clarification (which is the current subject of a long section over at talk:global warming [3]). TMLutas (talk) 17:17, 13 March 2010 (UTC)

Impact factor can certainly be taken into account when evaluating sources. It cannot be ignored entirely, as you proposed to do. I suggest that if you are going to make drastic changes to policy pages, you need to actually advertise that you are doing so. I believe the world "alone" reflects the old consensus about the use of impact factors. If you'd like to have a formal RFC about your change (I should note that changing policies to impact articles is backwards - polices reflect how things are done, they are not proscriptive), then please do so - until then, feel free to either accept the world "alone," or restore the version before your slightly discussed (and never proposed on talk) change. Hipocrite (talk) 17:58, 13 March 2010 (UTC)

We certainly disagree. I spent a couple of months painfully working through the topic on three separate pages. I made it very clear that I was doing so. I made no drastic changes but rather added clarifying text to an already established rule. You may have whatever personal beliefs you wish but so far as I can tell you did a naked edit without any attempt whatsoever to attempt to achieve consensus in talk beforehand (my apologies if I missed it). That means that your belief is exactly that, personal because you hadn't bothered to find out whether your understanding is correct. I understand the idea of being bold but is this page in particular the place to do it? Your subsequent section below seems a much better idea. I think you're wrong but at least everybody gets a chance to talk it through. TMLutas (talk) 00:23, 14 March 2010 (UTC)

Where are the archives?

Sorry for the stupid question, but how do I search through the archives? Using the "search" in archives on top of this page yields no results, nor is there an index.--Work permit (talk) 20:03, 13 March 2010 (UTC)

This page was recently moved without updating the archives to the new location. I'll try to fix it now. Paradoctor (talk) 20:20, 13 March 2010 (UTC)

Ok, triggered spam countermeasures inbetween, but now everything should work again. Let's hope the bots agree with this point of view. ;) Paradoctor (talk) 20:39, 13 March 2010 (UTC)

Thank you very much.--Work permit (talk) 20:56, 13 March 2010 (UTC)

Hmm, the archives are there but search doesn't return any results. As a test, I tried using the word Census, which returned nothing. The term is clearly used in Archive 21--Work permit (talk) 21:01, 13 March 2010 (UTC)

Also, I think some entries are missing. I remember a discussion of using Howard Zinn's A People's History of the United States as a source a few months ago, but I find no record of it by manually searching through the archives. I also see that I've made no entries before this month. I remember making comments and asking questions.--Work permit (talk) 21:17, 13 March 2010 (UTC)

That there is a gap is almost certainly due to the botched move messing with the bot's brains. Someone else is going to enjoy cleaning up that mess, because I certainly won't. ;)

As for the search problem, I thought it was a caching issue. But get a load of this:

As soon as I add even the "I" from "Identifiying reliable resources/Archive 21", the search breaks. I'll ask the techies. Paradoctor (talk) 21:30, 13 March 2010 (UTC)

OUCH! There's a GAP! There is (was) alot of useful information in these archives. I REALLY hope we can retrieve them. How do we get this done? Maybe the editor who moved the page should be tasked with cleaning this up?--Work permit (talk) 21:49, 13 March 2010 (UTC)

Sheesh... have we forgotten how to search data the old fashioned way... by manual scan instead of a bot. Blueboar (talk) 22:05, 13 March 2010 (UTC)

Do you know where the old archives are? I don't mind manually searching them. However, I can't find entries I remember existed by manually scanning (or manually looking). Did the move create gaps in the archives?--Work permit (talk) 22:53, 13 March 2010 (UTC)

According to TheDJ, archives 1-22 should be accessible to the searchbox automatically, can't tell you when, though.

This part (diff) of the page history should contain the missing content. You see, no need to go CAPS on SV. ;) Paradoctor (talk) 23:36, 13 March 2010 (UTC)

The move didn't "create" the gap, it was just perventing the bots from doing their work. I don't know if it works that way, but maybe the bots will catch up automatically. HTH, Paradoctor (talk) 23:38, 13 March 2010 (UTC)

I apologize for the caps. It's very unlike me to do that. I could have sworn there was a discussion of Howard Zinn's A People's History of the United States as a source a few months ago, as well as a discussion of the use of Marxist journals as wp:rs. Conclusions for Zinn was he generally needed to be used carefully, cited when used. Marxist journals are wp:rs, but not mainstream academia. I see it's not there. I must be going senile, losing my mind, or both.--Work permit (talk) 23:53, 13 March 2010 (UTC)

That would make two of us. ^_^ Don't sweat the caps, just razzing you. ;) I finally found out about the search index update interval: Once a day, occasionally more. Time to make tea, it seems. Paradoctor (talk) 00:08, 14 March 2010 (UTC)

The archives are still under RS. What happened was that I initially moved everything that was under RS, archives and all. Once I'd done it, I realized I had also inadvertently moved the RS noticeboard and all its archives too, because that was created as a subpage of RS. So I undid the move except for this page. Therefore we need to decide how much to move, and whether to leave the RS noticeboard as it is, or whether to unhook it as a subpage of RS. SlimVirgin ^{TALK contribs} 03:31, 14 March 2010 (UTC)

"archives are still under RS": Um, not anymore. ;) Search index has been updated, MiszaBot picked up right where it was interrupted, I'd say the archive problem is solved. Paradoctor (talk) 12:43, 14 March 2010 (UTC)

Fan site interviews and reliable sources

I am trying to improve articles for related Masters of the Universe. King Grayskull is an article I am trying to improve I wish to confirm are any of thes acceptable sources because its not unheard of writers communicating to fanistes[4][5] [6]

In regards to Shadow Weaver I ask the same thing [7]

I ask the same of Jitsu_(Masters_of_the_Universe) [8] Dwanyewest (talk) 00:17, 16 March 2010 (UTC)

I would say "no" for all. The exception might be a fan site that has itself been cited frequently by more solidly reliable sources (mainstream media, etc) when discussing related topics, as that establishes some degree of 'expert' status. Blueboar (talk) 01:48, 16 March 2010 (UTC)

Ph.D. dissertations

I object to the sentence Finished Ph.D. dissertations, which are publicly available, are considered publications by scholars and are routinely cited in footnotes. This may be true in some fields but definitely is not true in all. I'd like to change this to Finished Ph.D. dissertations, which are publicly available, are considered citable publications by scholars in some fields. (The "in footnotes" bit is a minor issue. The specific format of citation is an unnecessary detail, and the journals in my field explicitly discourage the use of footnotes.) Comments, questions, complaints? Short Brigade Harvester Boris (talk) 14:41, 5 January 2010 (UTC)

Can you provide any source that explains which fields accept PhD dissertations and which do not (or even some sources stating that PhD dissertations are not accepted as scholarly sources in specific fields)? We should keep in mind also that the standards of different publications vary. Journal X might decide PhD dissertations are not suitable as references in refereed articles, but we are not writing refereed articles here. Most journals would accept data gathered personally by the authors of an article, but Wikipedia does not accept that. --Jc3s5h (talk) 14:53, 5 January 2010 (UTC)

I have never heard of an field which accepts some scholarly sources but rejects PhD dissertations. All the standard guide books (like Chicago manual of Style) explain how to cite them as scholarly sources in humanities, social sciences, engineering, medicine, science, arts, business, law, theology, etc. All the job manuals and online sources for academics (like Chronicle of Higher Education) explain the importance of PhD dissertations.Rjensen (talk) 15:06, 5 January 2010 (UTC)

It certainly would reflect very badly on a department or external examiner if they let through PhD dissertations which can't be cited. I'd like to see some example too. Dmcq (talk) 15:26, 5 January 2010 (UTC)

I agree with Dmcq. Surely that is what differentiates a PhD dissertation from say a master's dissertation - that it should be up to the standard whereby it can be cited by others. Yaris678 (talk) 16:40, 5 January 2010 (UTC)

Work can be awarded a phd that is rejected for publication. This probably puts Phds in the situation where they can be reliably sourced but they've failed the notability test, so I would say that while phds can be used as further sources for result corroboration etc they shouldn't be used as a source for establishing new theory. Betty Logan (talk) 03:30, 15 January 2010 (UTC)

UMI has already published 2 million dissertations, so publication status is rarely an issue. Getting a PhD is a validation of expertise, which is the main criteria Wikipedia is looking for in a reliable source.Rjensen (talk) 04:40, 15 January 2010 (UTC)

The problem is due to the fact that phds in the US and UK can be of a much lower quality that you get in Europe where generally they are awarded based on papers you have published. The peer review is much more rigorous for papers than it is for phd theses. As a rule you won't generally find phds cited in Europe because you can cite the peer reviewed paper instead, so the point stands that not all acadamic communities would consider theses reliable sources, particularly UK and US ones where the work hasn't been subjected to peer review. Good phd work in the UK and US will be published in journals so there should be no problem citing notable work. Some these might provide further experimental work not published in journals so there are cases where it might be useful to cite them, but only in certain contexts. Betty Logan (talk) 05:12, 15 January 2010 (UTC)

I'm not sure what disciplines Bettty Logan is referring to. In the humanities and social sciences European PhD's are not held in as high regard as the U.S.--to get a professorship in Europe a candidate has to do additional work especially the "habilitation". But all this is beside the point: a PhD dissertation is a certificate of expertise that far exceeds the usual standard of Wikipedia articles. Rjensen (talk) 05:49, 15 January 2010 (UTC)

"In the humanities and social sciences European PhD's are not held in as high regard as the U.S." Rjensen, do you have a source for this rather extraordinary claim? --Hegvald (talk) 13:29, 20 January 2010 (UTC)

In much of Europe the PhD is not the final degree for an academic, as it is in the U.S., in large part because it is done in much shorter time than in the U.S. So Europeans have to get the "habilitation" as well if they want a professorship.Rjensen (talk) 17:47, 20 January 2010 (UTC)

I think you have misunderstood this. A habilitation is a qualification above that of the PhD. This does not mean that the the previous doctorate is somehow inferior to a U.S. degree, and I think you will have a rather difficult time finding an authoritative source claiming this. A habilitation, in German universities, usually implies that the "Dr. habil." has produced two substantial monographs (the first is the doctoral dissertation, the second is the habilitationsschrift) published by reputable academic publishers. Yes, it qualifies for a professorship at a German university, but that is a far more distinguished position than the college "professorship" that a U.S. Ph.D. will get you in the first instance.

BTW, I don't think it is meaningful to speak of a "European PhD", because I don't think such a thing exists, and I can only speak of the parts of (mostly northern) Europe that I am somewhat familiar with. --Hegvald (talk) 19:35, 20 January 2010 (UTC)

Well you'd take a phd from Harvard over anywhere else wouldn't you? The problem is the lower bound. Maybe it's different in the US, but it has a similar system to the UK and you get some right dross at the lower end in the UK. The best phd candidates tend to go to American Ivy league or Oxbridge so maybe the top tier in Europe isn't in the same league as the top tier in America, but in all likelihood this phd work will be published in journals anyway so there won't be a neccessity to cite the phd thesis itself. The problem is at the lower end where the work wouldn't really satisfy a peer review. If the work is notable, and is of a satisfactory quality why isn't it in a journal where it has been peer reviewed? I simply wouldn't tolerate phds being cited, the only exception being phds from the same institution due to the fact there may be some collaborative work. Betty Logan (talk) 10:54, 15 January 2010 (UTC)

Sometimes other considerations do get in the way of journal publication. My PhD was accepted by examiners - probably the same people who would've reviewed it, had it been submitted to a journal. But the work had commercial applications, so the thesis was kept off the shelves and we couldn't submit anything for journal publication while we dealt with patenting. By the time we could publish without interfering with patents, I'd moved on to other projects, and getting a journal publication wasn't a priority (also, I was getting pretty heartily sick of that topic - writing patent apps can do that). So in the end, the only publications I had for that work were the patent application and my original PhD. --GenericBob (talk) 11:35, 15 January 2010 (UTC)

It's a tricky question. In large parts of Europe the universities have mutually similar standards, at least within each country. In the US and UK it's very different, with a small number of universities absorbing all the top students. For the UK I can say that the state is actually pressuring the universities other than Oxford, Cambridge and Edinburgh into becoming diploma mills: To preserve their funding they are forced to accept as many students as possible, which is hard given that they only get applications by those who couldn't get a place in one of the top universities. And then there is strong pressure to let everybody pass. Traditionally students in some countries are also much older when they start work on their PhD, and typical time to completion is much longer.

I guess without going into extreme details about countries and subjects the only thing we can say is that some theses are reliable sources and some are not. And we can give guidance such as:

If a PhD was awarded by a diploma mill or other dubious university, then the thesis is not a reliable source.
If a PhD thesis has been cited (but not rubbished) in peer-reviewed publications, it is probably a reliable source.
If the results of a thesis have been published in a peer-reviewed publication, then it is probably a reliable source, although often it will be preferable to publish the peer-reviewed publication. Hans Adler 12:22, 15 January 2010 (UTC)

Personally, I think it is hard to imagine that accepted PhD dissertations aren't reliable sources. However, I'd say they are often of a lower quality than the primary literature. So, definitely citeable, but if there is a conflict or other issue one should assume the peer reviewed literature is more likely to be correct. Sort of like how we would usually regard a news account of a scientists' opinions as reliable, but would greatly prefer to cite their actual papers whenever possible. Also, is the section Boris quotes referring to external practices or wiki practices. I would have assumed the later, in which case it is correct that we routinely cite such material in footnotes. Boris' comments however seem to be treating it as commentary on external publishing practices, and I don't think that's what was intended. Dragons flight (talk) 12:09, 15 January 2010 (UTC)

Maybe I am missing the point here, but surely (and by definition) PhD theses are original research, which in one sense or another makes them a Primary Source - so, in general it is better to find a secondary source, but when there isn't one, to use the primary source? I think any attempt to split universities into 'diploma mills' vs 'legitimate research centres' at a policy level becomes an overtly political stance which is likely to bring all sorts of the wrong media attention knocking at WP's door. (20040302 (talk) 11:42, 21 February 2010 (UTC))

No, if something has merely been plagiarised or written pro forma by someone who needed to keep up the appearance that they were doing research, then it's not a primary source and not a reliable source. That's a typical situation with diploma mills, which is why we need to be careful with them. I don't think anybody is going to make a list that ranks or classifies universities according to some criteria. A normal PhD thesis may or may not be reliable, depending on its quality. (Even at proper universities a PhD may occasionally be granted even though serious problems with a thesis have become apparent.) Whether it is a primary or secondary source depends, as always, on the claim it is used for. Hans Adler 13:33, 21 February 2010 (UTC)

In the Humanities something can be a well researched, well reviewed book and nevertheless be completely utterly wrong! That is why there are tons of academic books and journals and only very few basic textes or Standardwerke). I like to see WP only cite those basic textes, but this won't happen, I guess, and so (No Original Research) all Ph.D dissertations should be citable.--Radh (talk) 10:58, 18 March 2010 (UTC)

WP:SPS and Free/Open Source Games

Due to the absence of reliable open source gaming publications, many open source game articles are deleted or are at risk of deletion. Respected open source gaming blogs such as Free Gamer inherently fail WP:SPS, so they aren't usable for proving notability. This proposal is to allow per-niche exemptions to WP:SPS (though not SELFPUB #5) such that it's unnecessary to argue an open source game's notability by dredging up half a dozen banal lists that it appears on. Singlemaltscotch (talk) 13:46, 15 March 2010 (UTC)

Note: This was originally written at Wikipedia_talk:VG/RS, but in retrospect this proposal entails a global policy change. Though the scope of this essay is limited to video games, I believe it's applicable to any topic which has poor "reliable" sources, yet otherwise-good sources which fail WP:SPS.

At present, I think policy is doing harm to free software games, at least those using a bazaar development model (which is almost all of them). Consider that there are dozens of reliable publications listed for sourcing at WP:VG/RS, yet none of them deal with free software/open source. A few of them occasionally publish lists of free games, but all too often they muddle the distinction between freeware and free software, typically focusing on the former. On the freeware (and independent, commercial) front there's a fair amount of coverage, because these development models mirror that of the AAA-budget commercial titles they typically cover, just on a much smaller scale.

The reasons why there are no professional free software gaming publications are severalfold. Firstly, relative to a commercial game, progress in a volunteer project is inherently slow, due to their inability to work dozens of hours a week on a hobby project. Secondly, progress in an open source game is typically evolutionary, not revolutionary: An average release will typically bring a number of bug fixes and some minor content additions. On the other hand, commercial games are largely binary: Progress is kept entirely internal, perhaps releasing screenshots or gameplay demos, until the game is complete, at which point it's released and the marketing department begins its work.

The distinction here is quite important. The commercial product goes from teaser screenshots to fully-playable game as soon as it's released, and even without the millions in marketing large publishers spend, there's still something wholly new out there to try (and write about). With an open source project, even major releases aren't total departures from the previous versions, merely building upon them. Lastly, community size, both in terms of developers and users. For every prominent open source game project, there are multiple professional game studios, and on the user side of things, the average gamer would rather play Halo 3 than the latest release of The Battle for Wesnoth, with the former game having orders of magnitude more popularity despite Wesnoth being one of the most popular open source games.

These factors combine to make professional open source gaming publications infeasible. You've got less readers, less games to cover, each game progresses more slowly versus a commercial counterpart, and there are rarely brand new games to cover.

Professional open source gaming coverage is basically non-existent. You get occasional blips in the form of lists in non-gaming publications like Techgage, Linux-Magazine and APC Magazine, but this isn't journalism. These lists are content for the sake of itself, filling out a light month with articles hacked together in 20 minutes from Google results. They come from "reliable" sources, but grading their video game articles independently, I wouldn't trust them as far as I could throw them.

On the non-commercial front, you have The Linux Game Tome and a handful of similar sites like LinuxGames, which publish user-submitted game news with minimal editorial oversight. These are very shaky, but at present they're cited rarely, due to lack of other sources, though more often appearing in external links where they probably belong. There is also LinuxLinks, which composes its own lists and articles (Such as this list of 100 games), but the quality does seem low. The per-game descriptions mostly amount to rewordings of materials found elsewhere, with zero insight into the games (or reviewing), and typos are encountered throughout. Yet... this seems to pass WP:SPS, and thusly I've seen it cited quite a few times.

As for things that definitely fail WP:SPS? That's where you find the good open source journalism, I'm afraid. Free Gamer, for example, is one of the best open source game blogs around. Unlike the Linux Game Tome and its ilk, Free Gamer goes beyond parroting news releases. It does reviews, like a good game publication should; it's well written, yet it's still a self-published blog and thus fails WP:SPS. There's also Linux Gaming News, though that focuses more on closed-source software, which is outside the scope of what I'm discussing.

My basic point is this: Open source games are not supported by a multi-billion-dollar industry. New releases don't get picked up by a dozen established publications, and there is no advertising to speak of. The "reliable" sources that publish open source game information mostly do so to fill space, with no actual journalism taking place. Thusly, good sourcing becomes very difficult unless policies like WP:SPS are partially ignored, and articles are able to be supported by locally-reputable sources like Free Gamer.

Note that I'm not suggesting that WP:SELFPUB #5 be ignored, as that would cause a breakdown of notability requirements.

To illustrate my point, take a look at a few articles from the List of open-source video games. Unless someone's used a game in a research project or it used to be commercial, you likely won't find any decent sources. I fully believe this is a case of policy getting in the way of improving Wikipedia, and per-article exceptions won't solve that.

Addendum: To be absolutely clear, I am not arguing for a change in the general notability requirements, merely for per-niche exceptions to WP:SPS such that notable articles in difficult-to-source niches can rely primarily on certain sources which fail WP:SPS, rather than resorting to scrounging through hack-journalism lists put out by otherwise-reliable publications. This basically amounts to a case of WP:IAR, but in this case a formal protection is merited, else articles may be put up for deletion unnecessarily. Singlemaltscotch (talk) 05:05, 13 March 2010 (UTC)

To be honest, we are already easier on open source stuff when it comes to notability in AfD debates. But as you point out, primarily this is indeed a question of notability rather than source reliability, since self published sources are OK within the guidelines for using them. SELFPUB #5 is mainly referring to the fact that such articles would have notability issues. Gigs (talk) 22:30, 13 March 2010 (UTC)

I haven't found any evidence of leniency thusfar. I was looking over the WP:WikiProject_Video_games/Deletion archives and noted a few things as I saw them. In February alone, TripleA, ToME and Teeworlds were deleted. SuperTuxKart, Stratagus, and the Spring project were nominated, but kept.

Going further back, Ryan C. Gordon and Secret Maryo Chronicles were nominated, and X-Moto was deleted. I wish to draw attention to the Secret Maryo Chronicles log, wherein the article was deleted six times over several years, and brought to AfD 3 times, with the most recent being in December of this past year.

There's also Widelands which has been brough to AfD twice and additionally deleted twice prior to the latest restoration.

Particularly egregious is that Netrek, cited as "the third Internet game, the first Internet team game" was put up for deletion with apparently zero notability checking, and there are many other examples similar to this.

To illustrate the risk that other existing articles face, I went and tallied articles in the List of open-source video games. Of the 118 there, 8 don't have their own pages (being sections within other pages or not Wikilinks at all), 10 have WP:GNG tags, 14 have references which all fail WP:SPS or are otherwise unreliable, and 6 lack references entirely. Based on this, 20-30 (17-25%) of the articles listed there are at risk of potential deletion.

This says a lot. There are several thousand free software game projects listed at The Linux Game Tome alone. Many of these are obviously non-notable, and yet even among those prominent enough to be featured on the List of open-source video games (which is a selected list and mentions on its talk page that included articles must be larger than a stub) many are vulnerable to deletion because good sources like Free Gamer are instantly thrown out as failing WP:SPS and being unreliable at an AfD.

Lastly, this does go beyond games, evident after giving this a cursory glance. It probably goes beyond free/open source software, as well. Given over-reliance on professionally-published media, any topic that doesn't get mainstream media coverage will suffer at the hands of deletionists. Given the shoestring budgets many modern publications operate with, there's increasingly reduced editorial oversight, putting the fact-checking on par with that of your run-of-the-mill blog, but I digress. Singlemaltscotch (talk) 08:57, 14 March 2010 (UTC)

I saw you added an RfC Tag. Per WP:RFC you need to "Include a brief, neutral statement of the issue" after the RfC tag. You've written so much here I'm not sure anyone is quite sure what you are proposing. I suggest you create a new section on this page, sum up your position in 2 or 3 short sentences, and them move the RfC tag to the new section. Gigs (talk) 13:06, 15 March 2010 (UTC)

I've added a summary at the top of the existing content and the RfC bot should update the RfC page in roughly 15 minutes. Singlemaltscotch (talk) 13:46, 15 March 2010 (UTC)

Strong oppose, if they are notable, they will have real coverage in real reliable sources, not random, self-published blogs. Being open source does not somehow make them exempt from basic notability standards. If they are actually notable, they will have the coverage and sourcing. Trying to find ways to claim notability by deregarding basic notability standards, and indeed from the opening of this argument, it seems clear that even the original poster realizes they are not actually notable in the real sense, and is seeking away to keep articles about the games he and others enjoy purely based on blogging and other non-reliable sources. Indeed, I would question whether this is purely an effort to save his own article, NAEV? This is not the purpose of Wikipedia. There are plenty of free/open source games and software packages. As with ANY software, most are not notable (same as most of the titles at the $10 rack at BestBuy or Wal-mart, few if any are notable for inclusion here either). -- AnmaFinotera (talk · contribs) 13:53, 15 March 2010 (UTC)
My entire point is that such coverage does not exist, due to the inherent lack of profitability in a magazine covering open source games. Also, as I mentioned, I am not attempting to erode notability standards: I am attempting to allow good, self-published reviews to be on equal standing with cruft lists put out by professional publications such that open source games do not need to use terrible 'reliable' sources to avoid a GNG tag and potential AfD. The 2008 video games category has 935 articles. 2009 video games has 801. Open source video games has 76. The entire history of notable open source games can apparently be condensed to less than 10% of what the professional game community puts out every year. Lastly, I find it insulting that you're suggesting I'm just attempting to game Wikipedia for the inclusion of one short article; perhaps you should focus on my points instead of attacking potential motivations. Singlemaltscotch (talk) 14:18, 15 March 2010 (UTC)
If real coverage, as it is already defined, doesn't exist, then it isn't notable. And yes, trying to claim that just because its "open source" that it isn't covered in reliable sources if completely incorrect. The game you want to save an article about is not covered, because it isn't notable. There are plenty of other notable, open-source software packages out there with tons of coverage. Claiming they aren't covered because it isn't "profitable" to the magazine is a spurious, unsupported claim. Please prove that no reliable source will cover any open source game based purely on its being "unprofitable" or for any other reason. They are professional publications for a reason - they are the experts one what is, and is not, notable. There is no such thing as a "good, self-published review". Its either a reliable source, or it is not. Trying to claim that just this very minute set of articles should somehow get a free pass because they are, in fact, not notable to the real world is a suggestion for denigrating notability standards and opening Wikipedia up to even more spam articles than it already gets from the open source community. And sorry, but when one is a fairly new editor, and tries to request a serious change in a guideline at the same time one of his only contributions, a game that is completely unnotable WITHOUT this change....anyone is going to question your motivations. -- AnmaFinotera (talk · contribs) 14:33, 15 March 2010 (UTC)
This is not about the ongoing AfD for an article I created, that is wholly off-topic for this RfC. My statement is that there are no publications dedicated to covering open source games, because it's unprofitable. Nothing else. My reasoning for that statement is addressed in the initial essay, and boils down to the incremental development model. I never said that professional publications never cover open source games. I did, however, mention that they tend to cover them in pathetic list format, such as the three examples I provided. Professional games magazines cover big-budget games because they can slap "Review of $GAME inside!" on the front cover along with $GAME's protagonist, and thusly the marketing spent on the game by the publisher indirectly benefits the magazine because people recognize the character and end up buying the magazine. Games magazines cover whatever is new and popular because it makes the money. The only time a major games magazine will cover open source games is when they need to fill space, and as mentioned, "they muddle the distinction between freeware and free software, typically focusing on the former." To put a nail in the questionable-motivation argument, of course having an article I created put up for deletion served as a catalyst, but my examples are valid. Many of the games deleted are very prominent in the open source community, yet get no mainstream media coverage because the open source games community is largely self-contained. As yet another example, Warsow (game) versus Teeworlds. The former has fewer than 200 players, typically. On the other hand, Teeworlds manages more than 7 times that (unfortunately it lacks an online server browser, but I just verified that figure myself)... yet was deleted relatively uncontroversially, because Warsow happens to be popular in the e-sports community while Teeworlds is not, and thusly Teeworlds has nearly zero "reliable" coverage. That one game can be 7 times as popular as an apparently-notable game but be easily deleted tells me that policy is damaging Wikipedia's open source coverage. Singlemaltscotch (talk) 15:37, 15 March 2010 (UTC)
What you are describing is a fundamental property of using secondary source coverage as our inclusion criteria. Our articles will tend to favor those products with large marketing budgets that can draw the attention and coverage. It's a systematic bias that we can't easily combat. Drawing an exception for a niche like this wouldn't make sense. Gigs (talk) 15:50, 15 March 2010 (UTC)
Though I doubt few will agree with me, I prefer to see notability as relative, not absolute. If you treat every commercial game with a few reliable sources as notable, you end up including just about every one with even a slight marketing budget because there are many gaming magazines, and they're all itching to cover something that their rivals miss. On the other hand, very few open source projects can surpass the notability requirement because in the vast majority of cases they have zero marketing. I think the absolutism of notability will be looked back on with regret a few years from now. For every Pac-man or Pong there are many failed arcade games, and I suspect many of them aren't on Wikipedia because the publications that covered them were likely long-gone as well by the time of Wikipedia's conception, and history proved them to be ultimately non-notable. And yet, we now doubtlessly have hundreds of irrelevant games from the mid-1990s and early 2000s that achieved notability through coverage, despite the fact that nobody reads their articles at this point. Singlemaltscotch (talk) 16:12, 15 March 2010 (UTC)
To some extent we already do treat notability as relative, in the sense that prominence or uniqueness within a genre does seem to garner more sympathy for marginal sourcing (but not a free pass). I used to cite WP:MILL often... this would be the inverse implication of that logic, if something is exceptional, then we probably should have an article on it. I think you overestimate our rigidity here. Notability is a guideline which can and does have frequent exceptions and a good amount of flexibility. Gigs (talk) 18:05, 15 March 2010 (UTC)
I think you place too much faith in the collective. When I was gathering sources for my first reply to you I noticed that most of the successful deletions went through with no opposition, and most of them originate from a handful of users who evidently have a strict interpretation of the notability requirements. The key is that you can find "reliable" sources for just about any open source game in the innumerable lousy lists from reliable publications, but that typically isn't done because most of that information is accessible from better sources that unfortunately fail WP:SPS. You end up with a notability tag on an article, it eventually gets nominated for deletion. The lists which assert notability aren't typically found on the first few pages (Google prioritizes in-depth content over list cruft, funny, that) and you end up with a non-controversial deletion, which is part of why 25-30 years of open source games amount to around 100-150 articles while the video game categories from 2000 to present alone encompass 7696. Singlemaltscotch (talk) 19:29, 15 March 2010 (UTC)

The arguments being presented are essentially the same as are made by adherents to obscure Fringe theories... that the mainstream all but ignores their pet topic, so the only way to cover a sub-topic is to allow the use of niche websites devoted to the topic. I don't buy the argument when proponents of Fringe theories make it, and I don't buy it when fans of niche hobbies like open source computer games make it. If something has not been noticed by the mainstream, Wikipedia simply should not have an article on it. I know this is hard to accept for fans of Niche topics that are not noticed by the mainstream... but it is how Wikipedia works. Blueboar (talk) 16:00, 15 March 2010 (UTC)
Perhaps a bit of clarification is needed on my part. It's pretty much invariable that a prominent open source game will be covered by reliable publications through their inevitable "Top N Open Source Games" lists. The argument I'm making here is that while these lists come from reliable publications, they are horrible journalism and I would not want to cite them in a Wikipedia page because they are typically poorly written and often do nothing beyond supplying a two-sentence "review" alongside a screen shot. Citing them decreases the reference list SNR because instead of presenting a deeper view of the topic, they skim over it typically in vastly less detail than the Wikipedia article itself. All I'm proposing is that if notability can be asserted through lousy-yet-reliable sources, it should be adequate to note that they exist (perhaps on the talk page), but actually draw in-line citations from a locally-respected yet unreliable (due to WP:SPS) source. This would not be arbitrary editor-level sourcing, I would opt for a niche-specific white-list of sources maintained at the Wikiproject level. Coverage in such a source would strengthen notability if it already existed, but it would not be sufficient to allow an article to exist in the absence of reliable sources. Singlemaltscotch (talk) 16:42, 15 March 2010 (UTC)

You miss the point... the article needs to establish that the topic is notable. And if the only way to do so is through lousy sources, then that is what you must use. After you have established notability, then you have a bit more flexibility when it comes to using SPS's to expand upon the topic. And, of course, there are limits to how and when SPS's may be used. You have to accept the fact that we intentionally make it difficult to write articles on niche topics. Blueboar (talk) 17:18, 15 March 2010 (UTC)

See my response above to Gigs for why I feel absolute notability is bad. I suspect that WP:SPS will eventually be scrapped, or at least heavily rewritten. Earlier, I stated "Given the shoestring budgets many modern publications operate with, there's increasingly reduced editorial oversight, putting the fact-checking on par with that of your run-of-the-mill blog..." and I suspect that will continue. I feel WP:SPS is partially lodged in the pre-Internet era where publications' ad revenue was able to support a large editorial team. I understand that the purpose of WP:SPS is to exclude cruft sources, but these days many magazines have gone online-only and most of their staff are paid "bloggers", in that there's no virtually editorial oversight and their words go to print, warts and all. The thing is that while WP:SPS does do well in excluding misinformation spread by writers who can't be bothered to check there facts, I don't believe there's any inherent difference in quality between two writers just because one is on the payroll of a magazine publisher. The probability of the source being high-quality is greater if someone else has elected to pay them for their writing, but that does not by any means mean there aren't awful professional journalists and great amateur bloggers. —Preceding unsigned comment added by Singlemaltscotch (talk • contribs) 18:40, 15 March 2010 (UTC)

Well, all I can say is, don't expect WP:SPS to be scrapped any time soon. It has a lot of support. Blueboar (talk) 19:37, 15 March 2010 (UTC)

I think Singlemaltscotch brings up some valid criticism of Wikipedia's approach to video game sources. However, I don't think there's a valid argument for using sources not considered reliable. Yes, most reliable video game sources are flawed, but they are not bad or untrustworthy.

What this boils down to is that a niche area of video gaming just doesn't receive significant coverage. That is not Wikipedia's fault though. And tweaking notability guidelines to cater to one niche opens the doors to others. One other point I think should be mentioned is that commercially released video games don't get a free pass. They have to prove their own notability and can be deleted if they don't. Admittedly, the have an easier job than indie and open source development, but the guideline still applies. (Guyinblack25 ^talk 20:38, 15 March 2010 (UTC))

I'm not sure how WP:SPS can be relaxed. We currently draw the line at accepting "established experts". The above proposal is to accept SPSs that provide quality writing, but I'm afraid this would lead to terrible abuse (I'll just get some of my chums to write extensive reviews about some homebrew game I knocked up last night. They can make it up if they like; there's no editor. Yay... I get an encyclopedia article.) Marasmusine (talk) 21:24, 15 March 2010 (UTC)

I think that quality writing combined with a proven track record of good coverage would be sufficient to exclude that case. It's still possible that someone is personal friends with an established author and gets biased coverage, but that can happen in professional journalism as well. Technically an editor should mitigate that, but as I've mentioned, quite often there is no editorial process at the running-on-ad-impressions publications. An example being this guy, who's something of a laughing stock on both the Windows and Linux sides of the fence because despite continuing to collect a pay cheque, he pens some of the worst drivel you can find, with factual errors abounding in just about every article. I suppose poor professional journalists and publications can be black-listed, but if so why not go the extra step and allow topic-specific white-listing of known good amateur journalists? Although I do think that particularly good WP:SPS-failing sources should be allowed to confer notability, I realize that's a very large step away from existing policy, so I would be very happy to just be able to use good amateur sources while enumerating bad professional/reliable sources on the talk page, or just in the references list without in-line citations. Singlemaltscotch (talk) 22:04, 15 March 2010 (UTC)

Oppose - I feel for you, there's much of the same problem with coverage of tabletop roleplaying games. But I see no reason for departing from the current policy here. If there are no reliable sources for a game, there are no reliable sources for it, and wishing doesn't make unreliable sources suddenly reliable. If that means you can't establish notability for the game, then it may be it's just not notable. That said, I think your dilemma is a false one; it's certainly the case that free and open source games DO from time to time get coverage on sites such as Kotaku, Jay Is Games, and other such reliable (read: editorially controlled) gaming sites. If a particular game doesn't have that coverage, it's probably a good sign that it's not (yet) deserving of a Wikipedia article. - DustFormsWords (talk) 04:26, 16 March 2010 (UTC)

I should say also, I think this proposal is probably the end result of a wholly commendable but flawed process of thought, which goes: (a) Reliable sites don't cover open-source games, (b) Reliable sites SHOULD cover open-source games, (c) Therefore some sites which cover open-source games should be deemed reliable. Propositions (a) and (b) are unobjectionable, the problem is (c). The correct response would be, (c) Therefore I will start a reliable site, and cover open-source games. Or (c) Therefore I will encourage reliable sites to cover open-source games. We agree there's a problem, we disagree that it's something that should be fixed via Wikipedia. - DustFormsWords (talk) 04:32, 16 March 2010 (UTC)

I would say (c) is impossible in either way, because 1) One can't start a reliable website to cover open source video games without first starting a proper business, hiring editorial staff, etc, etc, and that would lead to expenses. Reliable websites cover those expenses with (among other things) the advertising profits they make from the hits they get of people seeking the mass media marketed commercial video games, therefore open source video games would not get so many hits, therefore the business would not have profits and go bankrupt. And 2) One can't encourage existing reliable websites to cover open source video games because of the end consequences of 1). So we got ourselves a catch-22 here, and the policy that solves it actually reads "Leave it out". NeoGenPT (talk) 05:41, 16 March 2010 (UTC)

Well, no, it's not a catch-22. Open source and free games frequently ARE documented on reliable sources, which means that those that aren't are simply less notable, by any reasonable standard, than those that are. For your open source games, if these games were notable, one would think they would either have (a) a potential audience, who would be willing to be apprised of developments through a professional site, or (b) critical recognition by way of one of the many awards regularly given to independent game developers, such as those of the Independent Games Festival, the Mochis, et cetera. If the game doesn't have critical recognition on reliable sources, or a sufficiently large audience to support an editorially controlled site, it's hard to see how you could say it's notable. Even if there IS some sort of systemic bias going on here, it's not a Wikipedia problem. Wikipedia documents the revolution, it doesn't start it. The challenge is to explain how your sources are reliable, not to change the definition of "reliable" until it includes your sources. Also, as a side note, I think your understanding of the economics of games journalism is flawed. I've personally had great success in getting sites such as Kotaku and PalGN to cover indie and open source games by sending polite and well-explained tips to their editors. Writers on many of these sites are paid based on how many clickthroughs they get from the article headline to the article content, and nothing drives clickthroughs like the promise of a free game. - DustFormsWords (talk) 05:50, 16 March 2010 (UTC)

Well, if the challenge is to explain how the sources are reliable without changing the definition of reliable, can anyone take a stab at one of SingleMaltScotch's examples like Free Gamer and prove that it's reliable under the current definition of the word? That would kill two birds with one stone, as it would make SingleMaltScotch slightly happier and at the same time keep the definitions as they are. Is it possible? NeoGenPT (talk) 07:58, 16 March 2010 (UTC)

Sadly it's not a reliable source and I wouldn't support any proposal that deemed it one. Per its FAQ it doesn't purport to be a reliable source. Rather, it's a listing or directory of games, run by (as far as I can tell) a single person, driven by person interest, supported by donations and ad revenue. Now, there's no reason in principle that two like-minded people couldn't establish a similar site, designate one the editor and the other the writer, and develop a reputation for accurate, fact-checked articles. And that would, arguably, in time be a reliable source. But that's not the case here. However, I don't see a problem with sites such as Free Software Magazine or H Online Open. Is there really such a lack of reliable sources covering free and open source gaming? - DustFormsWords (talk) 08:12, 16 March 2010 (UTC)

I took a look around and found the page for the Free Gamer shared user, it appears that there are four people involved. My main gripe about WP:SPS is that it's impossible to verify if a source truly passes it. As I've said a few times, I don't see commercial backing as inherently conferring reliability, with the commercial nature of their postings merely entailing a higher probability of good journalism. Even at sizable publications it's impossible to know whether an author's work has been thoroughly scrutinized. In an ideal situation I think commercial publication would not entail any reliability. Each source would have their body of work reviewed and fact-checked by Wikipedians including them in articles. I understand that that entails a colossal amount of work, and that WP:SPS's job is to mitigate the load put on Wikipedians checking sources, providing an easy way to disqualify a large portion of them which, truthfully, do tend to be worse than professional equivalents. However, I do believe all rules have exceptions, and thus particularly good amateur sources should be reliable, if not conferring notability. Again, ideally they would confer notability, but I understand that would make it far too easy to include fringe theories and the like by sourcing fifty blogs and stating that you've judged them to be reliable and, others lacking time to check them, your article would never be challenged -- This is why I'm proposing a selective white-list maintained at the Wikiproject level. In most cases this isn't necessary but there are a number of communities that are largely self-contained which receive little mainstream media coverage. Singlemaltscotch (talk) 10:14, 16 March 2010 (UTC)

"...there are a number of communities that are largely self-contained which receive little mainstream media coverage"... absolutely... communities like UFO enthusiasts, believers in various conspiracy theories, ethnic/racial supremacy advocates, etc. No, I am not trying to say that fans of open-source games are like such groups... I am trying to explain that if we make exceptions for one project, it has ramifications in other projects and topic areas. If we loosen up on SPS and allow Wiki-projects to "white-list" sources that would otherwise not meet our standards, this would be heavily abused. There is a good reason why we make it difficult to use self-published sources, and I would strongly oppose any attempt to make it easier. Blueboar (talk) 14:05, 16 March 2010 (UTC)

Just so I understand what you are saying: If it were not for considerations outside of this particular case, you wouldn't oppose this one? Paradoctor (talk) 14:34, 16 March 2010 (UTC)

No... I am merely pointing out that changes to policy have ramifications that go beyond the narrow confines of any particular case. I see no good reason to make an exception for this niche topic, because I don't think we should make an exception for any niche topic. I strongly support the limitations of WP:SPS, and if that means it is difficult to write an article on a particular topic or sub-topic, so be it. Blueboar (talk) 17:24, 16 March 2010 (UTC)

This is a bad example, since NPOV requires us to fairly describe the views of these contentious groups, to the extent that they are notable. The problem of communities being 'self-contained' is resolved by attributing the contentious views to the community and contrasting them with the mainstream view, while verifiability is pursued by looking for sources which reflect the minority view as a whole, or naming prominent adherents. (See WP:FRINGE) In the case of FLOSS games, a good start would be allowing citations to SPS which are notable according to WP guidelines, and/or SPS which have been widely cited as reliable by organizations with stricter editorial oversight. Classicalecon (talk)

Oppose — In my opinion, the verifiability and notability policies here are parameters structured to enforce the more general (and thus harder to enforce) idea that Wikipedia is, like most encyclopedias, a repository of mainstream knowledge. The mainstream knowledge which can and should be included is both broader and deeper than traditional encyclopedias due to the fact that Wiki is not paper, and can and should recognize the existence of, and provide limited points of access to, knowledge which is not mainstream. But Wikipedia should not ease those standards for information such as this which is not mainstream. That doesn't mean, moreover, that those who want to preserve this kind of information are simply out of luck. There's no reason that they cannot start their own specialized wikis — perhaps through Wikia — to keep it. It's not that their information isn't worth preserving, it just doesn't fit within the mission of Wikipedia. Best regards, TRANSPORTERMAN (TALK) 19:40, 16 March 2010 (UTC)

That is what has been happening and Wikipedia's own internal statistics prove it, the number of editors and edits has been steadily decreasing over time, people are moving away because they do not feel their knowledge belongs here anymore. At some point in time Wikipedia will become obsolete because the amount of work to keep it updated will outweigh the capacity of the few editors still present. (If it doesn't do already...) And thus one of three endings can happen, it shuts down gracefully and is recorded in history as the world's greatest global collaboration project, it stays as a repository of mostly obsolete information, or it turns to a business model and is operated like any other encyclopedia with paid editors doing the job. I know this doesn't concern the topic in question, and that I may be painting a too grim scenario here, but it's the logical evolution of things when you say "NO" too many times to people who are just here voluntarily, they will eventually get tired and move on. And sometimes all it takes to counter this evolution of things is just to reach out a little... NeoGenPT (talk) 09:42, 18 March 2010 (UTC)

College papers

In light of the Mike Handel BLP incident, I think that college newspapers should be excluded as reliable sources. Now, the vast majority of college newspapers are fine organizations run by ernest students, but as a reliable source they suffer from a number of problems.

Many colleges have more than one paper, and typically only one is any good. Knowing which one can be trusted is impossible from a distance.
Editorial control is weaker than in major newpapers, and staff turnover is high. Few people are paid, so there is limited fear of losing one's job.
Students report on issues of extreme local interest, and cannot be expected to have the experience and perspective to weigh information appropriately. For example, they may overreact to minor incidents.
Online archiving is spotty. In the Mike Handel fake biography, the hoaxer "referenced" The Magdalen College Record which was not available online.

So, I propose that the line "College and university student publications are to be considered unreliable sources" to be added to the News organizations subsection. Abductive (reasoning) 23:33, 17 March 2010 (UTC)

I'm not sure how the Handel incident relates to college papers? The *insert term* who perpetrated this incident made it pretty clear he made up every "sourced", forged them when they were questioned, and basically did everything to just play a huge prank on Wikipedia. Even "Handel" was some made up guy. Ban the "editor" for being pointy and disruptive, but I'm not seeing how that at all relates to whether a college/university newspaper is reliable? Certainly just as "reliable" as some Ph.D. student's thesis. -- AnmaFinotera (talk · contribs) 23:43, 17 March 2010 (UTC)

It got me thinking. Anyway, "some PhD student's thesis" is written by a person with a few more years of experience than the undergrads that work on the student newspaper, and subjected to serious scrutiny by a committee of experts. Abductive (reasoning) 00:59, 18 March 2010 (UTC)

WP:BLPs

The first paragraph of the project page contains this sentence: "The policy is strictly applied to all material in the mainspace—articles, lists, and sections of articles—without exception, and in particular to biographies of living persons: unsourced material must be removed from those immediately." (Emphasis added.)

The emphasized portion is not supported by policy or the community. It was added apparently without discussion on this page.

I am going to delete "unsourced material must be removed from those immediately." Maurreen (talk) 07:47, 18 March 2010 (UTC)

FYI, in Phase I of the RFC suggested by ArbCom to address unsourced biographies of living people, a proposal to delete on sight "any biography that is poorly referenced or completely unreferenced" was rejected by the community, 157 to 54. Maurreen (talk) 07:58, 18 March 2010 (UTC)

Absolutely not. WP:BLP clearly states: Contentious material about living persons that is unsourced or poorly sourced—whether the material is negative, positive, neutral, or just questionable—should be removed immediately and without waiting for discussion. In this case we follow WP:BLP. If you don't like that statement, gain consensus at WP:BLP to change it. Then, and only then, will we change our policy. Blueboar (talk) 15:56, 18 March 2010 (UTC)

There is no consensus for requiring or even encouraging the removal of unsourced material that is not contentious or otherwise problematic. Maurreen is right. I'm not sure why you reverted her change. Gigs (talk) 16:04, 18 March 2010 (UTC)

Blueboar, see the word "contentious" in the sentence you quoted?

This has been discussed at length elsewhere. There is no consensus for your interpretation, as exemplified by the RFC !vote I linked to above. Maurreen (talk) 16:08, 18 March 2010 (UTC)

Yes, this has been discussed at length... here, at WP:V, at WP:EP, and extensively at WP:BLP itself. We don't follow an RfC !vote... we follow what is actually stated in the policy. Has WP:BLP actually been changed because of this !vote... If so, then bring our statement up to date so it matches what is stated at WP:BLP (the intent is that we reflect that policy after all).... but I don't see any change in that policy to justify complete removal of the statement here. (If you want to add the word "contentious", that would be fine - as it would be fully in line with what is stated at WP:BLP) Blueboar (talk) 16:13, 18 March 2010 (UTC)

And where has the community supported your interpretation? Maurreen (talk) 16:17, 18 March 2010 (UTC)

At WP:BLP itself. Blueboar (talk) 16:25, 18 March 2010 (UTC)

My take on this is that as a general application of WP:V, unsourced material can be removed at will, though with merely questionable material--as opposed to, say, complete garbage--we usually like to see a "citation-needed" template so it has an opportunity to be rectified. With WP:BLP policy the rules are more explicit, with little or no tolerance for unsourced material that is contentious, i.e. questioned by another user for any reason. It's fairly explicit w.r.t. WP:BLP. ... Kenosis (talk) 16:33, 18 March 2010 (UTC)

To hopefully resolve this, I have updated the language used here to exactly mirror that used at WP:BLP. If BLP changes, please modify what is used here to match. Blueboar (talk) 16:36, 18 March 2010 (UTC)

Blueboar, your latest change looks good. Thank you. Maurreen (talk) 16:40, 18 March 2010 (UTC)

It's better than what it used to say, but I still don't like it, under WP:Policies, guidelines and policies should avoid quoting or summarizing other policies and guidelines. Gigs (talk) 17:30, 18 March 2010 (UTC)

Gigs, what exactly do you not like... is your only concern the fact that we quote a policy, or do you disagree with what the policy being quoted says? Blueboar (talk) 18:46, 18 March 2010 (UTC)

That we are quoting and summarizing it. To quote WP:POL (hehe): "Policies should not be redundant with other policies, or within themselves. Do not summarize, copy, or extract text. Avoid needless reminders." Even though it says "policies", it's also referring to guidelines. It seems that our policies and guidelines are slowly growing to summarize each other in a very redundant way. We should fight that urge. Gigs (talk) 20:16, 18 March 2010 (UTC)

Yes, I saw that... I think that is fundamentally bad advice. When one policy is referring to another (and especially, as in this case, where a guideline is referring to a policy), I think we should make that fact explicit by quoting and attributing the policy statement in question to the policy, as I did in my last edit here. However, the place to discuss that issue is at WT:POL, and so I have raised the issue there. Blueboar (talk) 21:00, 18 March 2010 (UTC)

^ Seglen PO (1997). "Why the impact factor of journals should not be used for evaluating research". BMJ. 314 (7079): 498–502. PMC 2126010. PMID 9056804.
^ "Not-so-deep impact". Nature. 435 (7045): 1003–4. 2005. doi:10.1038/4351003a. PMID 15973362.
^ "House of Commons - Science and Technology - Tenth Report". 2004-07-07. Retrieved 2008-07-28.

[Seglen-BMJ-1] Seglen PO (1997). "Why the impact factor of journals should not be used for evaluating research". BMJ. 314 (7079): 498–502. PMC 2126010. PMID 9056804.

[2] "Not-so-deep impact". Nature. 435 (7045): 1003–4. 2005. doi:10.1038/4351003a. PMID 15973362.

[3] "House of Commons - Science and Technology - Tenth Report". 2004-07-07. Retrieved 2008-07-28.

[1]

[2]

[3]