Wikipedia talk:Article Feedback Tool/Archive 1

Archive 1

Archive 2

Archive 3

People are dumb

Wikipedia should be asking for ratings on whether the article is understandab!!!le. I was running through an article on holographic theory, and I was like, "What in the world are they talking about?" The article should be accessible to a wide range of audiences. — Preceding unsigned comment added by 67.182.208.196 (talk) 08:10, 22 July 2011 (UTC)

Right now, I'd maybe classify that under "well-written". If we decided to add a fifth item to rate, though, this would be high on my list. WhatamIdoing (talk) 15:46, 22 July 2011 (UTC)

The writer, while knowledgeable, is not comfortable in English and assumes more knowledge than is reasonable on the part of the reader. The result is a congeries of random facts that makes little sense. — Preceding unsigned comment added by Jacomo (talk • contribs) 22:35, 28 July 2011 (UTC)

Updating ratings

Can I update my rating once I give it?

Suppose I give a good or a bad rating, it wont necessarily apply tomorrow. Wikipedia articles can be updated and change. Frankly, sometimes for the worse. If an article changes in quality, I would like to re-rate accordingly. It makes sense, no? — Preceding unsigned comment added by 98.247.162.70 (talk) 20:22, 18 July 2011 (UTC)

Yes. I've added the simple instructions to the page. WhatamIdoing (talk) 15:47, 22 July 2011 (UTC)

Right word here?

'Surface' - do you mean 'service'? --Greenmaven (talk) 11:20, 29 June 2011 (UTC)

Nope, surface is what was meant, as in "For information or facts to become known." It's as a verb. Sorry if it's too corporate-speak. Steven Walling at work 22:31, 30 June 2011 (UTC)

Hmm. Even Wiktionary doesn't give the meaning this article appears to be using. I think it's just wrong. How about "expose"? False vacuum (talk) 11:43, 9 July 2011 (UTC)

From Wiktionary: "5. (intransitive) For information or facts to become known." That is the meaning. Steven Walling at work 17:51, 13 July 2011 (UTC)

Intransitive. False vacuum (talk) 15:41, 10 August 2011 (UTC)

Template vote still on page

Is this template supposed to display the last visitor's rating? Kudpung กุดผึ้ง (talk) 14:20, 16 July 2011 (UTC)

No, but I believe that it is supposed to display your last rating. Had you rated that page before? WhatamIdoing (talk) 15:51, 22 July 2011 (UTC)

Turning off

I did what it says - ticked the box in Preferences. Damn thing still appears. I'm never going to use it, so (like adverts on other sites which I never click - and nowadays block) I don't want to see it. I still use the old Wikipedia format as the new one irritates me and I can't find things quickly. Is this the cause of my problem? Peridon (talk) 00:08, 17 July 2011 (UTC)

I can't reproduce a problem in any of the old skins. Is this still occurring? Can you clarify which skin is selected in your "Appearance" tab in user prefs?--Eloquence* 05:57, 19 July 2011 (UTC)

I'm on Monobook. It appears on quite a few new articles that I find at CSD. I don't seem to find it much on older articles, and I tend to associate it with short articles, for some reason. I've not compiled any statistics on it. I'll take notes, if you want. To be quite honest, I have doubts about the usefulness of the tool. There is a possibility of rigging, unless accounts and IPs are recorded, and I can't see the point of it being only applied at random. It has appeared this morning (UK time), but I can't remember on which article - I think I deleted it... Peridon (talk) 11:31, 19 July 2011 (UTC)

Just came up on Fusion Asset Management LLP and The Prayers of the Faithful. Peridon (talk) 16:09, 19 July 2011 (UTC)

We'll do some more testing to see if we can reproduce this.--Eloquence* 21:37, 19 July 2011 (UTC)

I"m unable to reproduce this either. Peridon, can you provide more details on your system (e.g., OS, Browser + version)? 216.38.130.161 (talk) 21:48, 19 July 2011 (UTC)

Unbranded computer, CRT monitor, XP Pro (screen display set to Classic to get rid of Big Bright Buttons for Kids), Firefox 3.5.17 (with AdBlock Plus 1.3.9 and Tab Killer). I've not been into CSD yet today - catching up after being offline a couple of days. Peridon (talk) 10:34, 22 July 2011 (UTC)

Most so far clear, but appeared on Nizami Shirinov. Peridon (talk) 11:30, 22 July 2011 (UTC)

Interesting. I left the page open while I did other things, then refreshed it (to see if someone had deleted it yet). The thing disappeared on the reload. Peridon (talk) 12:02, 22 July 2011 (UTC)

I wasn't able to reproduce it either (I've tried creating Just a test of AFT). Helder 13:40, 22 July 2011 (UTC)

Hasn't appeared tonight. Weird. Peridon (talk) 23:32, 22 July 2011 (UTC)

Back again - only seems to be on short articles and goes away on refresh. Peridon (talk) 19:47, 23 July 2011 (UTC)

Are these, by any chance, short articles that you'd previously visited? I'm wondering if perhaps it's a caching problem. Does it turn up on, say, Special:Random pages?

And I apologize for this "is your computer plugged in" kind of question, but you did click the "Save" button after ticking the box, right? WhatamIdoing (talk) 21:40, 23 July 2011 (UTC)

They're mostly articles I'm looking at for the first time (and quite often the last time, as I delete them...). It appeared on my Rover Scarab article, which I haven't been into for ages. Don't worry about the question - I ask questions like that in one of my other lives.... (Often find they hadn't done whatever it was, too... The worst was a computer that hadn't had AV updated in three years, with a little girl downloading 'free' games from all over everywhere - plus every spyware around except one.) I've just resaved - still coming up. Tried Random - came up. Why me??????????? Just in case it matters, I run Spybot S&D with TeaTimer, and WinPatrol too, on top of AVG antivirus and ZoneAlarm firewall. Peridon (talk) 22:10, 23 July 2011 (UTC)

Why you? Doubtless it's one of our computers little jokes on us all. Would you mind leaving a message at Mediawiki about this? I think the devs are watching that other page fairly closely. WhatamIdoing (talk) 23:27, 24 July 2011 (UTC)

Done. Peridon (talk) 19:47, 25 July 2011 (UTC)

Doesn't seem to be getting much response... Peridon (talk) 20:48, 27 July 2011 (UTC)

I've checked the database, and it does quite clearly say

 articlefeedback-disable  | 1  

for your user account. I don't know what's going on here either, I'll bounce it to the front-end dev. --Catrope (talk) 04:33, 29 July 2011 (UTC)

Intellectual capability, Validation and Audit

IMO, it's too dangerous to encourage the use of such an easy way for the average person to alter the credibility of a Wikipedia article. Here's why:

• Encyclopedic content must be verifiable, however, a check box does not contain enough information about the pertinence of the voter nor about his ability to construct an objective critique.
• Who validates these votes? Any person, authenticated or not, can vote, even entities that are not human (bots?)
• Votes are additive, there's no other way to revert or rectify the ranking of an article but to add more votes. The huge amount of votes induces a chaotic behavior.
• Speaking about chaotic (Brownian?) behavior, we are about a massively distributed global optimization problem where the objective function does not gratify excellence nor scientific validity. It rather gratifies satisfaction by aggregating simplicity, accessibility and vraisemblance. Complex articles with high abstraction and scientific/engineering skills requirements will be penalized. The overall quality of Wikipedia articles will converge to the exact level of accessibility of the normalized IQ of the readers (must be around 100?) When you target global satisfaction rather than excellence, it's what you get.
• There's some kind of psychological obstacle that protects articles from being profaned by people who don't have the skills to write valid and verifiable content. This obstacle doesn't exist or isn't enabled for other social media, and sure, the voting system doesn't escape to this rule due to the points explained above. This parameter sums to the objective function that served the widespread and success of Wikipedia, just don't remove it. — Preceding unsigned comment added by 145.248.195.1 (talk)

The overall quality of Wikipedia articles will converge to the exact level of accessibility of the normalized IQ of the readers.

Interesting... Helder 17:54, 19 July 2011 (UTC)

I think it's exactly what Wikipedia needs. The closer your are to something, the less you see. Those who try to edit or re-establish a NPOV in a "protected" article will now have another way to voice their opinion when they their edits are reverted. Further, does it take an expert to determine if an article is Trustworthy, Objective, or Well-written? To the author, of course they are. To the person reading it, maybe not. Logical fact (talk) 20:12, 21 July 2011 (UTC)

I agree with the objections to the voting box. People with dynamic IPs will have an ability to vote many times, and it takes up a lot of space too. Only people who want to see it should see it, and the box should be invisible on the article page by default, perhaps visible from the talk page unless a box is checked to show it in the article page. 173.180.214.58 (talk) 21:01, 22 July 2011 (UTC)

Regarding your concerns for normal IQ users, I don't agree with you. Imho, wikipedia should be a place where average users should be able understand articles. Also, I don't think that high level articles will be at a disadvantage as I don't see the average Joe reading an article about string theory and rate it low because he doesn't understand it. — Preceding unsigned comment added by 195.243.127.210 (talk) 11:23, 29 July 2011 (UTC)

Dynamic IP users are not able to provide multiple ratings. The tool has mechanisms to prevent this.--Jorm (WMF) (talk) 16:47, 29 July 2011 (UTC)

• Totally agree. The "Article Feedback Tool" should go. It serves no purpose since the perceived trustworthiness, objectivity, and completeness are totally subjective. -- P 1 9 9 • TALK 18:31, 3 August 2011 (UTC)

'Trustworthy' vs. Reputable Sources

I know articles that use reputable sources, but the articles themselves are not 'trustworthy', because the sources are cited incorrectly or cited out of context. This makes it difficult to give correct feedback using the current feedback tool. Perhaps there should be a section about accuracy and one about citations? Sushilover2000 (talk) 18:57, 19 July 2011 (UTC)

If you believe that the sources are being misused, then your trust for the article is fairly low, right? So rate it that way. There's still a whole talk page available for complex points. WhatamIdoing (talk) 15:54, 22 July 2011 (UTC)

Exactly, so why does the tool refer to the sources instead of the article itself? The definitions in the tool are flawed and don't make sense.190.19.154.181 (talk) 20:21, 22 July 2011 (UTC)

I think the definitions make sense, but that you're also supposed to use common sense. For example, there's no setting for "some top-quality sources and some completely worthless sources". This is supposed to be a very simple, big-picture rating, with all of the advantages and disadvantages that implies. Whenever the situation is complicated, you can explain your concerns on the talk page. WhatamIdoing (talk) 16:51, 23 July 2011 (UTC)

I agree. It doesn't make sense to conflate the quality of the sources cited with the quality of the article itself under a single 'trustworthiness' parameter. There should be two separate parameters for the quality of sources cited and the factual accuracy of the article.--Harumphy (talk) 10:58, 24 July 2011 (UTC)

You're assuming that the phrase "Good reputable sources" means "The article names good reputable sources" rather than "The article is based primarily on and properly uses good reputable sources". WhatamIdoing (talk) 18:16, 26 July 2011 (UTC)

This is supposed to be a measure of trustworthiness in the article itself, not its sources. The tooltip steers the reader towards the latter. This contradiction is very confusing and when I first saw it it nearly put me off rating the article at all. The problem is not in something I may be assuming, but in what the tooltip unambiguously states. --Harumphy (talk) 08:04, 27 July 2011 (UTC)

How exactly do you know that the measure is supposed to be about the "trustworthiness in the article itself, not its sources"? WhatamIdoing (talk) 17:03, 27 July 2011 (UTC)

Because it says "Rate this page", not "Rate the sources cited by this page". --Harumphy (talk) 17:26, 27 July 2011 (UTC)

The sources are part of the page. The full question says, "Do you feel this page has sufficient citations and that those citations come from trustworthy sources?" Although I think people should take a broad view, I don't really see any reason to assume that this question means something different from what it asks. WhatamIdoing (talk) 18:27, 27 July 2011 (UTC)

The tool asks to rate the page. The tool tip asks to rate the sources. This is clearly ambiguous. Choose the one that you want rated and edit the tool and the tool tips so they are in agreement. What benefit is there to leaving it unclear? Sushilover2000 (talk) 19:12, 27 July 2011 (UTC)

The sources are not part of the page. That's an absurd suggestion. --Harumphy (talk) 19:30, 27 July 2011 (UTC)

The citations are certainly part of the page, and as such are subject to our content policies like any other part of the page.

The question that the tool asks seems pretty clear to me. Deciding whether the page names a sufficient number of trustworthy sources seems like a pretty straightforward step in rating the page. It would be silly to have to re-write the headline to say "Rate certain, specified parts and qualities of the page, as explained in greater detail both below and on associated documentation pages)". Although we certainly do have a few users (e.g., some of our users who are on the autism spectrum) who find any sort of imprecision hopelessly confusing, I don't think that our average reader is either so obtuse or so rigid as to require such verbose descriptions of a fairly simple and straightforward request. WhatamIdoing (talk) 20:11, 27 July 2011 (UTC)

Citations are not sources. They're links to sources. And please stop changing the argument every time you lose it. Deciding whether the page names sufficient trustworthy sources seems straightforward to me too, but that's not the same thing as deciding whether the page is trustworthy. Are we rating the general trustworthiness of the page as a whole, or just the citations that form part of the page? They are clearly different, and your apparent inability to understand this very simple point is beginning to look like trolling. We need a clear decision about which of the two things is to be rated, and then every part of the tool including the tooltip should be worded to reflect that. The fact that you understand what you think it should mean does not mean that an end-user will interpret what the tool actually says the same way. Two end-users have pointed the problem out to you and you really show some recognition of that fact. This is my last entry in this thread, because I see no point in banging my head against the wall in the face of such intransigence and irrationality.--Harumphy (talk) 10:06, 29 July 2011 (UTC)

Apparently unlike you, I know that WP:Nobody reads the directions, so it doesn't actually matter whether we provide a precise definition. If you can't cope with the absence of precision, then please feel free to follow the directions at Wikipedia:Article Feedback Tool#Disabling and leave the tool to users who have a higher tolerance for ambiguity than you. As with anything on Wikipedia, you're a WP:VOLUNTEER, and your participation is voluntary. If you don't want to use this tool, then don't. WhatamIdoing (talk) 22:41, 30 July 2011 (UTC)

Wow. Thought better of this yet? False vacuum (talk) 15:54, 10 August 2011 (UTC)

I also think that this part of the tool is not well equilibrated. The headline is "trustworthy", so it should ask, is the article trustworthy or not? But to answer the users are asked to rate presence and quality of the cited sources. Cited sources alone are only indirectly correlated with the trustworthyness of the article. Even if 55 sources are cited, and if all are trustworthy and excellent, this does not make the article looking trustworthy if in addition to that, information is contained that is apparently not based on any sources, or if incorrect terms are used, or if the sources were cited but not thoroughly read, or if basic details had not been understood so that repeatedly published mistakes were copied from those otherwise reliable sources. Nonsense information in WP is usually based on published references. WP authors have to know which part of the published source they can rely on - the tool should ask if this was true here. -- FranciscoWelterSchultes (talk) 13:16, 15 August 2011 (UTC)

It's the wrong question. Visitors are being asked to make a deep general assessment, which most likely they are not qualified to do. Many may not even understand what WP (collectively) would like to think of as trustworthy. This is just flagrantly inviting the pushing of POV, one way or the other.

WhatamIdoing - You are expecting common sense and insight? (Demented chuckling, aside)

I quite agree with Sushilover2000's point that worthy sources do not a trustworthy article make, even before considering confirmation bias & one's own favoured cites.

What you need to ask is an honest question for an honest answer, even if it goes against the grain of our imagined objectivity:

• "Do you trust this article?"

That is something to which most visitors can probably have an authoritative response, without attempting to second-guess the beliefs of others (or what they ought to think).

Memethuzla (talk) 12:01, 20 October 2011 (UTC)

I have discovered

I have discovered an article that apparently cites reputable sources to claim nonsense. When I try to use the “trustworthiness” of the article to provide feedback on it, the tool simply doesn’t allow it. See what pointing at each of the stars says. Marcvs (talk) 15:30, 18 November 2011 (UTC)

It sounds like your concern is about the misuse of good sources, rather than the use of poor sources. The tool, especially in the current version, isn't able to handle every situation. Have you tried to fix it yourself or left an explanation of the problem on the article's WP:TALK page? WhatamIdoing (talk) 04:30, 21 November 2011 (UTC)

Is this article too long?

Some articles are incredibly long. Is it possible to rated their length? Politis (talk) 22:27, 19 July 2011 (UTC)

This is a great suggestion! Logical fact (talk) 20:14, 21 July 2011 (UTC)

I am inclined to agree with both of you, Politis, and Logical fact, though only where the writers of the piece in question have not been sufficiently concise, or have been repetitious, and the subject of the article does not in all honesty merit such a lengthy exposition as it has been given (to whit, there exists a state of disproportionableness). On further (somewhat superficial) consideration, I would perhaps be just a little more inclined to be in favour of what could be considered a slightly more profound variation on the bare assessment of "length" which you seem to suggest (by which term I hope I am correct in my assumption that you mean word-count, sheer yardage?):

that is to say wordiness, or, if one were not being too uncharitable, what one might possibly be inclined to call a style of excessive long-windedness, superfluity of expression, pleonasm. And/or, as I mentioned before, repetitiousness.

But some beasts do have a long tail. Isn't it enough that articles have a lead/lede which is generally kept brief, and a summary index, so appropriate length shouldn't present a problem.

How about suggesting a TL;DR bot?

Another thought - apart from plethoric verbalisations, reliance on obscurantist multisyllabilic linguistic units and complex grammatical constructions could be a tad exasperating, too.

Memethuzla (talk) 14:41, 20 October 2011 (UTC)

Grammar Mistake on Rate This Page Feature

Moved from Wikipedia:Village pump (technical)

 – Helder 16:19, 20 July 2011 (UTC)

Hi. The Rate this Page feature has a grammar mistake in it. After rating, a box says:"Did you want to create an account?" It should be "Do you want to create an account?" Cr6564 (talk) 22:02, 19 July 2011 (UTC)

The text seems to be defined by means of MediaWiki:Articlefeedback-pitch-join-message. Helder 16:19, 20 July 2011 (UTC)

It really doesn't rise to the level of a grammatical mistake; it's a common construction in informal, spoken English. I do agree that it should be changed (although I'd rather see the whole feature disappear quickly and permanently). Rivertorch (talk) 17:55, 20 July 2011 (UTC)

I agree with the original poster that it is a grammatical error and I have made an edit request at MediaWiki talk:Articlefeedback-pitch-join-message. Informal, spoken English be damned. We wouldn't accept that in an article, so we shouldn't in this case. – ukexpat (talk) 20:00, 20 July 2011 (UTC)

It's not really an error, but it's probably not the question they mean to ask. A correct answer to that question could be, "Yes, I did want to create an account, but now I'm so tired of sitting here that I think I'll turn off my computer and go outside." WhatamIdoing (talk) 18:18, 21 July 2011 (UTC)

Article expansion or improvement

This on the face of it seems a good idea, but what happens to votes once an article has had a major revamp? A suggestion would be once XXXX words or characters have been added old votes are cancelled. Jim Sweeney (talk) 11:48, 22 July 2011 (UTC)

 Done. Article ratings automatically expire after 30 edits to the page. WhatamIdoing (talk) 15:56, 22 July 2011 (UTC)

A thought...

Is it possible that the viewing figures on YouTube and 'like it' ratings elsewhere in social networking will now get considered as reliable indications of notability? I may be cynical, but I can see this going the way of many other measuring schemes. I can visualise Justin Bieber fans wanting to boost 'their' article, once this gets into the blogs and forums. (And similarly fans of football clubs, and people roped in by the CEO to boost their company's carefully written hidden spam article - or even to downrate their opposition's articles...) No, I can't offhand think of a better system - one that avoids the social networking connotations of 'rating'. Until someone can, leave it to Facebook and meerkats. Peridon (talk) 11:59, 22 July 2011 (UTC)

Since ratings expire after any 30 edits to the page, gaming the system would take quite a bit of persistence, especially on a high-traffic page.

I do not see any possibility of us re-writing WP:Notability to pretend that how well-written the page is determines whether the page ought to exist. There may be inexperienced people who make that mistake, but it can be addressed with all of their other common mistakes with an entry at WP:ATA. WhatamIdoing (talk) 15:59, 22 July 2011 (UTC)

Contributors

So far I refuse to use the feedback form on articles where I contributed as registered user until 2006 or as unregistered user later. If that is not as you want it maybe offer a potential bias checkbox in the form, or add a note that a bias won't harm whatever you intend to do with the feedback. –82.113.99.162 (talk) 11:40, 23 July 2011 (UTC)

Since the hope is that many people will rate the page (and the ratings are usually ignored if very few have rated the page), having one or two "biased" people is not a significant problem, just like having one or two people that misunderstand the rating system (or that don't know anything about the subject, or rate the quality of the article based on how much you like the subject) shouldn't much matter. WhatamIdoing (talk) 16:55, 23 July 2011 (UTC)

Thanks for info, I will now use it when I feel like it on more articles. –89.204.153.138 (talk) 21:29, 23 July 2011 (UTC)

Doing that it's slightly annoying that the tool always proposes a registration for intentionally unregistered users — apparently a state not exactly matching WP:LEAVE. –89.204.137.192 (talk) 08:51, 28 July 2011 (UTC)

I don't think so. Have you read LEAVE? It says, "The right to vanish is only available to users who are also exercising their right to leave. The "right to vanish" is not a "right to a fresh start" under a new identity. Vanishing means that the individual is vanishing, not just the account. Vanished users have no right to silently return under a new identity or as an IP."

So if you've actually exercised your right to vanish, LEAVE says you should not edit anything, ever again, not even as an unregsitered user. WhatamIdoing (talk) 22:37, 30 July 2011 (UTC)

[Off topic]: Of course I read it, that's why I wrote apparently a state not exactly matching. An intentionally unregistered user can disagree with a guideline like anybody else. But I didn't bother a bureaucrat to change the name five years ago, if that's what you're worried about. I moved the user cum talk page, cleaned up the redirects with a speedy, and randomized the password beyond repair. In theory you could still find the hidden (moved) subpage. –89.204.137.228 (talk) 11:47, 1 August 2011 (UTC)

That means that you did not actually WP:LEAVE, and therefore nothing at all about the invitation to create an account could possibly conflict with LEAVE. "I prefer to edit as an unregistered user" is not at all the same thing as "I am permanently leaving the English Wikipedia". WhatamIdoing (talk) 15:53, 1 August 2011 (UTC)

ACK, I'll stay away from using my 2011 commons account as sock of the long dead account here, and you'll stay away from {{!}} while it's still used ;-) –89.204.153.166 (talk) 11:36, 3 August 2011 (UTC)

Socks are normally defined per-project, not WMF wide. You may legitimately choose to work with a registered account on commons and as an unregistered user at en.wiki. You're just not entitled to claim any sort of rights at en.wiki under LEAVE while you edit en.wiki in any fashion, because if you are editing en.wiki in any fashion, then you did not actually LEAVE en.wiki. WhatamIdoing (talk) 17:11, 10 August 2011 (UTC)

Cancelling "Rate this page" box

What do I have to do to cancel the "Rate this page" box at the bottom of articles. I find them absolutely irritating. Scrivener-uki (talk) 13:07, 23 July 2011 (UTC)

Found out how to do it on the project page. Went to My preferences --> Appearance --> and checking the "Don't show the Article feedback widget on pages" box. At last, they don't appear. Scrivener-uki (talk) 13:30, 23 July 2011 (UTC)

You're lucky - I've turned it off but it won't go away - see 'Turning off' above... Peridon (talk) 19:48, 23 July 2011 (UTC)

Low traffic articles

Many of the articles that I significantly contribute to have had the article feedback tool on them for a while. These articles tend to have quite modest page view stats, in the hundreds on a good day. I am wondering whether any valuable information can come from the tool on this kind of article. One high traffic article that I watch, Albert Einstein, has had 5 ratings between this time yesterday and now. Based on yesterday's page view stats, 11.6k that makes 0.04% of the readers of that article submit a rating, admittedly a very rough metric, but the best I can do at the moment. Extending that number to a hypothetical article that receives 100 page views per day, that would mean one rating per 25 days. One rating on its own is meaningless, the power of the tool comes from having a large sample size, which might be impossible on a low traffic article. In an article with a more modest page view rate, one very bad review could mar a reasonable article, I would imagine that this would be especially relevant on articles about controversial topics.

It may be that the use of the tool may increase as people become more aware of it, they may be more willing to persevere beyond the boring references section. I think that my calculations above can be taken with a large pinch of salt, being based on one article over one day. It may be worth doing more work when everything settles down to see whether it is actually worth having the tool on low traffic articles. Any other thoughts on this? Quasihuman | Talk 14:50, 25 July 2011 (UTC)

I think it's going to be the rate of traffic-to-changes that matters. Ratings will eventually accumulate on low-change articles. It might take years, but it will eventually happen. WhatamIdoing (talk) 21:04, 25 July 2011 (UTC)

Good point, so the question is, based on the current rules, is the period over which 30 edits would be made long enough to accumulate an acceptable number of ratings. I know articles which would probably fail that, having only 10 or 20 views per day, and would still have about 30 edits per year. Are there plans to do any more rigorous investigations into the proportion of readers actually using the tool, or to determine whether the tool fails or succeeds in achieving its stated aims? Quasihuman | Talk 21:50, 25 July 2011 (UTC)

An excellent observation. It's in-line with what I have observed as well: participation is rather low for AFT. I expect the rate to get even lower as the "hey what's this?"-factor dies away. Your concern raises another important issue. The more obscure articles tend to have a higher edits-to-views ratio. So at the same time that votes are coming in, the quality of the article is changing! When somebody decides to base action on the voting, they are mis-applying the votes to the present state of the article. In a extreme instance, suppose 100 people rate article as lousy (which perhaps it is), then an editor comes along and "makes it nice". Even if this benefactor rates the article as high quality, the votes still indicate the article is low quality. What's to be done??? The votes indicate the article is in bad shape despite the article being in good shape. That is not a good thing! Scenarios such as these apparently are not being considered regarding this "tool", or at the least nobody is giving them any due weight. I really think that the WMF's Strategic Plan is usurping the tried and true editor-based model that has a 5 year proven track record. They are trying to "fix" something that isn't broken. I'm angry about it. Jason Quinn (talk) 13:12, 26 July 2011 (UTC)

Indeed, and the whole 30 edits thing is a bit of a blunt instrument, take for example, this one edit which changed an article from being non-neutral and poorly sourced, to being fully sourced and neutral. I'm sure I could also find an example where 30 edits made no substantive difference to an article.

If we are to accept the aims of the Foundation's strategic plan as valid, at the very least we should examine whether measures intended to achieve them actually work. It is all very well to propose things like AFT in the aspiration of increasing reader participation etc., but if the vast majority of readers are not using the tool after a month or two, then it has unambiguously failed, and should be dropped. I'd like to know under what circumstances the foundation would be willing to drop the tool, and what investigations the Foundation is planning on doing to see whether that line has been crossed. Quasihuman | Talk 14:03, 26 July 2011 (UTC)

By that standard, the "anyone can edit" idea has failed, because the vast majority of readers don't ever edit a page.

I think it's too early to decide whether the 30-edit bar is the right call. (I do think it would be nice if it didn't count simple vandalism-reversion pairs as two edits, but that might be technically difficult.) The primary reason that this doesn't bother me is because I'm expecting the person who takes any action based on the ratings to use common sense—exactly like I'd expect them to use common sense when considering the frequently outdated, occasionally vandalized WP:1.0 ratings (which, BTW, this process does not replace). So if you see an article with kind of low ratings that looks okay to you, I'd expect you to hit the history page first, and notice that it was improved recently, and therefore that the ratings are probably out of date (and then, I hope, to rate the page yourself, by way of nudging them in the right direction). WhatamIdoing (talk) 15:32, 26 July 2011 (UTC)

Ok, fair enough, but what standard should be used to judge the success or failure of the tool? Does the foundation have such a standard in mind? Quasihuman | Talk 20:54, 26 July 2011 (UTC)

"Anyone can edit" and "the vast majority of readers don't ever edit a page" are different things and the later does not logically imply the former. This is an unnecessary broadening of the scope anyway and I think is an unfair way to dismiss Quasihuman's points. His questions are good ones. How is the WMF determining when an initiative is a failed endeavor? Are they trying to determine that? Would they being willing to admit a misstep? The entire history of bureaucracy suggests that people are willing to avoid taking blame. I am worried that the inertia behind these decisions steamrolls direct user feedback. Jason Quinn (talk) 03:18, 27 July 2011 (UTC)

WMF has a reputation for being very metrics-driven. I don't know how they are measuring success or failure, but I am certain that they are doing it. They've got limited resources and a long list of things they'd like to be doing. As a result, they may overall be too quick to drop projects that don't show immediate success. They don't seem to play the shame-and-blame game when they drop projects: either it's working and they're happy with it, or its not, and they move promptly on to the next one.

I know that there are multiple hopes associated with the tool, so it's possible that it will fail on one and succeed on another. For example, it might produce garbage ratings, or ratings that nobody ever bothers to look at (which would presumably be "failure") but increase the rate at which new accounts are created and new people become active editors (which would be "success"). Consequently, I think that it may be naïve of us to think in digital (either it's "on" or it's "off") terms for evaluating the tool's utility. WhatamIdoing (talk) 05:52, 27 July 2011 (UTC)

I don't have any general objection with that approach, if that is the approach of the WMF. On one point, it would be a mistake to associate the increase or decrease in the rate of new accounts with the success or failure of the tool unless this is done in a very sophisticated way, correlation does not imply causation. Also, it is new editors we want, not necessarily new accounts. Quasihuman | Talk 12:34, 27 July 2011 (UTC)

The whole rating system is pointless in my humble opinion. It does not give any useful feedback at all, if that is the point of it. It gives an arbitrary 5-point feedback (what is the difference between 4 or 5 points? 3 or 4?). It is also provided by random people, and I see articles which have been translated from non-English sources being reported as “untrustworthy”, obviously by people who are not even able to judge that.

I am simply not voting because I see the system as pointless. Are low ratings supposed to make editors care more about certain articles than others? Personally, it makes me annoyed and less inclined to work on an article when I disagree with the ratings it gets, because people seem to be ungrateful for it regardless. I am sure an actual editor can on one's own decide whether an article can be improved or not. Feedback in the form of text is a form of feedback that is actually useful, as the old article quality scale as well as discussion pages provide. This system provides a lazy and useless alternative to that, and it is backed because slightly over 50% of those asked thought that it would positively improve an article. 158.37.73.38 (talk) 21:22, 6 September 2011 (UTC)

Disabling on disambiguation pages

There's no reason to rate a disambiguation page, so is there a template we can place on the page (or incorporate into the plethora of disambiguation templates) which will turn that off for these pages? ···日本穣^? · 投稿 · Talk to Nihonjoe · Join WikiProject Japan! 03:55, 27 July 2011 (UTC)

We could usefully rate dab pages for completeness. Perhaps what we need is a stripped-down version. WhatamIdoing (talk) 05:45, 27 July 2011 (UTC)

I don't see any use or value in that as disambig pages are in a constant state of flux as more items require disambiguation. It would be more useful to encourage people to just add the missing items (if any, or note them on the talk page if they are uncomfortable making the changes themselves. Having a disambig page marked as "incomplete" will serve no purpose because they will almost never be "complete". ···日本穣^? · 投稿 · Talk to Nihonjoe · Join WikiProject Japan! 06:35, 27 July 2011 (UTC)

Actually, some disambiguation templates were already blacklisted by User:OlEnglish. See the comments in the following thread:

• mw:Thread:Talk:Article feedback/Give admins ability to remove the ratings box from pages where it's useless

But I do agree that dab pages can be rated for completeness and that for this kind of page the free-text fields for readers to add dettailed comments would be very important (if not essential) to make this data useful. Helder 13:59, 27 July 2011 (UTC)

MediaWiki Page

Out of curiosity, which MediaWiki page is this located on? Thanks in advance! ---****--- Roads 23:19, 31 July 2011 (UTC)

That is, if it has a MediaWiki page. ---****--- Roads 23:19, 31 July 2011

(UTC)

Found via the feedback what is this  link: mw:Talk:Article feedback, from there you can go to mw:Article feedback and eventually arrive at mw:Extension:ArticleFeedback. –89.204.137.228 (talk) 11:16, 1 August 2011 (UTC)

Redirect

Odd, I proposed 512e on Wikipedia:Articles_for_creation/Redirects#Redirect_request:_512e, and after it was created I wanted to try something new and add the external reference for this redirect with some <noinclude>#REDIRECT magic. That failed, or rather MW complained that this is an odd idea, and I used undo to restore the original redirect. But at that point the article feedback tool asked me what I think about the two lines in 512e breaking the original redirect. Either I miss something simple (manual purge), or that's a bug, or you really want feedback for redirects. –89.204.137.228 (talk) 11:16, 1 August 2011 (UTC)

An idea

What do people think of the idea of including something like the standard deviation of the data for each category in the ratings, this would help editors judge how diverse the data is and take action on that basis. For example, a very large standard deviation could mean, among other things, that everyone is rating at the extremes, and few are rating near the mean, you might see this where there are lovers and haters of the subject of the article, and it might be safe to ignore the ratings. On the other hand, a low standard deviation coupled with a high sample size might mean that the ratings are picking up a genuine problem. Of course a low σ is not always a good thing, it could be a sign of a large number of people gaming the stats to make that article look good or bad. Quasihuman | Talk 13:40, 1 August 2011 (UTC)

I think that some descriptive statistics would be useful. I hope that eventually the data dumps will be processed into useful forms, e.g., something on the Toolserver that would filter article ratings by WikiProject, or compare them to the WP:1.0 assessments, or identify pages with high levels of ratings diversity, or compare them to ratings received by similar articles. It is likely to take a while to get such fun and useful toys, though. WhatamIdoing (talk) 16:03, 1 August 2011 (UTC)

Unable to participate in voting

Where can I report I cannot participate because my Submit link does not respond? Ottawahitech (talk) 09:23, 14 August 2011 (UTC)

Are you running any sort of script-blocking software? WhatamIdoing (talk) 17:21, 19 August 2011 (UTC)

No, I am not running script-blocking software. Ottawahitech (talk) 18:37, 28 August 2011 (UTC)

How strange. I assume that you know that it's not working because it keeps saying "Your ratings have not been submitted" even after you click the button, right? Would you please try it out at ODDD for me? At this point in time, the page has zero ratings, so it should be obvious if it works. (You might check before rating.) Please rate it and click the submit button, and then click on "View page ratings" to see whether the ratings actually show up (no matter what the display says). If it says "no ratings", then we'll know that it's not submitting. If it lists your ratings, then it might just be a display bug.

Either way, if it's still not working perfectly, I suspect that the devs will want to know which operating system and browser (including version numbers) you're running. WhatamIdoing (talk) 16:22, 29 August 2011 (UTC)

Sorry I missed the above comment. For a while I was able to rate articles, but today I noticed that my Submit button is greyed (actually light-blued) out, including at ODDD. Ottawahitech (talk) 23:20, 1 October 2011 (UTC)

The normal color of the button is a light blue-gray, which changes to gray on mouseover. The article previously had zero ratings and now has one. Are you sure it didn't work? WhatamIdoing (talk) 21:40, 10 October 2011 (UTC)

@WhatamIdoing: Try to send the page name to Ottawahitech by email or any other way so anybody else who would see it here can't interfere. --08:40, 11 October 2011 (UTC), Utar (talk)

Please change the order.

Please have a look at statistics and change the order of "Trustworthy,Objective,Complete,Well-written" to respect the frequency of usage. If it was me personally I would put "Complete,Well-written" first, because for me it is often easier to judge this than "Objective" which is very difficult to answer unless one has good knowledge about topic. --Jakubt (talk) 12:48, 15 August 2011 (UTC)

Comments

What I find surprising, even this early in the operation of the tool, is the skew towards the upper end of the ratings. It appears that readers hand out 3s, 4s and 5s (particularly 4s, I suspect) without much thought. It's surprising to find stubs that receive average ratings of 3.5+ for completeness.

I'm unsure how to encourage readers to use a bigger range of ratings (1–5) and to be more demanding. Or is it that in interpreting the scores, we should regard the difference between an average of 3 and of 4 to be critical? (That is, 3 bad; 4 good.) Should there be an implicit aim to reach 4+ in all four aspects? (NB I inserted this post at MediaWiki, too. Tony (talk) 12:59, 19 August 2011 (UTC)

I think we pretty much have to live with what our readers are going to do. How they use this scale will have far more to do with how they use similar scales on other websites, and many of the ratings probably compare a given Wikipedia page to what they would expect to see on other websites, rather than to our very best articles.

You should aim for a good article according to what you know about Wikipedia's policies, which might not be an article that readers rate highly. A perfectly NPOV-compliant article on Homeopathy is doubtless going to be rated as being horribly biased by the fraction of users who believe that homeopathy works as well as that fraction of users who believe that homeopathy is utterly worthless. The "complete" rating probably tells you something practical, but even that needs to be interpreted with care: a low rating probably means that the reader didn't find what he was looking for, but it doesn't tell you whether what the reader was looking for is something that ought to be in an encyclopedia article. WhatamIdoing (talk) 17:27, 19 August 2011 (UTC)

"You should aim for a good article according to what you know about Wikipedia's policies, which might not be an article that readers rate highly."—what is the purpose of the ratings scheme, then? Tony (talk) 02:06, 20 August 2011 (UTC)

As it says in the FAQ, the purpose is to get readers—not expert editors like yourself—engaged on the page. We want them to get used to clicking on the page, to thinking about whether the page needs improvement, and to get messages that suggest they create an account or try to edit the page. The devs have specifically said that reader assessment should not be relied upon for decisions about article improvement. WhatamIdoing (talk) 18:15, 26 August 2011 (UTC)

AFT data on toolserver

I wanted to give everybody the heads up that we are in the process of generating an index that will allow per-article AFT data to be easily queried via the toolserver. This will allow us to play with different dashboard designs and experiment with alternate ways of filtering this data. DarTar (talk) 21:23, 19 August 2011 (UTC)

Just curious: what does AFT mean? I entered it into the Wikipedia Search box, but came up with an unrelated artcile. Ottawahitech (talk) 18:44, 28 August 2011 (UTC)

Article Feedback Tool. It's a wiki-only abbreviation for a non-WP:Notable feature, so there's no reason for it to be mentioned on a dab page. WhatamIdoing (talk) 16:14, 29 August 2011 (UTC)

How are we supposed to use these ratings?

If an article gets, say a 2.3 average rating, what are we supposed to do to improve it, seeing as the raters don't list their reasons? Serendi^podous 08:34, 23 August 2011 (UTC)

You're supposed to do nothing, unless you think there's a real problem. An article might get a low rating because the readers are hate the subject, or because they're unhappy that you didn't supply a bunch of non-encyclopedic information, or because the article doesn't agree with their POV. You should feel free to ignore the ratings entirely when you make editorial decisions. We have no proof that these ratings are useful for article improvement. (They are almost certainly not useful for certain articles, like infamous politicians or celebrities.) WhatamIdoing (talk) 18:22, 26 August 2011 (UTC)

Then (and forgive me for being glib) what the heck use are they? Serendi^podous 17:12, 27 August 2011 (UTC)

Well, there's a chance that they'll be useful to editors, at least within certain topics. Comments from individual editors suggest that the ratings for articles about pop culture may have more to do with how much the reader likes the song, film, or actor than anything about the article. It's more likely to be useful for an article on a more traditionally encyclopedic article like Cancer or Queen Victoria.

The fundamental goal here is to turn readers into editors. If you think about it, the absolute minimum requirement for an editor is that they click on the page. (This is a bit like saying that the absolute minimum requirement for a career in public speaking is being able to breathe, but you've got to start somewhere.) So the devs have made something that gets more people to click on the page than any of our previous efforts (like the universal "edit" button or the tags that suggest specific kinds of edits). The tool gets readers to think about the contents of the page and to click on something. Then they (sometimes) get a message that suggests that they take another action, like creating an account or editing the article. Some people even do the tool's suggested action.

This tool turns passive readers into people who feel like they are contributing (at least a tiny bit) to the project's overall goal. It is just a baby step in turning them into experienced editors, but it seems to be a step in the right direction. WhatamIdoing (talk) 19:08, 27 August 2011 (UTC)

I don't see the difference between leaving a rating and posting a random change to a page expressing an opinion. The reason new editors don't sign up is not because we don't make it easy; it's as easy to leave a rating as it is to edit a page. The reason is because we tell the world that anyone can edit Wikipedia, then scream rules they've never heard of when they try. If you want new people to edit Wikipedia, give them a place where they can learn how without being shouted at. Serendi^podous 19:35, 27 August 2011 (UTC)

I believe there is a big difference between rating and posting to wikipedia: the first is anonymous and much easier for technically-challenged individuals to accomplish than the second. Ottawahitech (talk) 18:57, 28 August 2011 (UTC)

Serendipodous, I think you might want to read these recent comments from newbies. I don't think that these people found trying to edit a page nearly as easy as they would find clicking on a star (something they've learned how to do at other websites). The experience of the very few people with about 50,000 edits to our credit is an extremely poor predictor for the typical new editor's experience. You and I are outliers; our experience is no more relevant than a mathematical genius's experience is relevant to teaching basic addition facts to six year olds. WhatamIdoing (talk) 16:11, 29 August 2011 (UTC)

And any news on when general data about ratings in topics and within topics will become available? Tony (talk) 09:32, 23 August 2011 (UTC)

Not that I've heard. The last I heard was that it will probably happen, but I don't remember seeing any sort of timeline. WhatamIdoing (talk) 18:21, 26 August 2011 (UTC)

I believe this tool conflicts with the spirit of Wikipedia policy, and would like it removed

I am concerned that the article feedback tool conflicts with the spirit of Wikipedia's policies, and I would like to see it removed. If anything, I have a bias in favor of rating things, after all, I created RateTea.net which is a tea rating website. I like rating things and I like when people rate things. However, I think that Wikipedia is very different from the rest of the web. In particular, the official Wikipedia policy Wikipedia is not a Democracy seems relevant here. Wikipedia is a place where a vote is meaningless and is (rightfully) not counted for anything; rationale, following of policy, reasoning, and consensus is everything. The first rationale given for including this tool is "Quality assessment", but I think that this sort of assessment, voting and rating, is against the spirit of Wikipedia's policies, which emphasize discussion, consensus, and quality of reasoning. We could have 100 positive votes or ratings, but if they do not cite or fit with policy, they are all meaningless, and the 1 deviant vote that clearly fits with policy overrides it.

I have also read the rationale for this tool; one of the rationales is to, quote "quickly elicit qualitative feedback and to make more readers aware that they can directly improve Wikipedia". I think this is fallacious reasoning. We want to attract intelligent, thoughtful editors. If someone is not aware that they can directly improve Wikipedia, and we need to actively recruit them, they likely are not the sort of person who would make a good editor. People who are intelligent, curious, thorough, and good at researching will already know that anyone can edit Wikipedia.

In order to stay true to Wikipedia's core values and policies, I think it would be better to remove this tool. Cazort (talk) 17:10, 27 August 2011 (UTC)

I'm not sure if your voting/democracy analogy is apt. When you "vote" you are voting for an action to be done/not done while the AF tool seems more like an opinion survey of readers. While you're quite right that "the spirit of Wikipedia's policies [emphasize] discussion, consensus, and quality of reasoning.", you miss the important point that those policies relate to editors and not readers. Outside of our personal micro-sphere, Wikipedia is a product and our readers are the consumer of that product. They are the ones who build or destroy Wikipedia's reputation in the public sphere and, therefore, it is vitally important to get their thoughts on Wikipedia articles. We don't need to force the readers into the "spirit of Wikipedia" and require them to create an account and contribute to long discussions based on policies and consensus in order to give their opinion on a product that they are using. Agne^Cheese/^Wine 17:27, 27 August 2011 (UTC)

I actually really like the idea of getting opinions from readers who do not edit wikipedia. However, I'm skeptical of the value of gathering data or getting opinions through a widget like this one. Wikipedia already has a grading system, wikipedia editors already know which articles are good and which ones need work. But many articles are inaccessible. And maybe there's a way to give feedback on this, but...I just don't like this tool. The current tool measures: Trustworthy, Objective, Complete, Well-written. These are categories where if someone is knowledgeable enough to assess these things honestly, we probably want them editing and not just voting on a widget. What category is missing? How about Accessible. Many pages are all four of those things rated by the widget, but are not accessible...they're written in a style that is not really appropriate for the audience that's reading them. But there's no way to give this sort of feedback. This is the sort of feedback I'd like to see in a rating widget. But the existing four categories? I'd like them removed. Cazort (talk) 18:09, 30 August 2011 (UTC)

Re: "Wikipedia already has a grading system, wikipedia editors already know which articles are good and which ones need work." You're right but none of those tools tell us what the actual users of Wikipedia think. Having the 15,000 or so active Wikipedia editors think an article is good is actually meaningless when you consider that editors don't shape the public perception of Wikipedia's trustworthiness, objectivity, completeness, etc, it is the 1 mil+ daily readers.

Re: "What category is missing? How about Accessible. Many pages are all four of those things rated by the widget, but are not accessible...they're written in a style that is not really appropriate for the audience that's reading them." I think an article that is "not accessible" to readers is not really well-written. But perhaps adding a 5th category is a worthwhile suggestion to bring to the developers? However, the absence of that category is not a reason to shelve the entire tool. Agne^Cheese/^Wine 21:14, 30 August 2011 (UTC)

Personally, I would prefer, if there is a tool at all, for it to only have the category of "Accessible" (or perhaps other categories that are not yet implemented), and not the 4 currently in the tool. I think the existing 4 categories are kinda redundant and possibly conflict with / undermine the current way wikipedia works. At best, they're unnecessary and superfluous, IMHO. But in the absence of these changes, I would personally prefer that the tool be shelved. Is there any formal process for opening a discussion on whether to do this? I'd be curious if there is really a consensus to include this tool, or if it has just been forced on us by top-down structures. Cazort (talk) 14:50, 4 September 2011 (UTC)

Would you count medical students and doctors as people who are more likely to be "intelligent, curious, thorough, and good at researching" than the average editor (who is, if you aren't up on the demographics, a young, white, single, childless male)?

No, I would not. I would never assume that medical students or doctors are any more intelligent, curious, thorough, or good at researching than the average person. I personally tend to be biased against western medicine, and are skeptical of people's credentials or status in this area being a valid testimony to their intelligence, knowledge, or capability in medical fields. Cazort (talk) 18:01, 30 August 2011 (UTC)

Some of the folks at WP:MED have been making concerted efforts to recruit physicians and other licensed healthcare workers to Wikipedia. A substantial number of the people we have approached have reacted with surprise. "But I thought only certain people were allowed to edit articles at Wikipedia" and "Don't you have to register and prove your credentials first?" are not uncommon responses. While your claims might be true for students, most adults do not "already know" this fact. WhatamIdoing (talk) 19:16, 27 August 2011 (UTC)

This reaction by medical professionals does not surprise me. These people spend most of their lives within a system that very highly values status and credentials; the system selects for people who think and operate this way, and it further conditions them to operate this way and to believe that it is a good way to operate. Western medicine also tends to be characterized by fast-paced, stressful, highly focused and specialized workplaces and educational systems which do not give students or workers the time to be well-rounded individuals who would be aware of things out in the rest of the world, things like the culture and policies of wikipedia. So there are two factors I can think of that would lead medical professionals to be less aware of what Wikipedia is or how it works. Cazort (talk) 18:01, 30 August 2011 (UTC)

• WhatamIdoing, interesting that folks at WP:MED have been making concerted efforts to recruit physicians. I would like to see this discussion, because I believe it would probably apply to other professionals such as lawyers/accountants/etc. Can you point me to such discussions. Thanks in advance Ottawahitech (talk) 16:02, 3 September 2011 (UTC)

• Since it's off-topic for this page, I'll reply on your user talk page. WhatamIdoing (talk) 01:24, 4 September 2011 (UTC)

Data over time

Is it possible to see the data over time? Apparently an aim is "measuring change over time." I would like to see wiggly lines showing an improvement or otherwise in the rating. Or perhaps just the raw data - a load of dates and ratings, which we can fiddle with ourselves e.g. in Excel.

Yaris678 (talk) 12:15, 2 September 2011 (UTC)

I would also be interested in seeing this data over time - it may encourage editors to improve articles, something that is sorely needed in many articles Ottawahitech (talk) 15:51, 3 September 2011 (UTC)

I believe that the data can be downloaded from the Toolserver, and I believe that someone said each week's data was separate, but my memory might be wrong, and I'm not sure where the link is. WhatamIdoing (talk) 01:28, 4 September 2011 (UTC)

Yes, I believe (or at least personally hope) that this longitudinal data is the eventual aim of the tool. As it currently stands I think the primary value of the tool is to encourage new people to consider contributing to Wikipedia - a value that is not yet clearly proven (There is also the Extended Review concept being planned which builds on that). Personally, I see the ultimate value of the tool in providing "wiggly lines", as you neatly put it, demonstrating the improvement (or not) of the article over time. This will give great (and quickly understood) quantitative feedback to the authors of the article. If there is a place where I can vote for that feature - please point me to it! :-) Wittylama 14:19, 6 September 2011 (UTC)

I'm glad there's a number of people interested in this idea. I have posted a message at Wikipedia:Help desk#Data over time on Article Feedback Tool. Yaris678 (talk) 13:48, 9 September 2011 (UTC)

That post received no response and was archived to Wikipedia:Help desk/Archives/2011 September 9#Data over time on Article Feedback Tool. It's starting to look like there is no way to see the data over time. That kinda defeats one of the proclaimed purposes of the tool.

Yaris678 (talk) 15:13, 2 October 2011 (UTC)

I would love to show histograms for the data over time about specific ratings and articles, and in fact included such a feature in one of the earlier design documents. However, the feature was cut due to time and resources - it's a bit of a sticky wicket to have the data available without overburdening the servers, for instance. Frankly, we simply do not have the developer manpower to add this feature. The data has been saved, however, so it may be possible to do so in the future.--Jorm (WMF) (talk) 19:27, 2 October 2011 (UTC)

To be precise this feature was actually planned but see my response here. I hacked a bunch of tools to display the evolution of article rating data (along with other metrics such as citations, length, page views etc.), but I got the data directly from the database. Until the toolserver folks generate an index to filter/aggregate AFT data per article or per day it won't be possible to make an article feedback dashboard available to the community. Also note that the next generation of the article feedback tool (whose plan will be announced shortly on the blog/lists) will address a large number of issues raised on this page and on mw.org (an article feedback dashboard is among the plans discussed for phase 2 of this project). --DarTar (talk) 23:40, 18 October 2011 (UTC)

Sudden shifts in rating

I'm working on an article called Timeline of the far future. Less than a week ago it was getting 3s and 4s. Now it's getting 1s and 2s. I can't see how such a drastic shift in rating could have occurred without someone spamming it. So I would like to be able to check the rating history of the article to see when it happened. Serendi^podous 09:12, 6 September 2011 (UTC)

You've made more than 30 edits to that article in less that 48 hours. The ratings you're looking at represent only those ratings given for the last 30 versions. I'm not sure, but the "number of ratings" may be the number of ratings ever (a cumulative count), not the number being counted at the moment. If that's so, and if your readers are fairly typical, then the low ratings are probably represent only three to six people's opinions. With very few responses, it's easy for the score to vary dramatically, especially if one or two people see the article while you're in the middle of fixing a mess. I suspect that if you wait for a few days, the ratings will go back to normal. WhatamIdoing (talk) 00:07, 7 September 2011 (UTC)

VAD 43 MRC Klang Chapter

According to the Article feedback dashboard the highest rated article in all of WP is this particular social service team associated with the Malaysian Red Crescent - VAD 43 MRC Klang Chapter. It has an average of 32 ratings with an average score of 4.9 out of 5 - Impressive for an article that's been previously deleted and is without wikification, reliable sources, talkpage, wikiproject, references of any kind or any inbound links! Looks like the most blatant case of Gaming the system I've ever seen with the Article feedback tool. Wittylama 15:26, 6 September 2011 (UTC)

The page views show an interesting spike, which could mean that the article was linked somewhere (e.g., a moderately high-traffic blog), or it could be one individual playing sockmaster. WhatamIdoing (talk) 00:10, 7 September 2011 (UTC)

I think is more likely that the person who wrote the article told all the other people in the organisation to visit the article and rate it up. To be honest, especially given it's been deleted before, I think it's on the wrong side of the Notability criteria - especially lacking in RS. Wittylama 14:38, 7 September 2011 (UTC)

In any case, I think it's actually good that we caught such a gaming and can either delete the article or improve it. (P.S. I just rated it a 1 on all scales. If 30 outsiders can game the ratings, so can the thousands of Wikipedians. ;-) Steven Walling (WMF) • talk 19:19, 7 September 2011 (UTC)

Ah, that's how AFT works. Now I understand! LOL (and that from an WMF guy!) --Subfader (talk) 19:08, 20 September 2011 (UTC)

I successfully had this article listed for deletion. Perhaps we can use the AFT as a method for uncovering more blatant spam articles :-) Wittylama 00:29, 18 October 2011 (UTC)

This is actually not the first time that we see AfD nominations trigger a spike in volume and values of AFT ratings. In June the (now deleted) article Natural Constraint Language, showed an anomalous conversion of ratings per views. As it turns out, it had been nominated for deletion via an AfD on June 8 and its traffic spiked soon after that (with a peak of ca. 500 visit per day on June 15). Let's put things straight, any rating system can be gamed, but the only way to fight gaming is to get more observations, which would allow us to analyze cross-article ratings by the same user, to flag rating spam via an external referrer and to adapt the rating aggregation algorithm accordingly. All the data AFT generates is public and we are now starting an entire new iteration of the project based on community feedback and lessons learned from the first generation of the tool. --DarTar (talk) 17:01, 18 October 2011 (UTC)

In this case it was the other way around DarTar - the reason that I noticed the article in the first place was because it was the highest rated article. I then had it listed for deletion afterwards. Wittylama 22:34, 18 October 2011 (UTC)

Relevant CfD

Please see here Thanks. —Justin (koavf)❤T☮C☺M☯ 03:58, 22 September 2011 (UTC)

This is a proposal to delete Category:Article Feedback Pilot, which currently contains 4,362 articles. Is this still being used for any purpose? WhatamIdoing (talk) 04:49, 22 September 2011 (UTC)

It is not being used for anything, so may be safely deleted.--Jorm (WMF) (talk) 05:05, 22 September 2011 (UTC)

Artificially skewed ratings

I have seen pages where the ratings clearly did not reflect the actual content. In these cases the primary author may have artificially inflated the values. Hence, it may become useful to apply robust statistical methods when examining the results. Of course, that would require sufficiently large sample sizes. Regards, RJH (talk) 15:06, 23 September 2011 (UTC)

The ratings may also be out of date, if someone has dramatically improved the article within the last 30 changes, or if a vandalized version was up for a long while. Additionally, there seem to be subjects that attract ratings that may say more about how much the reader loves or hates the subject than about the article itself. Entertainment professionals and politicians seem to be targets this way. One editor has suggested that articles attractive to pre-teens are particularly prone to this.

I've wondered if a separate item on how much the rater likes the subject would be useful, because we could control for the bias in the other ratings (and perhaps by offering an outlet for that emotion, we'd get less bias). I'm not sure how you'd ask for that. Something like "Do you love or hate <name of film star>?" would probably work, but "Do you love or hate <name of horrible disease>?" would not. WhatamIdoing (talk) 21:36, 23 September 2011 (UTC)

RJH, agreed, the trouble is that – as you may have seen – the current version of the AFT widget collects a fairly small amount of data. There are roughly 40 articles per day out of the entire English Wikipedia that get 10+ ratings per day (i.e. 0.001%) and as WhatamIdoing pointed articles are likely to be revised faster than they get rated. The new iteration of Article feedback should definitely try and address the issue of the small volume of ratings that we get. WhatamIdoing, as to your second point, we were actually having a similar discussion as to how tell apart love/hate ratings from actual ratings on the quality of the article. A colleague pointed me to this interesting example. I am not sure this is an idea we want to adopt, but it's definitely a potential solution to this problem. --DarTar (talk) 17:09, 18 October 2011 (UTC)

Is this thing actually getting used?

Is this thing actually getting used? Or has usage died off now that it's no longer new? Jason Quinn (talk) 00:13, 27 September 2011 (UTC)

This kind of data is being requested on Bug 31348. Helder 14:02, 5 October 2011 (UTC)

A static plot is fairly easy to generate, unfortunately I cannot write an actual tool on the toolserver as an essential index to filter/aggregate ratings per page or by time has never been deployed on the toolserver, despite this request. --DarTar (talk) 17:18, 18 October 2011 (UTC)

Here's a first stab at the data (not including page view data yet). --DarTar (talk) 22:16, 18 October 2011 (UTC)

Great work on that, DarTar! I didn't see this reply until today. It does appear that perhaps usage is perhaps on a downward trend. Jason Quinn (talk) 19:36, 31 October 2011 (UTC)

Help!

I see everyday this template on EVERY article. How could I remove it? --Catalaalatac (talk) 10:21, 4 October 2011 (UTC)

See the following section of the FAQ: mw:Article feedback/FAQ#How do I disable the tool?. Helder 14:02, 5 October 2011 (UTC)

Crazy calculations?

Have a look at Roque de Agando. There were no ratings there until I rated it quite low. Then I started improving the article. I wouldn't normally rate articles I had done major work on, but in this case the low rating needed to be changed, so I gave it 3 or 4 stars in each category. However, the "average" rating did not increase, but probably went down. So I gave the same rating again, and the average went down again, to 1.3 in each category! I remain the only rater, according to the ratings counter. Rwxrwxrwx (talk) 10:33, 5 October 2011 (UTC)

As you can see in the data returned by the API, the values for each rating category are

"total": 4,
"count": 3,
"countall": 1

and the JavaScript determines the average by the formula

var average = Math.round( ( rating.total / rating.count ) * 10 ) / 10;

giving you the 4 / 3 = 1.3 average. Helder 13:53, 5 October 2011 (UTC)

Then either the figures are wrong or the algorithm is wrong. It's supposed to count only my last rating. In any event, the result it gives (1 rating, average 1.3) could not possibly be accurate. Also, it would be good to have a "remove my rating" button there as well. Rwxrwxrwx (talk) 14:59, 5 October 2011 (UTC)

One report and two ideas

• Completeness, grade 3: the last word of text is partially hidden behind I am highly knowledgeable about this topic (optional) (Vector, 1280x800, Mozilla Firefox)
• There should be some list of pages you rated. It would be part of "My watchlist":
  1. as another tab: The list would list all pages you have rated. or
  2. merged with it: The list would make a mark of all pages you have rated and have them in your watchlist. It will have some button like "Show me all watched unrated pages".
• For both of these lists (but standalone) I advice to make a new gadget: "Place my rated pages to watched ones too" (better for version 2). --20:59, 10 October 2011 (UTC), Utar (talk)

Thumbs Up/Down vs. 1-5 Rating

Presentation by Fabrice Florin on ideas for V5 of the article feedback tool, October 10, 2011

Recent studies have found that a thumbs up/down system may provide more useful data than a numbered rating system when rating online content. I don't have the studies in front of me but from I have read, the issue came from the reason people go to the content to view it. The claim is that people go to content (or an article in this case) because their feelings have prompted them to, meaning that there's often an inherent bias in their ratings. This results in a large number of 1s and 5s as opposed to a more expected distribution.

Is this something that has been discussed here? From reading through discussions, it looks like most topics revolve around some complaint of the system or how its working in individual cases. OlYeller^Talktome 18:09, 11 October 2011 (UTC)

It seems this is being considered for the next version of the tool. See page 5 of the presentation on the right. Helder 18:46, 11 October 2011 (UTC)

You make an excellent point. Particularly notable is the Internet Movie Database (IMDB) who have struggled with this bias. They have had to adjust their algorithms to protect against it. Very few people vote rationally and use the scale correctly. You can already see this effect at work in the ratings here at Wikipedia. Your comment underscores a point I have made several times now in regards to the AFT: surveys are HARD to interpret. I feel that the people advocating this tool do not understand this enough and that using over-simplified interpretations can lead to false conclusions being drawn. Jason Quinn (talk) 13:30, 20 October 2011 (UTC)

A number of the ratings I've seem are completely discordant with the actual content of the article. Some of the articles appear to have biased ratings in favor of an article that would not even pass the minimum criteria for page notability. In other cases, otherwise excellent articles have received very negative votes for reasons that are unclear. (An editor can't determine what aspects of a page the article caused the negative vote, nor why.) Finally, if a page has changed significantly since the votes were put in, how representative can the votes be? At present I'm tending to discount the ratings for these reasons, as well as the bias mentioned above and the statistically insignificant sample sizes. Regards, RJH (talk) 14:39, 21 October 2011 (UTC)

Expiration scheme

Most articles don't change significantly after a time span or after a certain number of edits, so I think the current expiration scheme is counterproductive, and is effectively just throwing away ratings. You're expecting people to rerate the articles periodically, but they usually won't. And if they don't then the rating relies on only the last few people's ratings; so you get an unrealistic result. Some of the ratings seem to have been influenced by only one or two people now because all the old ones have expired.

I think a better scheme would be to just average the last 'n' ratings, by keeping the last ten or twenty and averaging them.

If the article changes significantly, either for the better or worse, then the rating will start to swing and will take a while to correct for it, but that doesn't happen all that often; but right now, the rating is changing and becoming less accurate even if nobody is changing the article, just from vandalism reverts.GliderMaven (talk) 15:21, 15 October 2011 (UTC)

It depends on the article. The majority of articles get very little vandalism, so you can have year-old ratings still in play. Some articles change dramatically in just two or three edits, but all the old ratings remain active. There's no perfect solution to this problem. WhatamIdoing (talk) 18:53, 18 October 2011 (UTC)

Actually there probably are much better schemes, you can statistically prove whether the last few edits were significantly outside the distribution that had already been built up, and change the weighting scheme appropriately. So for example a couple of ratings at 1-2 when the average is 4+-2 probably isn't significant, but if it carries on getting 1-2 then the true rating has changed, and you should shorten your averaging period until it's statistically plausible again.GliderMaven (talk) 15:42, 20 October 2011 (UTC)

You are assuming that all ratings accurately reflect the then-current article's status, so that the most recent ratings more correctly reflect the now-current state of the article. In practice, it doesn't work that way. Some articles appear to attract "fan votes" or "hate votes", i.e., this well-written, well-sourced article gets low ratings because I hate George W. Bush.

We had one report of strange ratings appearing on one article because a group of activists decided at a meeting to change the article, and when their egregious POV pushing was reverted, they bombed the article with low ratings as retaliation. In those instances, the most recent ratings are less reliable and less indicative of the current state of the article than the old ratings. Your proposal to give extra weight to the most recent ratings would be exactly backwards in that situation. WhatamIdoing (talk) 17:50, 20 October 2011 (UTC)

I don't think I was assuming anything, but you don't seem to understand how bad the current system is. The current system throws away ALL ratings after 30 edits anyway, so it's EVEN more susceptible to manipulation because it keeps so few votes to start with. As in 80 people may have rated it, and the article hasn't changed much but the rating system only kept (say) 5 of the ratings. So it only takes 10 votes or so to completely trash the score. If you base it around statistics, it would take a lot more votes for the score to nose dive (because there's usually a fair amount of statistical variation anyway, and standard deviations go as the square root of the number of scores you're averaging). By way of contrast I saw an article today that was completely unrated only because it had been edited. It currently claims it has been rated 32 times, but it's obvious that there's only really been 2 ratings that matter, because all the average scores are multiples of a half. The current scheme is clearly unusably bad, we need a more robust system, even if it is still somewhat susceptible to manipulation (as with essentially all rating systems).GliderMaven (talk) 20:23, 20 October 2011 (UTC)

I don't understand how bad the current system is? Bad for what purpose? Any ratings at all appears to get non-editor readers involved in improving articles, and that's the primary goal of the tool.

You, as a reasonably experienced editor, are not supposed to be assuming that the ratings show actual problems with the articles. This has been in the FAQ for a long time: we don't know if it works that way (and we're now pretty certain that it doesn't for some subjects). You are supposed to be using your best editorial judgment, not ratings from people who usually know nothing about Wikipedia's policies and guidelines. WhatamIdoing (talk) 15:49, 21 October 2011 (UTC)

Let me put it more bluntly: you should completely ignore the ratings on articles unless and until someone proves that they are meaningful. WhatamIdoing (talk) 15:53, 21 October 2011 (UTC)

I'm standing by my comments, I reported a flaw with this tool which has been deployed across the entire Wikipedia, and you haven't addressed my complaint in any way. The implementation has a basic, obvious flaw that means it often gives bad results even when used in good faith by people on pretty typical articles.GliderMaven (talk) 17:56, 26 October 2011 (UTC)

I don't expect it to be perfect, but I expect it to at least be implemented sensibly. And this rating expiration scheme isn't sensible. It simultaneously loses all the ratings but claims an article has been rated over 30 times. That's a bug, right there.GliderMaven (talk) 17:56, 26 October 2011 (UTC)

Suggested revision

Too Long ; but Definitely worth a Read ? (-;

With apologies to the designers: good effort, well-meant, I hope the idea succeeds, but I am not happy with this current implementation. It seems unfocused & prone to error.

There is ambiguity & overlap of the categories.
"Trustworthy" & "Objective" are bound to be conflated in some sense, according to one's sublimated point of view.
"Objective" is described as a balance of all views, which is not the same as NPOV.
A beautifully-crafted polemic could be "well-organised & well-written", but would it really constitute a "well-written" WP article?

Is it clear, in underlying purpose, what we are rating and why? Presumably we need a rating to lead to an action to lead to a course of improvement, to set an agenda.

Suppose an article is adjudged not objective, for example. It might draw attention to an established "owning" clique, where such exists, but what are we to do about that, more than we already do? It's a pertinent question, but it's unclear where we go with it.

(Personally, I would want to insist that opinionated views are shifted to referenced stubs, banished from the sheerly descriptive main article.
Main page - an exposition of the subject here; ardent detractors, please go write on critics' stub; supporters, go write on defenders' stub; discussions on discussion pages, please.
But that isn't how we go. And it's away from this topic.)

So, in the current set-up, all-comers are invited to rate, whether registered & well-versed in WPedianism or not.
It is therefore some kind of popularity contest, and one cannot expect the usual WP standards to apply. Some might hanker for that, but one cannot reasonably expect it.

In these circumstances, it seems to me that the questions are badly formed.

It is close-on pointless to expect a casual visitor to have the expertise to make a deep general assessment of the nominated categories.

• Trustworthy----do you feel that this page has sufficient citations from trustworthy sources?......................(Shifting the focus of trust?)
• Objective------do you feel that this page shows a fair representation of all perspectives on the issue?.....(According to your own confirmation bias?)
• Complete------do you feel that this page covers the essential topic areas that it should?..........................(Are important things missing?)
• Well-written---do you feel that this page is well-organised & well-written?..................................................(Is it well presented and accurate?)

Each one, note, "how do you "feel" about an objective matter?". Confused. (Sorry. Just Confused.)

Suppose I am a casual reader, impelled by some strange compulsion to rate a page. A box is there, I must tick it.

I can opine, or I can say what I, personally, feel about these categories. That's easy enough, I'm close to expert on my own feelings.
But then, each of these categories requires an objective assessment. That's at odds with asking my feelings.

Now suppose the page is about some celebrity figure, Justin Bieber, Margaret Thatcher, or Carl Popper?
Fans may be inclined to disregard what is actually being asked of them, and give the article a high rating throughout (unless the article is antagonistic).
Likewise, those who dispute the worth of said celebrity, or what they stand for, may be inclined to mark the whole page down.
If I'm really keen, & have pals of the same mind, I might even organise a multiple click-fest. Interesting socialogically, but not useful.

The same might apply to any subject of contention.
In fact, the first time I came cross this rating system - and tried to honestly rate the each part - I found myself compelled give a much higher rating to the individual aspects than I felt the whole article deserved. That felt unsatisfactory.

No doubt this was not the intention of the designers, but I believe a poor outcome is highly likely, & unhelpful.
(OK, maybe that's ORiginal. So is the current rating system. I can't be bothered to back it up, but that's what I feel.)

Incidentally, though I firmly resisted the temptation to rate dishonestly, I can't say I really trust the rest of you... ;-).

What can one do to gain more reliable scores? Casual readers are not encyclopaedists, remember. Give them a lecture? I don't think so.

As a counter to error, I suggest this :

• a) Whatever questions are asked should help provide an agenda for editors. (I think that's almost been done.)
  
• b) Since we cannot expect objectivity, let's be honest that we are asking about personal feelings and assessments, and run with that.
  
• c) Defuse cheerleading pro- or con- factions, and provide a statistical qualifier, by asking the pertinent "supporter" question first.
  

May I suggest these more direct alternatives, just as a guide, in a form which may look similar, but which I feel have very different effect:

• Focus...................How can we improve this article?
• 1 - your opinion-----Do you approve or disapprove of the central subject of this page?
• 2 - accuracy---------Do you believe this article is accurate and reliable?
• 3 - completeness---Are important relevant facts missing? (Please sign in, & share your knowledge.)
• 4 - style---------------Do you find it well-presented, articulate & clear?
• 5 - bias----------------Does the page seem fairly focused on the subject itself (good), or on opinions of the subject (bad)?

(The actual rating values are left as an exercise for the reader.)
Here, I'm attempting to make direct requests for honest personal opinions rather than invite unqualified wider judgements.

I feel (1) is bound to affect the others, but if we had it, at least we could measure.
The technicalities of (2) can be left to superior intellects, editors & bots.
It would be incomplete without (3).
(4) seems concise and to the point.
I find (5) most problematical to concoct, because of our faulty human cognition. (Responses are likely to be distorted by confirmation bias, cognitive dissonance, firm beliefs, suspended disbelief, Semmelweis effect, faith or the lack of it, sublimation, indoctrination, propaganda, groupthink, schadenfreude, competitive advocacy, convergence, megalomania, psychopathy, etc., etc., etc.) If one can't rely on categorical imperative, one can only appeal to common decency.

In the light of which, perhaps I should add (6): "And should we change it?".

Memethuzla (talk) 08:52, 24 October 2011 (UTC)

Hey Memethuzla. Thanks for taking time and thinking of a better way to present this tool. You've obviously put a good amount of thought into it and I appreciate you doing that for WP.

The problem is see is that this change, while good, won't solve the problems that are inherent with the current system. There's a lot to know about statistics in that there's a whole lot of ways to screw up data taking, take the wrong data, and analyze the data incorrectly. Until we address those problems, I don't see that any measuring change will make the data any more useful than it is now (which is to say, not very useful at all). We're not even to the point of achieving statistical significance let alone analyzing correlations with any sort of accuracy.

I hope you're join the office hours session tomorrow at 19:00 GMT/UTC in #wikimedia-office to discuss completely changing the system. OlYeller21^Talktome 15:19, 26 October 2011 (UTC)

So, Memethuzla, let me ask you your first question:

Do you approve or disapprove of cancer? Do you approve or disapprove of multiplication? Do you approve or disapprove of the Moon?

Do you see how inappropriate this question is, once you find the parts of the encyclopedia that aren't about popular culture and other matters of personal preferences? WhatamIdoing (talk) 17:29, 26 October 2011 (UTC)

My first question? That was, "Is it clear, in underlying purpose, what we are rating and why?". Are the questions clearly separated, do they lead to a clear agenda?

Did you see my point (*c)? That's why it's "appropriate" to ask. You might be surprised what people are passionate about, subjective about. My suggested first category was intended precisely as a "spoiler", hopefully to get the personal bias, the knee-jerks, over with. I'm not alone here in seeing problems with "fan club" voting (and not just from the Face-book generation). I found some cracking examples; they're in this page's edit history (TL).Memethuzla (talk) 18:58, 27 October 2011 (UTC)

I fully understand the significant problem you're trying to solve. Do you understand how stupid we would look if we asked people whether they approved or disapproved of cancer? WhatamIdoing (talk) 19:45, 31 October 2011 (UTC)

Thanks OlYeller21, I'll try to sit in. Memethuzla (talk) 18:58, 27 October 2011 (UTC)