Wikipedia talk:WikiProject Good articles/Reform

Initial comment

This page is intended to facilitate discussion of changes to the GA process. The front page is a place to present the issues, and ultimately, proposed changes, but I recommend that most discussions (ideas, proposals, thoughts) should take place on the talk page first. Geometry guy 19:28, 28 May 2008 (UTC)

Process

Maybe it would be useful to start by agreeing on an order to take these subjects in, to help avoid dilution and confusion on this page. I think possibly the review process or the GA criteria would be logical starting points, but what do others think? EyeSerene^talk 08:59, 30 May 2008 (UTC)

I'd probably vote for analyzing the review process first, since I personally don't have any major issues with the GA criteria. We did just implement a new system for moving GA reviews to subpages of article talk, which ties in to the process, so it might be good to start off with seeing how this is working so far and such. Also, I'll be reviewing GA reviews over the last two weeks for the 'reviewer of the week/month' award this weekend, and I plan to take another look especially into how well this transition to review subpages is going. Dr. Cash (talk) 14:59, 30 May 2008 (UTC)

Yes, those are pretty much my thoughts too - the criteria themselves seem reasonable to me, and I think any significant changes in them would be counterproductive and possibly even disastrous (thinking of Nehrams's 230ish swept articles that could end up back in the sweep queue...) EyeSerene^talk 16:17, 30 May 2008 (UTC)

AGGHH! Please don't change the criteria now! Just kidding, whatever we have to do to improve the process is fine, although I'm sure the articles swept by me and everyone else are still going to qualify as good articles if we were to change the criteria. I think the review process would be best to go over first as well. --Nehrams2020 (talk) 18:56, 30 May 2008 (UTC)

process first please. Let GA then stand for scrutiny then maybe a few changes might be necessary, might be though! I have yet to be convinced, but giving doubters etc the opportunity to follow GA reviews will hopefully answer many of their questions. Edmund Patrick ( ^confer _work) 20:08, 30 May 2008 (UTC)

Agreed. The "green dot debate" indicated that some in the wider community had concerns that that GA criteria were not suitable, since "substandard" articles were passing. I think it's more likely that the criteria is, by and large, appropriate, just not always applied rigorously (ie process). So, that's a good place to start; sort this out and we might find everything else slots into place with miminal hassle.

So, then, we have criteria for GA articles. We have articles submitted for assessment according to the criteria. What process would allow these articles to be measured against the criteria, in a consistent and appropriate way? Do we feel the current process is pretty much what we want (with minor tweaking), or does it need a complete overhaul? What do we think is the best way to review? Let's throw some thoughts down. Gwinva (talk) 20:24, 30 May 2008 (UTC)

How can we ensure that the community will accept the new process? OhanaUnited^{Talk page} 20:28, 30 May 2008 (UTC)

(remove indent) we cannot, we must just put it out there and let it stand, it is such an improvement that I think it will. ½ the points raised were I thought bought about by the "unaccountability" of previous GA reviews as well as the standard of those reviews. In a sense I felt one could not disprove the point because of the nature of the existing process. With sub pages of "contained" reviews hopefully some of the point can be answered by saying here is the history, here is the discussions! Then the criteria might need to be looked at. i will say that I feel that the subpages are a great step forward, one of those things that lead one to say "why didn't wew do this ages ago". Edmund Patrick – confer 20:38, 30 May 2008 (UTC)

We are merely here to thrash out ideas and create a proposal, which we will present to the community at large. The general feeling was that "something" needed to be done, but no one knew quite what. (Or, rather, a lot of people had ideas, but no conclusions were reached.) Here are some questions to get the ball rolling. (They are not original, but concerns/ideas I recall being raised during various discussions.) Do we like the one reviewer idea, or should it require more? A second opinion? An open process? Should it have a set time (ie no quick passes or fails)? Can anyone review or do they need approval? Organised review teams? A panel to approve the status once review complete? GA criteria checklist, or detailed reviews? → What do you guys think? Gwinva (talk) 22:56, 30 May 2008 (UTC)

OK, we seem to have consensus that we should be looking at the review process itself first. I've had a go at listing below each suggestion that has been raised here and elsewhere, in no particular order. As Gwinva says, everyone has their own ideas, so the list is unfortunately rather long (and I've probably missed some). I suggest we give a few weeks for comment, so that all interested editors can have their say, then start to winnow down the ideas into a few concrete proposals. EyeSerene^talk 08:24, 3 June 2008 (UTC)

Review process proposals

This list is not intended to be exhaustive, but as a starting point for discussion. Please comment, add and/or edit as you see fit.

No change

Having introduced review sub-pages and gone some way to addressing transparency concerns, are further changes still necessary?

Reviewer mentoring

Do we need to provide formal training to reviewers?

Comment - I like this idea. What I've been doing with the handful of GA reviews I've been doing is getting Giggy to check them over once I've finished to make sure I haven't missed anything, although sometimes I'm nervous about being overly harsh or critical. It's worked quite well for me and I'd reccomend it to anyone. Gazimoff ^Write_Read 15:54, 7 June 2008 (UTC)

Reviewer approval

Should only 'approved' reviewers be permitted to carry out reviews?

• Comment This would be the ideal solution to ensure consistent quality, IMO, although naturally it has potential problems. However, perhaps combined with reviewer training or mentoring, it would be possible to maintain a list of 'qualified' reviewers that are willing to be contacted on their talk pages to carry out reviews (much in the same way as the WP:PRV volunteers list). This could be instead of, or in addition to, posting nominations at GAN. EyeSerene^talk 09:07, 3 June 2008 (UTC)

• Comment I'd be more comfortable doing something like requiring two or more reviewers to sign off on a GA approval. I have found that when I have done GA review, the bad ones are easy to fail, but I don't think I've ever passed something I've done review on without asking someone else's opinion first anyway. Montanabw^(talk) 21:47, 4 June 2008 (UTC)

• "Comment2' I am not sure I like the concept of "approved" reviewers, would hate to have the RfA criteria applied to GA reviewers, who seem to be in short supply anyway, but on the other hand I also do think that there should be some way a reviewer does need to prove that they actually read and understood the criteria, maybe having an "apprenticeship" period where they can review articles, but need that 2nd opinion to pass. So, maybe not so much approval as training? Montanabw^(talk)

• Not a bad idea to have "apprenticeships". After so many reviews (needn't be high: just enough to see they've understood the citeria and there aren't any glaring issues), they can become "full" or "approved" or whatever. Gwinva (talk) 22:14, 4 June 2008 (UTC)
• I think this entire debate boils down to the question "how can we have faith in our reviewers?". There seem to be two basic solutions emerging - we either supervise them more closely (with open reviews, oversight or some such), or provide some sort of training then trust them to do a decent job. My preferred option is the latter, because it retains the one-reviewer, collaborative, unbureaucratic system that I believe is our greatest strength, and just seems more 'wiki' to me. EyeSerene^talk 10:37, 5 June 2008 (UTC)

Standardised reviews

Would changing the way the reviews themselves are formatted help with quality and consistency?

• We want to encourage detailed, helpful reviews, and discourage tick-boxing. Set formats might encourage the latter. But if they could be developed to encourage the former, then it might be worth considering. Gwinva (talk) 22:16, 4 June 2008 (UTC)

Multiple reviewers

Should reviews be carried out by more than one reviewer?

• Comment Reviews are invited by more than one reviewer under the current system, but it rarely happens from what I've seen unless someone requests a second opinion. Personally I'm not convinced about making this mandatory - we'd need to be careful about slowing down the process and making the backlog even bigger. I also think we should maintain GA's major selling point: the collaborative nature of GA reviewing, which may be diluted by multiple reviewers. EyeSerene^talk 08:49, 3 June 2008 (UTC)
• I like the idea of combining this with the "Reviewer approval" option... those who haven't been around GA for as long could take a review and have it "certified" (this sounds really formal but we can work on those kinks in time) by one of the more experienced folk, so two reviewers look at most (if not all) articles. dihydrogen monoxide (H₂O) 09:53, 3 June 2008 (UTC)
• I think I like the idea of a reviewer approval, though who becomes one and how could be fraught with difficulties. I am never sure if anyone followed my request for a second opinion, if they did presumably they liked what they saw and left. Edmund Patrick – confer 17:46, 3 June 2008 (UTC)
• I checked with the GA promotion process in a few other top-20 languages and they seem to have the multiple reviewers system. I suggest we can follow suit. OhanaUnited^{Talk page} 15:46, 4 June 2008 (UTC)
• Some kind of certification or approval process may be worth introducing, but I would be very much against the FA voting style of review. That would be the final nail in GA's coffin I believe. --Malleus Fatuorum (talk) 16:03, 4 June 2008 (UTC)
  • But how would we handpick those individuals for the certification process? OhanaUnited^{Talk page} 17:15, 4 June 2008 (UTC)
    • How were the first administrators picked? What about any editor who's reached some arbitrary number of reviews, 50 for instance, to kickstart the process? --Malleus Fatuorum (talk) 17:31, 4 June 2008 (UTC)
      • I have the same reservations as Malleus about creating an FA-lite system. However, the problem with using appointed/elected GA promoters (per Raul & Sandy at FA), is if we don't combine it with some sort of review voting process, any GA promoters won't be so much interpreting consensus, as acting on their own judgement. I don't think we want to end up with the perception that GA rests solely on the personal opinions of a few individuals. We could perhaps have an approvals panel, the membership of which rotates regularly, but I'm still unconvinced that's the best way to go. EyeSerene^talk 17:35, 4 June 2008 (UTC)
        • I'm very much against the idea of appointed promoters. I'm not even sure it always works that well at FA, never mind GA. This all, I think, echoes something that Geometry guy and Ling nut were discussing elsewhere. What is the purpose of GA? If it's to get as much of the encyclopedia as possible up to some minimum standard as quickly as possible, then it needs to be as process-lite as is consistent with some minimum qa. If it has some other purpose, then perhaps a more formal process needs to be considered. --Malleus Fatuorum (talk) 17:45, 4 June 2008 (UTC)

What we seem to like about one-article-one-reviewer is that it's much more efficient than having multiple people read the same article, and as long as we educate each other and look over each other's shoulders, we can probably trust the review if we can trust the reviewer. I am fine with the way we do it now; I would also be fine with any division of labor that doesn't involve significant overlap of responsibilities. Some people are better with process, some with language, some with content. - Dan Dank55 (talk)(mistakes) 19:05, 4 June 2008 (UTC)

I'd prefer we not have appointed reviewers as well. We do want to prevent that a new reviewer unfamiliar with the process doesn't approve a review by another inexperienced reviewer. This could rarely occur, but could still allow some articles to pass when they shouldn't. It might be best to ensure that the second reviewer has at least a minimum amount of articles they've reviewed to be able to serve as the second reviewer. --Nehrams2020 (talk) 19:07, 4 June 2008 (UTC)

I strongly feel that there are multiple reviewers. I find it rediculous that an article makes a semi-important list when there's only been the nominator and the possibly POV reviewer supporting an article. It's likely good, but there needs to be multiple reviewers. Reywas92^Talk 19:36, 4 June 2008 (UTC)

If I can pull something out of left field here, a recent (cough) contentious RfA has been generating a lot of discussion. My take is that humans like to have some kind of reasonable tests to pass, after which they are examined by their community and pronounced ready to "belong". My take with discontent with the RfA process is that the bar is set way too high for the RfA process to perform that function, so either the bar needs to come down a bit, or else people need to have other processes that are RfA-like but apply to smaller and more tightly-knit communities. Conceivably, the community of GA reviewers might benefit from an RfA-like process, and if that's true, then as a bonus, we maintain the efficiency of the current review process (one reviewer per article) while adding oversight and reliability that many people would like for GAN to have, rather than just saying "it's okay if stuff goes wrong, GAR will fix it". - Dan Dank55 (talk)(mistakes) 20:48, 4 June 2008 (UTC)

I think I broadly agree with all that. I hadn't considered the option of having some kind of RfA-like process to determine who can pass articles, perhaps due to my distaste for that process. But, if done properly ... perhaps. --Malleus Fatuorum (talk) 21:37, 4 June 2008 (UTC)

• Support I strongly support the concept of having two or more reviewers, but no more than three. Note my comments above in the "approved reviewers" section. Montanabw^(talk) 21:49, 4 June 2008 (UTC)
• Strong Oppose. Well, if we've started voting ... :-) --Malleus Fatuorum (talk) 21:50, 4 June 2008 (UTC)

Comment I've striken the two support & oppose votes. There is no vote going on in this discussion, yet. Dr. Cash (talk) 04:01, 5 June 2008 (UTC)

• I wouldn't like to see a voting system for GAs. The strength of the GA system is the collaborative dialogue which can take place between the reviewer and editor(s). So we need to place the emphasis on one lead reviewer. However, there are occasions when other opinions are useful, such in the case of novice reviewers, or where specific expertise adds another dimension. We also need to draw on the strengths of those involved in the project: some are good on image policies and reliable sources, but not so good at picking up copyediting issues (for example), others know physics inside out but wouldn't know where to start with TV. We also want to avoid situations such as inexperienced editors passing sub-standard articles (or those with glaring problems, such as image use), editors passing friends' articles in a "you scratch my back" kind of pact, driveby reviews, inconsistent application of criteria, poor understanding of subject-specific requirements etc etc, (ie. the current weaknesses) while still allowing/encouraging the GA strengths of collaborative dialogue and neutral non-wikiproject community recognition. I think a number of the issues overlap (time limits, reviewer approval, and so forth) but can be combined within one approach. I will outline a proposal of this kind below. Gwinva (talk) 22:54, 4 June 2008 (UTC)

Proposal 1

Proposal: When a reviewer selects an article to review, they effectively "open" the review (tagging it on GAN page with an appropriate tag), and become lead reviewer, carrying out a review in much the same manner as now. But the review stays open for a minimum period (say, 5 days). During this period, others may also add comments (constructive comments, not support or oppose votes).

These people might include: novices, who wish to begin their reviewing by contributing to open reviews; specialists, such as a scientist who has a quick look at every open scientific article to spot any glaring problems; experts, such as a copyright expert, who regularly goes through open reviews double checking image use tags; time-poor GA reviewers, who haven't time to contribute a full review, but want to be involved by offering helpful comments; mentor, who has agreed to oversee a new reviewer's first few reviews; concerned reviewer, who has some concerns about the approach of the lead reviewer and wants to keep an eye on things; 2nd opinioner, from a list of "senior reviewers" who has responded to a lead reviewer's invitation to provide a 2nd opinion in a difficult case (senior reviewers targetted especially, rather than vague "2nd opinion requested" tag on GAN page); random passer by, who spotted the GAN template on the talk page. But there is no requirement that others must contribute before a review is completed.

Once the minimum period has passed (regardless of how many have been involved in the debate), and issues have been resolved in some manner, then the "lead reviewer" closes the review, passing or failing the article accordingly. The lead reviewer should know the article and discussion well enough to make an appropriate judgement.

Explanation: This actually requires little change to the process. Many reviews, especially those carried out by experienced reviewers, will carry on much as they have been. But it allows for greater hand-holding of newbies, quality control, an elimination of "driveby" quick passes, an encouragement of small contributions, and a forum for using reviewers' strengths and expertise. Gwinva (talk) 22:54, 4 June 2008 (UTC)

• Comments:

• That seems like quite a sensible and workable proposal to me. --Malleus Fatuorum (talk) 23:01, 4 June 2008 (UTC)
• I have some reservations. Assigning the title of "lead reviewer" to whoever that opens the review could lead to argument and drama. It promotes a few individual to "hog" over and start a large amount of reviewers (e.g. more than five) at the same time just for the title of lead reviewer. It also leads to struggle for power. What if the lead reviewer disagrees with senior reviewer and both of them are not willing to compromise? We cannot let the lead reviewer to have a trump card and ignore everyone else. And although this process does have some quality control, it does not stop a determined user from promoting a poor-quality article to GA (especially when you say "there is no requirement that others must contribute before a review is completed"). I only feel that placing such a roadblock hinders the overall promotion process rate but do not see a large improvement in QC. OhanaUnited^{Talk page} 03:55, 5 June 2008 (UTC)

• "although this process does have some quality control, it does not stop a determined user from promoting a poor-quality article to GA" - nor does the current system. Nor does FAC, or any other process on Wikipedia. What are you proposing? giggy _(:O) 05:11, 5 June 2008 (UTC)

• I don't like the mention of "time poor GA reviewers". We shouldn't be promoting such "quick looks" at articles by reviewers that don't have time for a full review. This will only lead to sloppiness. You either have time to review the article or you don't. Dr. Cash (talk) 04:05, 5 June 2008 (UTC)

• The "lead reviewer"—he who does the main review, the bulk of it (I'd hope), wouldn't be the time poor one. That could just be if, say, you've got five minutes, and you notice a few typos that weren't brought up in the GA review. You add them on to the review because you don't have time to fix them yourself. giggy _(:O) 05:11, 5 June 2008 (UTC)

• Like Malleus, I like this idea. giggy _(:O) 05:11, 5 June 2008 (UTC)
• Our strengths under the current system are collaboration (especially in the 'training' of article editors), the personal touch, minimal bureaucracy, and (relatively fast) throughput. I know I keep banging on about it, but I think these are so important that, if our chosen solution minimises any of these, we'll do terrible harm to the project. Hence this proposal seems to be potentially the best so far, as it plays to our strengths - only the speed of throughput might be affected by doing away with quick-passes and fails, but I don't necessarily see this as a bad thing. Perhaps we could integrate reviewer training/mentoring into it somehow - maybe by asking new reviewers to sign up and assigning them a mentor, who would drop in on their first few reviews and give guidance where needed. I have a suspicion that in practice this system might still end up as one reviewer per article, but as long as they are trusted reviewers that's not a bad thing either ;) EyeSerene^talk 16:37, 5 June 2008 (UTC)

• I'm very much in agreement with what you say EyeSerene; we need to play to our strengths more than we need to pander to our critics. --Malleus Fatuorum (talk) 00:29, 7 June 2008 (UTC)

• Like Malleus, I like this idea. Very happy with the proposal. Edmund Patrick – confer 16:51, 5 June 2008 (UTC)
• This proposal actually makes a bit of sense. Though I would state it as keeping all articles 'open for review' for a minimum of three days. The person that initiates the reviewer makes his/her comments on the article, and anyone else that wishes to offer comments on it is free to do so. If no one else offers comments after three days, though, and the reviewer is satisfied that the article meets the criteria, then he/she is free to pass the article. If there's still a lot of work to do, keep it as-is (effectively "on hold", but the review is still "open"), or fail it. So effectively, we'd kill "quick fails", though if an article falls seriously short of the criteria, the nominator would have three days to address the issues and then it would just fail after three days.

Also, I'm not sure if we want to establish a heirarchy in reviewers, but we might want to strongly encourage newer reviewers to not initiate reviews on their own, instead focusing on making comments on open reviews, until they become more familiar with the criteria. Dr. Cash (talk) 17:22, 5 June 2008 (UTC)

This looks like a good idea, and like Dr. Cash, believe that a shorter period of three days would be better. If this was implemented, would we be able to create a subpage that automatically lists all of the articles currently on hold, so that reviewers could be better informed about what articles are open to a second look? I'm sure it could be updated by a bot. --Nehrams2020 (talk) 23:47, 5 June 2008 (UTC)

• I like it. - Dan Dank55 (talk)(mistakes) 23:54, 5 June 2008 (UTC)
  • P.S. This would dovetail nicely with some kind of vetting process. If someone fails even a lightweight vetting process, they could still help out, and also demonstrate that they're moving in the right direction, by helping out in reviews that don't "belong" to them. - Dan Dank55 (talk)(mistakes) 00:19, 7 June 2008 (UTC)
• I like the idea of supporting people who are new to the GA process and encouraging them to get a more seasoned reviewer to double-check their work. Gazimoff ^Write_Read 15:59, 7 June 2008 (UTC)
• I like this proposal. I don't see an issue with calling the initiator of the review the "lead reviewer". That doesn't come off as a hierarchy to me. Anyone can be a lead reviewer. It's just a matter of who chooses to initiate reviews and who chooses to help out with open reviews. The only "power" given to the lead is the responsibility of closing. Lara❤Love 19:06, 7 June 2008 (UTC)

• Great idea! The only problem I think is that the time is too long. 3 days ought to cut it as a minimum. If something either really, really good or really, really bad, IAR can lengthen or shorten the time as needed. bibliomaniac15 22:00, 8 June 2008 (UTC)
  • I don't have a preference how many days it is, as long as it's long enough so that everyone who wants to play some role in other people's reviews won't miss out because the window is too short. I would think a review that lasts only one weekend wouldn't be enough time to give everyone a chance to look at it, so 3 days seems a little short. - Dan Dank55 (talk)(mistakes) 22:46, 8 June 2008 (UTC)

I have made some modifications to the GAN review guidelines and put them on this temporary page. It essentially outlines a revised review process keeping all GA reviews open for a minimum time period of three days, or allowing a review to be passed/failed earlier if at least three reviewers agree on pass/fail. It also deprecates the on hold & second opinion statuses, letting reviews stay open for a longer period, if necessary (I put five to seven days here, but I don't think we want that to be a hard and fast rule).

If we implement this new procedure, the {{GAReview}} template should be modified to say something like, "Open Review – This review has been initiated by Dr. Cash (talk) 16:25, 9 June 2008 (UTC). Additional comments are welcome by other reviewers for three days." I would also recommend changing the icon used by {{GAReview}} to the same icon that is currently used by the "on hold" template, since it is more visible. Perhaps G-guy could figure out a way to link directly to the open review page from the link using some of his template magic? Also, if we implement this, all current on holds & 2nd opinions would be changed to the {{GAReview}} template.

Let me know what you think. Dr. Cash (talk) 16:25, 9 June 2008 (UTC)

I would have to disagree with the proposal, especially the requirement for three reviewers before an article can be quick failed. It gets too close to making GA like FA, and will leave GA clogged with articles that should be straight failed for massive issues. -- AnmaFinotera (talk · contribs) 16:30, 9 June 2008 (UTC)

3 to quick-fail? That's not quick enough. 1 is enough to quick-fail, but I would say that the decision of quick-fail can be overturned in case of mistake or abuse. OhanaUnited^{Talk page} 02:49, 10 June 2008 (UTC)

The proposal looks fine except for having three reviewers in allowing a quick-fail. A single editor can determine this, and would only occur if the article was in such bad shape that it was no where near passing the criteria. I would think leaving it to one person is best, and as OhanaUnited stated above, we could allow for overturn with GAR if necessary. Quick-fails are not too common, but we don't want to continue to bog down our resources with requiring three (or even two) editors focusing on determining if an article merits a quick-fail. --Nehrams2020 (talk) 03:17, 10 June 2008 (UTC)

3 to quick fail is too many. I also think the 3 needed to pass is a bit much... I guess it depends who's reviewing, but in most cases 2 should be fine (what say we IAR? ;-)). giggy _(:O) 09:17, 10 June 2008 (UTC)

When the proposal is finalized, can someone type it out so that we can review the terms? I say that there is a lot of discussion and sometimes people (including me) just want to get straight to the point without the need to read an entire section. OhanaUnited^{Talk page} 14:14, 10 June 2008 (UTC)

Just to clarify, I think the above is referring to three days, not three reviewers, for article assessment. EyeSerene^talk 22:16, 10 June 2008 (UTC)

That is partially correct; the proposal calls for either three days to elapse, or three reviewers to agree that the article does or doesn't meet the criteria. So if three reviewers all come together and agree within one day to pass/fail the article, it's done. Though we could easily go with two reviewers just to make sure you get more than one person's opinion. I just went with three days/three reviewers for consistency here. Dr. Cash (talk) 23:18, 18 June 2008 (UTC)

• I like the proposal. It subsumes the quick pass/fail process, which I have always found suspect, and simplifies the pipeline — there is now only one review process. The concept of keeping an article 'on hold' is subsumed by the notion that a review is open for a period of time. That other editors may participate in open reviews subsumes the '2nd opinion' concept. This proposal integrates the mentoring effort; editors new to the game may participate in open reviews as long as they care to and need not initiate a review of their own until they are comfortable with the ropes. Ditto for experienced reviewers who do not have time to administer a review, but who can offer commentary on other editors' open reviews. The concept of keeping a review open for a minimum period reduces the liklihood of erratic quality arising from quickly passed or failed articles. OhanaUnited raises the issue of editors 'struggling for power', which I take to mean as two or more editors disagreeing on whether an article is good or not. The default outcome of this lack of consensus, I suggest, would be 'fail' — not an unreasonable conclusion if editors fail to agree on the overall 'goodness' of an article. If particular editors battling over particular articles becomes epidemic, then I would suggest that editors who have significantly contributed to a review of a given article should henceforth be regarded as having contributed to the article itself, and, as such, should recluse themselves from future Good Article evaluations of that article. I'm not sure at what degree an editor's review becomes a 'significant contribution' to an article. Probably not spelling and grammar corrections, but probably if and editor has administered a review as lead, or has passed an opinion on whether the article passes or fails. Gosgood (talk) 13:49, 13 June 2008 (UTC)

Oversight

Do we need a mechanism to approve GA assessments before they become official?

• Comment: Maybe this would be the answer to the multiple reviewers/approved reviewers issue. Usually the poor articles are easy to fail, but another set of eyes before approval may be good. Montanabw^(talk) 21:50, 4 June 2008 (UTC)
• Comment - I think this would drastically improve quality. Unfortunately, we face a huge problem. Critics of GA cry that it's of inconsistent quality. However, any changes made to complicate the process and/or bring it closer to FA in anyway will be met with cries that it's redundant of FA. If it is agreed that such oversight is the best way to go, there are various options. Is it a panel of appointed GA veterans, or is it anyone with x number of reviews, who can approve reviews? And do they do a minor review of the entire article and accompanying review themselves, or simply scan over everything to ensure a complete review was carried out and complied with? Lara❤Love 19:11, 7 June 2008 (UTC)

Specialised reviewers

Would it help to have subject-expert reviewers that specialise in certain types of article?

• 'Comment: Maybe, though the view of a total outsider is good too. I think if there were multiple reviewers in some form, having one reviewer with a bit of expertise paired with an outsider might be the best of both worlds. Don't know how feasible that would be, but couldn't hurt to bat the concept around a bit. Montanabw^(talk) 21:52, 4 June 2008 (UTC)
• I've mixed feelings. GA is a community project, outside the wikiprojects (and the wikiproject assessements), which is a strength. However, specialists are more likely to pick up content problems, questionable sources, inaccuracies, and also be more familiar with subject specific MOS issues, or guidelines. A possible approach would be to have subject specific Ga criteria, so those unfamiliar with an area can get a quick idea of the specific requirements. Gwinva (talk) 22:10, 4 June 2008 (UTC)

Many such guidelines already exist; WP:FILMS and WP:UKCITIES spring to mind for instance. --Malleus Fatuorum (talk) 22:20, 4 June 2008 (UTC)

At present the articles are reviewed by volunteer wikipedians without any condition. I think in some cases the issues are completely technical and just an expert or somebody who is knowledgeable can judge correctly about them. While only one wikipedian review and judge about the article, it's better to use a knowledgeable wikipedian to do the task. Thus I suggest separating the technical issues and establish a technical review team, that is a group of wikipedians who are experts or knowledgable in some fields. We can also ask each wikiproject to introduce some knowledgable wikipedians.--Seyyed(t-c) 05:00, 3 June 2008 (UTC)

For example, please compare the two reviews about Sources of Islamic law. The first one has done by somebody who says I am by no means an expert in any sort of law, let alone Islamic law, but I think that will give me a great "outsider" perspective[1]and the second one has done by somebody who says I'm a Muslim and review the lead from Islamic viewpoint[2]. As you can see these are completely different viewpoints and in some cases we need both of them.--Seyyed(t-c) 06:31, 4 June 2008 (UTC)

This has been suggested several times before, including by me. The concept is fantastic and would most likely work. The problem is finding the experts willing to help. I've been out of this project for about 8 months (a message on my talk page led me here tonight), so I'm not sure what happened with my last few proposals, but if a page is created where experts can list themselves under their field of expertise and tandem reviewing can occur, I think that would help solve a lot of the problems GA faces with quality. Lara❤Love 07:00, 4 June 2008 (UTC)

I think we can ask the wikipedians who are active in wikiprojects. I think in some wikiprojects such as Military history we can find several reviewer. In some others like Wikipedia:WikiProject Sociology and Wikipedia:WikiProject Biology it may be difficult. But it's not a major problem, because few articles have been nominated which relate to such inactive wikiprojects.--Seyyed(t-c) 10:14, 4 June 2008 (UTC)

Some projects have too few members, while others may not have any members specialized in reviewing articles professionally. OhanaUnited^{Talk page} 03:51, 5 June 2008 (UTC)

As I told above the projects which have few member are inactive and we don't have problem with them. I think most of the articles which needs technical review are nominated by wikiprojects which are active and have enough specialized members in those fields. --Seyyed(t-c) 09:46, 5 June 2008 (UTC)

Yes, I think we'd have to closely integrate with the projects for this to work as a normal part of the review process. However, I don't think it's enough of an issue at GA, where we're concerned with broadness rather than comprehensiveness, that we need to worry too much about it. When expert opinions are needed - and it doesn't happen that often - it's fairly straightforward to request them, on a case-by-case basis, on the relevant project talk page. EyeSerene^talk 10:02, 5 June 2008 (UTC)

I think it ultimately comes down to reviewers knowing their boundaries in terms of what they can review and what they can't. For instance, I stick entirely within the parameters of reviewing History GANs, and rarely (if ever) step outside of that area, simply because that is my primary area of expertise. We should definitely make some sort of note to new GA-Reviewers to "remain within your knowledge boundaries". It might help to curb the instances shown above. Cheers! Cam (Chat) 21:19, 8 June 2008 (UTC)

I think that's about right. Video games, for instance. I don't like video games, I don't play video games, I know very little about video games, and I wouldn't consider reviewing a video game. --Malleus Fatuorum (talk) 21:30, 8 June 2008 (UTC)

• No harm in making this optional, but I don't think we should have a requirement of specialist reviewers. There are some really versatile people (Blnguyen is a good example) who shouldn't be held back by WikiProject membership. (And there are people like me who probably should, and so are by choice. :)) giggy _(:O) 10:06, 5 June 2008 (UTC)

• Hey, don't do yourself down giggy, I've got an opinion on just about everything as well. ;-) More seriously, I'll suggest that subject area experts are possibly the last people that you'd want to be involved in a GA review. --Malleus Fatuorum (talk) 00:40, 7 June 2008 (UTC)

• The last people involved? I'm thinking we disagree somewhat there - I'm curious as to why you think so? If it's because they might miss basic stuff, perhaps have a subject area and a... well... me (;-)) pair up to do the review? giggy _(:O) 02:44, 7 June 2008 (UTC)

• No, not because they might miss basic stuff, almost the opposite in fact. Because if you put two experts in a room, you'll get three opinions. --Malleus Fatuorum (talk) 17:00, 7 June 2008 (UTC)

• Am I correct in that the proposal is that GA reviewers do their review and, for the technical articles, they ask for input from someone with knowledge in the topic? In many cases, this would be unnecessary, as in cases where members of the Wikiproject are the ones that have done the work to bring the article up to standards. However, when that is not the case, I think it's important to bring in an expert before the article is passed. Lara❤Love 19:24, 7 June 2008 (UTC)

• That's how I see it, though I don't believe asking for expert opinion is always necessary. If a reviewer can't understand a technical article, they probably shouldn't be reviewing it anyway, and unless the expert is familiar with GA criteria we could find ourselves battling against unrealistic expectations of the criteria. The alternative might be to insist that all technical articles have already gone through a project A-class review prior to coming to GA, which I think has some merits. I'll wait to see what happens with proposals elsewhere before arguing that too hard though ;) EyeSerene^talk 19:43, 7 June 2008 (UTC)

• I disagree with the idea that if someone doesn't understand the technical aspect, they shouldn't review it. The entire point, for me, in tandem reviewing, is for two people with different expertise (technical aspect of the topic vs understanding of the GA criteria and the MOS) to review based on their expertise. One focusing on the technical aspect, the other on the MOS side. Having someone who doesn't understand the topic of the article is key, in my opinion. The point of our articles, in many cases, is to inform the uninformed. Having someone who doesn't know the topic ensures that the article is readable. Someone who already has an understanding of the topic is more likely to miss jargon and confusing prose, because they already know what they're reading about. Lara❤Love 19:50, 7 June 2008 (UTC)

• There's truth in that, certainly, but it does help to have at least, say, some mathematical savvy when reviewing mathematical articles. I don't subscribe to the view that we're writing articles to teach the uninformed; that's Wikiversity's job. Many technical articles will only be meaningful to readers with an in-depth subject knowledge, and I have no problem with that. I do believe there should be enough non-technical content (probably in the lead or introductory sections) that a layperson can understand why the subject is notable, but I'll leave it to the textbooks to do more than that. Hence my suggestion for post A-class reviews, when content will already have been verified by experts. EyeSerene^talk 20:02, 7 June 2008 (UTC)

• I agree a lot with LaraLove here - we're not trying to teach people, but we do want articles to be accessible to readers unfamiliar with the topics. Although wikilinks will help to a certain extent with defining words, I personally feel that being able to read and understand an article in isolation is a good thing. Because of that, having someone unfamiliar with the topic review an article can only be a good thing. I also think that a review is a two-way street, where the article nominator can discuss feedback with the reviewer in order to develop the article. Because of this, I'd expect the nom to feel comfortable with challenging the reviewer on areas they feel apropriate. Gazimoff ^Write_Read 20:50, 7 June 2008 (UTC)

• (outdent) I think there's a difference between accessibility and understanding. The article's prose should be of a high-enough standard that the article is readable, and explain why the subject is important and how it is used, but I think to expect a non-specialist to navigate to, for example, Hilbert space (which is currently a GA) and come away understanding the topic is to misunderstand what we are for. Per WP:NOTTEXTBOOK, we shouldn't really be in the business of instructing readers, just informing them. As you say, links can be used to follow up definitions and the like, but especially with technical articles I don't think it would be realistic to start from first principles every time. EyeSerene^talk 21:08, 7 June 2008 (UTC)
  • I think it probably should be... at least, the first principles should be made prominent somewhere (in conjunction with this). I don't think it's good practice to be giving the tick of approval to something a non-expert can barely make meaning out of. I suppose you're not saying this either... in any case, I think we agree that a non subject expert should be able to make some sense of the topic after reading the article. The specialist reviewer—the subject expert—should be able to point out factual inaccuracies in the article that I, for one, would never pick up on. giggy _(:O) 01:39, 8 June 2008 (UTC)
    • Precisely. That, in my opinion, makes tandem reviewing perfect. It covers the accuracy, broadness and technical details. The GA reviewer focuses on the style aspect. Ensuring the lead is well-constructed, the punctuation is correct throughout, images are compliant, etc. Together the two reviewers and the article custodian(s) can work to ensure the prose flows well while maintaining accuracy. As far as requiring such articles be at A-class before receiving GA, I just don't think that's going to happen. FA is the step that generally follows A-class, which is considered higher on the rating scale than GA. Lara❤Love 05:22, 8 June 2008 (UTC)
      • Surely the spirit of GA is that the article's authors/nominators are the experts? And that those subject-matter experts work in tandem with a GA reviewer? I'm afraid this proposal just seems like another move in the direction of FA-lite to me, another nail in the coffin. --Malleus Fatuorum (talk) 21:36, 8 June 2008 (UTC)

Is "FA-lite" a real threat? I guess I mean, don't we have a firm consensus that we don't more than one or two people to be reading the same GAN article, unless people just want to train or vet or learn from the main reviewer(s)? That seems like the opposite of the FAC philosophy to me. And on the other point, a collaboration between the reviewer and the editors as "content experts" is nice when it happens, but sometimes they're not experts. - Dan Dank55 (talk)(mistakes) 22:39, 8 June 2008 (UTC)

I believe that FA-lite is a real threat, although I'm beginning also to believe that I'm perhaps the only one who does. As for the article's authors not being being "experts", well, as I said above, if it's not a topic that you feel you know enough about to review, then don't review it. --Malleus Fatuorum (talk) 22:50, 8 June 2008 (UTC)

So do I, as it happens. I think we can just ask around the projects on those (infrequent IMO) occasions when an outside subject expert is needed and no-one in the GA community has the requisite knowledge to fill in, or as Malleus says, leave the review to someone who knows a bit more about it. We've run into difficulties in the past with GA criteria being applied inappropriately to technical articles (as Lara will remember better than I, having had to deal with the fallout!), sometimes expert advice will be needed, but I agree that, in general terms, we can still assess against the criteria for layout, MoS, prose, NPOV, image copyright etc, and the authors should supply the content. EyeSerene^talk 12:12, 9 June 2008 (UTC)

FA-lite is a real threat. And I warned against this somewhere on this page, I believe. If you make improvements to the GA project, you're going to hear "It's becoming redundant of FA" from opposers of the project. If you don't make improvements, you're going to hear "It's a poor quality project worth nothing." This is why I left the project last year. It was lose/lose. We made several major changes to the project that, I believe, significantly improved the quality. But the squawking from the opposition was steadfast. However, I've had time to reflect, and I've come to the following conclusion:

We have two options. 1/ We leave the project as is, knowing there is need for improvement and hearing it regularly from those who oppose the GA project, or 2/ We make improvements and receive the customary "They're moving more inline with FA; GA is redundant of FA, delete it!" arguments, the project survives because, let's face it, there's support on this project to keep GA, and the end result is an improved project. Short version of my conclusion: Who cares? Everyone here is a volunteer in a self-appointed position. We put ourselves in GA, so let's do what we do and not worry about what those looking in from a distance have to say unless they offer valid concerns for which we can build upon. Lara❤Love 12:16, 9 June 2008 (UTC)

I see that a lot of GAN reviewers provide lists of their reviews on their userpages. Surely a history of reviews that were later confirmed at GAR or FAC (or at least, confirmed with respect to the GAN criteria) suggests that the reviewer is competent and trustworthy? Maybe we should be more explicit on our userpages that we're not just boasting, that pointing out that our judgment has been confirmed is an essential part of making GAN more efficient that FAC, by allowing and encouraging public scrutiny of the reviewers. I don't have anything bad to say about FAC reviewers, but it does seem to me that having one reviewer instead of 10, in addition to being 10 times more efficient, is also braver and shows a willingness to take personal responsibility for one's work. - Dan Dank55 (talk)(mistakes) 15:16, 9 June 2008 (UTC)

Time limit

Would changing the period allowed for reviews help? (for example, by insisting all reviews are posted for a set period)

• I don't think there should be a hard and fast rule regarding a time limit for reviews (on hold status). Granted, we don't want reviews to take forever, and I think most will take about 5-7 days or so. But if it takes a little longer to get it done, and as long as the editors are responding to comments and actively editing it, it's fine to take longer. It's probably best to stick with the status quo here and let reviewers find the best times for reviews/on holds themselves. Dr. Cash (talk) 21:40, 3 June 2008 (UTC)
  • Agreed, hold periods under the current system should be flexible where constructive work is ongoing (within reason!). Do you think it would help to leave other reviews (ie outright passes, quickfails) up for a set period too, so no assessment is complete until it's been open for comment for, say, one week? EyeSerene^talk 08:11, 4 June 2008 (UTC)
    • A minimum time limit would be a good idea; it would go some way to eliminating the "drive-by" reviews that occasionally occur. This would basically rule out "quick passes" and "quick fails", and (hopefully) encourage more detailed/specific reviews. A one line "yes it passes" is as unhelpful as "sorry, it fails"; most people submitting for GA want feedback as well as green dots. A time limit would also allow some level of quality control: it doesn't need to be a full "multiple review", so much as an opportunity for others (if inclined) to cast an eye over things. Better to spot problems (with reviews) at this stage, rather than waiting for GAR. Gwinva (talk) 21:57, 4 June 2008 (UTC)
• I disagree with a time limit, and always have. I've passed holds in less than an hour, and I've had holds go for over a month. If the article is being worked on, or if there is a promise of work on the article (author has exams/is on holidays/whatever) soon, there should be nothing arbitrary holding the review back. giggy _(:O) 05:13, 5 June 2008 (UTC)
  • That said, some newer reviewers may benefit from a set time (at least 24 hours?) to have someone else take a quick look over their work, in conjunction with one of the other proposals gaining some weight here. giggy _(:O) 05:13, 5 June 2008 (UTC)

I think we should rely on the reviewer's judgment. Reviewer can wait more, if he/she finds the editors are active to improve it and the problems are minor. On the other hand he/she can fail it soon when editors are inactive or problems are serious. --Seyyed(t-c) 11:21, 5 June 2008 (UTC)

Let the reviewer decide, but still recommend a week if possible. This will hopefully prevent the editors of taking their time to getting around to fixing the article, allowing for more time on focusing on new articles. We don't want to rush the editors, but don't want them to take an extreme amount of time that isn't necessary. I've passed articles in a matter of a few days or a few weeks, based on the editors continually working on improving it. --Nehrams2020 (talk) 23:50, 5 June 2008 (UTC)

• I wouldn't remove the option for quick-fails. There are some articles that just don't come close to meeting the standards. Much to often the backlog at GAN is over a month. If we lose the option to quick-fail articles that are obviously below the standards for GA, the backlog is only going to grow. I haven't looked recently, but the quick-fail criteria may still be in need of tweaking. As far as a minimum time-limit on reviews and holds, I wouldn't make it more than 3 days for reviews. I'm not a fan of drive-by reviews, but when I was in the mood to review, I could to a thorough review in a matter of a couple hours. However, whether a pass or fail, a couple more days for others to possibly look could never hurt. For holds, I think seven days minimum is good. I, like giggy, would always extend as long as progress was being made. Lara❤Love 19:29, 7 June 2008 (UTC)
• I think the merit in getting rid of quick-fails is that since we're trying to drive up quality, it might help to ensure that one-line fail notices become a thing of the past. Although we're not intending to review an article, it doesn't hurt to leave a decent explanation of why, and what can be done to fix it (as we would after a normal 'fail' review). Nice to see you back, BTW! EyeSerene^talk 19:48, 7 June 2008 (UTC)

Statistics on quality

It would be helpful to get an idea of what the quality of our reviews currently stand at. I'm not sure if any statistics are being gathered from sweeps, but it could be valuable information. I also think it important to attempt to get some information from SandyGeorgia regarding what the common issues are with recently promoted GAs that she finds at FAC. If she has any information about what the most common issues are, or even the most common reviewers to pass deficient articles, it would give us a good starting point for project improvement in regards to reviews. Lara❤Love 19:31, 7 June 2008 (UTC)

Review process proposals II - the winnowing

There has been little movement here for a few days now, so can I take it we're all discussioned out and it's time to move on? I've tried to distil down the comments that have been made so far, and summarise the state of play. Again, if I've left anything out, or over-simplified/misrepresented anything, please make corrections as you see fit ;) EyeSerene^talk 18:37, 12 June 2008 (UTC)

Unsupported

• There was little support for maintaining the status quo, although it was pointed out that whatever we do, we will not change the minds of all GA's opponents, so we may be better off simply ignoring calls for change. There was also little enthusiasm for putting in place some kind of official approval mechanism for signing off on GA reviews - whilst perhaps a good idea in theory, there seem to be too many potential problems to make this a practical solution. The idea of addressing review quality by insisting all reviews follow a standard format was also largely unsupported. Voting on GA assessments was generally disapproved of, and felt to be too similar to the FA assessment system.

No consensus

• The suggestion to team up with subject experts when conducting reviews reached no real consensus in how it should operate; although it was widely agreed that expert advice concerning article content is very useful and sometimes indispensable, how this should be obtained remains undecided. It was noted that GA reviewers should expect the article authors, as subject specialists, to provide content advice where needed, and reviewers can in any case assess most of the GA criteria without any need to fully understand the article content. A system for 'approving' reviewers was thought by some to be useful, but there was no agreement on how this could be achieved without over-bureaucratising the project.

Supported

• The remaining proposals, after some debate, crystallised into broad agreement on a slightly different review mechanism to that used at present. Incorporating the proposals for more than one reviewer, an adjustment to the review time-limit, and the guidance of new reviewers, general support was expressed for the idea of 'open' reviews. These could be initiated and closed by a lead reviewer but open to anyone interested enough to comment, and could remain open for a minimum set time period. It was felt that this would address the concerns with review transparency and drive-by reviewing, and provide a means for new reviewers to 'wet their feet' by joining in with in-progress reviews before taking the lead role in subsequent ones. It would also not preclude single-reviewer reviews, collaboration, and a personal approach, thus retaining the strengths of the current GA process. However, some of the details remain under discussion.

Since only one proposed solution emerged, I've set this out below, point by point, and indicated where there may be a need for further discussion. I don't think there's any need to turn this into a vote (it's pretty much a one-horse race), but if anyone wants to add alternative proposals, please do!

Open review proposal

Reviews are open for anyone to leave comments

Consensus reached

Reviews are opened and closed by an established 'lead' reviewer (the reviewer who initiates the review)

Consensus reached

All reviews remain open for a minimum period (no more quick passes/fails)

No consensus as to how long, though between 3-5 days seems to be the preference

• 3 days is fine; after all this is a minimum, and the lead reviewer may choose to keep it open, in much the same manner as an "on hold" now. 3 days is probably enough for all concerned to have a look; but I am not averse to 5 days. I don't think we need to be concerned about "clogging up" the GAN page with open reviews: when people wait a month or so, another couple of days won't make much difference. I see this as replacing the current "on hold" status.Gwinva (talk) 21:52, 12 June 2008 (UTC)
• I'm still (a bit defensively, perhaps) against imposing a minimum time limit on more experienced reviewers. giggy _(:O) 10:43, 20 June 2008 (UTC)

• I'm with giggy. People who know GA inside-out shouldn't get bounded by this rule. OhanaUnited^{Talk page} 15:20, 20 June 2008 (UTC)

• And keep quick-fail please. OhanaUnited^{Talk page} 06:00, 21 June 2008 (UTC)

• Three days is good for a minimum; two days is probably sufficient, too. It can be extended if necessary. Dr. Cash (talk) 13:55, 20 June 2008 (UTC)
• three to five days plus would be my preference, if only to be able to undertake a review. Edmund Patrick – confer 19:49, 20 June 2008 (UTC)
• I'm still opposed to getting rid of quick-fail. But I think a three day minimum is good. LaraLove|Talk 20:42, 20 June 2008 (UTC)
• I'd prefer not to remove the quick-fail option. Some articles are in really bad shape, and reviewers should focus more of their attention on articles in reach of becoming GAs. Reviewers can still leave constructive comments on the talk page before quick-failing the article, guiding them to pursuing the steps they need to take before it is near-GA quality. Concerning a minimum time period, perhaps this can be set up for reviewers who are new until a set time (based on a number of reviews/established time period). Editors who have been established in completing so many reviews/time period should be able to bypass the minimum time period. --Nehrams2020 (talk) 04:48, 21 June 2008 (UTC)
• I agree with keeping the quick-fail option. If an article is quick-failed by mistake, the worst that can happen is a relisting. If an article is quick-passed, it can lead to concerns about not applying enough care. I would re-word the proposal to state that nominations can be quick-failed, but that successful nominations have to remain open for a minimum amount of time before closing.
• Personally, I favor abolishing quick fail, I have seen it abused too often. Give people a warning and not just a smack in the face, it sort of is biting the newcomers to GA, IMHO. Montanabw^(talk) 04:11, 22 June 2008 (UTC)

Gazimoff ^Write_Read 06:29, 21 June 2008 (UTC)

• Oops, looks like I misjudged things here - I thought we mostly agreed that quick-fail was being done away with. Apologies for that! Since I've kind of ended up facilitating the discussion, I'm a little hesitant to comment in case it's seen as being less-than-neutral. However, a couple of points do occur to me:

• Abolishing quickfail and starting an open review (even if only for procedural purposes, to be failed after the minimum 3(?) days) would give new reviewers the opportunity to see the sort of articles that are unready for nomination, and thus help with their mentoring. It would not be a comment on the judgement of experienced reviewers, but a learning opportunity for inexperienced ones.
• If quickfail is retained in its present form, maybe it would help address some of the objections by changing it slightly from an actual 'GA fail' to a simple 'review request declined'? This would still allow such articles to be removed from the GAN page, but without all the attendant baggage of editor disappointment, permanent talk page records etc. It would essentially alter the current judgemental outcome to a much friendlier "not now, but we'd like to see it again when it's improved" outcome.

EyeSerene^talk 10:52, 23 June 2008 (UTC)

That's a good idea. giggy _(:O) 11:47, 23 June 2008 (UTC)

it is an excellent idea, simple and friendly and supportive of the article in question. What though would be the criteria the same as a quick fail? Edmund Patrick – confer 13:18, 23 June 2008 (UTC)

This sounded good to me too at first, but on reflection it is just sugar-coating a bitter pill: "your article is not even good enough to review"! I'm also not sure how it would work in practice. For instance would it be recorded as a GA action, and hence get recorded in article history? Geometry guy 16:22, 23 June 2008 (UTC)

No, I don't think it's recorded as a GA action - part of the point is that it isn't. giggy _(:O) 11:02, 24 June 2008 (UTC)

• I'd like to suggest a compromise on the quick-fail issue, which is to remove the quick-fail bureaucracy, but allow for exceptions to the 3 day minimum for articles which are a long way short of GA. I think such articles still deserve at least a short paragraph on a review subpage. That way every GAN action has a permanent review, new reviewers are encouraged to provide proper reviews, while experienced reviewers can apply common sense when failing articles quickly. Geometry guy 16:22, 23 June 2008 (UTC)

I think that a "warm and fuzzy" quick fail is a suitable compromise, sometimes we wikipedians can be kind of harsh without realizing it. Geometry guy's idea may also work, my thinking is that maybe such "warm and fuzzy quick fail" articles could simply not have the GA action recorded at all?? No black marks, so to speak? Montanabw^(talk) 05:14, 24 June 2008 (UTC)

You have to remember that quick-fails are only quick from the point of view of the reviewer; an article could have been nominated for over a month, only to receive the response "review request declined". That is unacceptable, however it is phrased. Furthermore, how do we instruct reviewers to be warm and fuzzy? The central line of WP:CREEP says it all: "The fundamental fallacy of instruction creep is thinking that people read instructions."

The key benefit of subpages is that they provide accountability, and we can take advantage of that to reduce bureaucracy by giving reviewers more freedom on condition that they always leave a review. We cannot undo a quick fail because it wasn't warm and fuzzy enough; we can undo it if the reviewer did not leave a review. A failed GA nomination is not a black mark; it is feedback and so useful feedback should be provided. Geometry guy 09:19, 24 June 2008 (UTC)

As I see it, the quality issues we need to address are one-line (or, worse, no-line) quickfails, and misapplication of the quickfail criteria. Removing quickfails altogether is my preferred option, since drive-by reviewing is then eliminated, a minimum-period open review pretty much obliges the reviewer to leave a helpful assessment, and errors can hopefully be caught. Retaining them means we need to find a way around the problem - declining the review would seem to me to be the second best solution. Reviewers should of course still leave pointers on what needs to be done, but it seems to me that however we organise it, we've done nothing to actually address the drive-by/unsafe assessment issues. Given that, I'd prefer not to assess such articles at all: I think it will harm GA less to face complaints about declined reviews rather than charges of unsafe fails, and therefore Montanabw's suggestion of no recorded action makes sense. On a personal note, the first article I wrote was quickfailed, and after waiting a month after nominating, I was gutted to suddenly see the GA fail template slapped on the talk page... and there for all time. It comes over as impersonal, bureaucratic, and peremptory - all the things GA is not supposed to be. I'd much have preferred to be told that my article was not ready for review yet, with some hints as to how to fix that. EyeSerene^talk 10:50, 24 June 2008 (UTC)

• To the contrary, I'm strongly opposed to keeping the "quick-fail" option. I still think an option should be made available for those of us that are experienced to know how to handle it, for example, to handle "drive-by nominations" and articles that just plain don't meet the criteria. But we should seriously not advertise this option on the WP:GAN page in the instructions. GA needs to focus on providing editors with a good, concise, and thorough review, and a "quick-fail" is contradictory to this overall goal. I think a much better solution is to do away with "quick-fails" entirely, and maybe have an "administrative removal" option for the more experienced reviewers (those that are listed as mentors, perhaps?), so that the crud can be filtered out quickly. Dr. Cash (talk) 20:57, 24 June 2008 (UTC)

• That's (I think) fairly similar to what EyeSerene has proposed above. giggy _(:O) 23:36, 24 June 2008 (UTC)
• Yes, we're thinking on the same lines. Maybe "administrative removal" is a better way to phrase it than "review declined" ;) EyeSerene^talk 08:23, 25 June 2008 (UTC)

  Administrative removal (with no review, just a talk page explanation) is a solution that I'm willing to go along with, but it would have to be what it says on the packet. As such it would be more akin to the kind of thing I sometimes do at GAR when someone nominates an article to get leverage in a content dispute (not that I doubt their good faith, of course :) than the current quick fail criteria. Basically, this amounts to an application of WP:IAR that there is no point, and certainly no net benefit to the encyclopedia, in having a review of that article at that time. An example would be an article that is in the midst of a significant edit war, or someone nominating a stub or an article with absolutely no reliable sources. In contrast, an article with possible NPOV issues or clean-up banners might benefit from a review, and administrative removal seems far less appropriate. Geometry guy 19:00, 25 June 2008 (UTC)

The option for "Administrative removal" of those few which are not suitable for review is a good idea, but it would need to happen soon after nomination, not sit in the waiting list.

EyeSerene made a good point about the 3-day period being for the benefit of others. A couple of people have suggested that experienced reviewers should not be bound by the 3 day wait, and that some articles can be passed quickly. But this is not really for the benefit of the reviewers; an open period allows new reviewers to observe reviews in action. An open period also allows the nominators the chance to make comments, or to receive a little more feedback/suggestions. Also, coupled with the subpages, helps to keep things all transparent and "in the open": not activity happening behind closed doors, but a process all can observe. Most importantly, we want to simplify the process: we ceratinly don't want lots of different rules for different editors, and cases. Gwinva (talk) 05:05, 26 June 2008 (UTC)

There is no requirement for reviews to have additional reviewers

Weak consensus, although some support for tandem reviewing, perhaps with a subject expert

• I see this as case-by-case. At the moment we get good reviews from experienced reviewers, and we don't need to complicate those by adding extra requirements. But some mechanism for requesting extra opinions (from experts or experienced reviewers) might need to be established. This might be flagging the review on the GAN page (like "2nd review requested") or by the lead reviewer issuing an invitation to a list of mentors/subject experts. Gwinva (talk) 21:52, 12 June 2008 (UTC)
• No requirement, just recommended (strongly, especially for newer reviewers). giggy _(:O) 10:43, 20 June 2008 (UTC)
• No requirement. Some articles pass the GA criteria with flying colors, and are relatively easy for one reviewer to pass. Others, particularly those on some of the more controversial topics, are a bit more tricky, so encouraging more than one person is a good thing there. Dr. Cash (talk) 13:55, 20 June 2008 (UTC)
• No requirement, but in an unreal world I would live to be able to call on a specialist occasionally, but in the real world! Edmund Patrick – confer 19:49, 20 June 2008 (UTC)
• Don't require this, but allow/recommend the option for reviewers. --Nehrams2020 (talk) 04:48, 21 June 2008 (UTC)
• I think if it were possible to require this, it should be. However, I don't see that as being possible, so I'll have to agree with giggy that it should be strongly encouraged for new reviewers. LaraLove|Talk 05:55, 21 June 2008 (UTC)
• I agree here. If we come across a new reviewer not applying criteria adequately, one of the more experienced reviewers may want to step in and offer help, guidance and mentoring. Well-versed and experienced reviewers should not be compelled to seek seccond opinions unless needed, but the process already caters for this. Gazimoff ^Write_Read 06:29, 21 June 2008 (UTC)

New reviewers are encouraged to participate in current reviews before taking on the lead role

Consensus reached; may need to discuss how this should be monitored (if at all)

• I don't think we need to formalise anything beyond the current mentor programme (I'm assuming the mentors will wish to keep an eye on their protege's first few reviews). Merely clarify the "review instructions" to suggest new reviewers start by contributing to open reviews, and a pointer to the mentor programme. Some reviewers may feel confident starting out on their own, and we shouldn't discourage that, or devalue real-world experience. Gwinva (talk) 21:52, 12 June 2008 (UTC)
  • If we say "Wait a while until you take the lead role", it puts the new reviewer in the position of having to guess how long we want them to wait. I'm happy with what Gwinva is saying, but I think it goes hand-in-hand with some kind of light-weight approval process after, say, the new reviewer has done 5 of their own reviews. - Dan Dank55 (talk)(mistakes) 23:20, 12 June 2008 (UTC)
• What Gwinva said. giggy _(:O) 10:43, 20 June 2008 (UTC)
• I think Gwinva's approach is the right one. It's going to be hard to force new reviewers to use the mentor system, but it should be encouraged. Dr. Cash (talk) 13:55, 20 June 2008 (UTC)
• What I would have liked (but didn't have the nerve) and also what in the future I would be prepared to do is offer the new reviewer to in the first instance put their thoughts on the article to me at my talk page. There hopefully I will offer a slightly more possibly secure, friendly, exploratory place to hone the points s/he wants to make. These could then be added to the GA subpage. Edmund Patrick – confer 19:49, 20 June 2008 (UTC)
• Although this would be a great option that we could push, we shouldn't require it as some new reviewers may be ready on their own. --Nehrams2020 (talk) 04:48, 21 June 2008 (UTC)
• Same as above, this should be strongly encouraged. LaraLove|Talk 05:55, 21 June 2008 (UTC)
• Agreed. Mentoring for this process, as with other processes, should be strongly encouraged.Gazimoff ^Write_Read 06:29, 21 June 2008 (UTC)

Review status can be monitored

More discussion needed; possibilities include listing open reviews somewhere, categorising them, changing the way they're listed on the GAN page etc...

• There have been some criticisms recently about the complicated "paperwork" of the GA process. An easy categorising or bot-assisted process would be good. One radical idea might be to dispense with the GAN page altogether (or turning it into a instruction page). The GAN template can categorise from status. "nominate" → Category:Ga noms (by subject); "open" → Category:open reviews; "failed" → Cat:Failed GAs; "passed" → Cat:GAs. Reviewers go to Cat:GA noms rather than the GAN page when they feel in the mood for reviewing. Or Cat:open if they like looking at open ones. Status changes take place on the article talk page, rather than all this listing and commenting at GAN. Gwinva (talk) 21:52, 12 June 2008 (UTC)
• Yeah, more discussion needed (some good proposals coming up). giggy _(:O) 10:43, 20 June 2008 (UTC)
• I think automation of the GAN page via bots is probably the best way to go. I don't like the category idea because it would take away the ability of reviewers to make comments on nominations that can be viewed by everyone, which is a valuable part of the GA process. The categories would just be a list, with no comments. Dr. Cash (talk) 13:55, 20 June 2008 (UTC)
• I like the stages and I like the pathway that they highlight. Bot automation is great as long as we do not loose the stages. Edmund Patrick – confer 19:49, 20 June 2008 (UTC)
• If it is possible to get a bot to do this, then it would be great to set this up. We can get more opinions for intial reviews by new reviewers or weigh in on complicated reviews. --Nehrams2020 (talk) 04:48, 21 June 2008 (UTC)
• I link Gwinva's idea, only subpages rather than Categories. That eliminates Dr. Cash's concern. The better organized the better. And any steps we can cut out of the process is a good thing. Bot automation would be wonderful. LaraLove|Talk 05:55, 21 June 2008 (UTC)
• I like the idea of having a bot to review GA status pages, as well as sweeping through the list of GAs and ensuring that lists of GAs are updated. Maybe once an idea for page structure etc is in place, we should look at putting a bot proposal together. Gazimoff ^Write_Read 06:29, 21 June 2008 (UTC)

General discussion

Thanks, EyeSerene for taking this forward. I've bolded the above points, to make them stand out, and added a few comments. Seeing it laid out here all looks very positive; I feel we are not far off a workable proposal. Gwinva (talk) 21:52, 12 June 2008 (UTC)

Well-written criterion

Do we want to handle this criterion any differently than the others, when evaluating reviewers as well as articles? Obviously, you can know all the style guidelines and have a shelf full of writing awards and otherwise suck as an article reviewer, or be a great article reviewer except for insufficient language skills. What makes this difficult is that opinions are all over the place on what constitutes good writing; see WT:MoS#Three threads for a list of some of the tradeoffs that are judgment calls that everyone will make differently. Opinions vary so much that we don't even include WP:MOS as one of the suggested style guidelines pages at WP:WIAGA, for fear of starting more arguments than we settle. I think it would be a good idea to do something, but I don't know what. As Einstein might say, heirarchy should be as flat as possible, but no flatter; if we cause more problems than we solve by taking no position on what constitutes good writing, then we should take a different position. - Dan Dank55 (talk)(mistakes) 20:19, 18 June 2008 (UTC)

So far as GA is concerned, sans all of the FA baggage of "brilliant prose", "professional writing", and so on, aren't we just expecting correct spelling and grammar, and at least a consistent interpretation of the MoS in terms of the way that dates, quotes, and so on are formatted throughout the article? --Malleus Fatuorum (talk) 20:41, 18 June 2008 (UTC)

Agreed that we want to nip "brilliant prose" in the bud, in the sense that we don't want people to think they have to labor over prose to pass WP:GAN. But almost all of the 31 out of 31 respondents in the WP:GAU survey wanted more feedback on their writing for the article being reviewed, and the few that didn't were happy to get more feedback in general. I'm wondering if by giving ourselves a short checklist, we're making our lives easier at the expense of not giving the editors what they want. I am particularly concerned (as always) about the impression we're making on content experts, people who don't "buy into" Wikipedia yet but have a lot to offer; I wonder what the effect is of having their language reviewed by someone who isn't good with the language. - Dan Dank55 (talk)(mistakes) 20:55, 18 June 2008 (UTC)

Have you made the results of your survey available anywhere yet? --Malleus Fatuorum (talk) 20:59, 18 June 2008 (UTC)

At WP:GAU. I drew the obvious conclusion, and other conclusions could be drawn if there's interest. - Dan Dank55 (talk)(mistakes) 01:53, 19 June 2008 (UTC)

I see the answers there, but I don't yet see any analysis of the answers. --Malleus Fatuorum (talk) 02:04, 19 June 2008 (UTC)

The only conclusion I've needed in arguments is WP:GAU#Contentment. Is there anything you want to know? - Dan Dank55 (talk)(mistakes) 02:49, 19 June 2008 (UTC)

Are you sure the numbers at GAU are accurcate? I found Juliancolton responded 5 times in different sections. Just making sure that you didn't quintuple counted an individual's view as 5 different individual. OhanaUnited^{Talk page} 05:23, 19 June 2008 (UTC)

Julian is the reason I'm saying 31 out of 31 instead of 35 out of 35. The others didn't repeat. - Dan Dank55 (talk)(mistakes) 11:45, 19 June 2008 (UTC)

These numbers do not appear to me to be accurate. On a quick count through it seems apparent that while many wanted more feedback on writing, a significant number did not. I make it 21 out 31 said "yes", 8 said "no", and 2 appeared to be undecided. In other words, almost 33% said "no". That's far from "almost all" in my book. --Malleus Fatuorum (talk) 12:56, 20 June 2008 (UTC)

The short answer is that I asked on the user's talk pages for those 3 people who gave responses that didn't indicate that they either were generally happy with the process or who specifically asked for more feedback. I didn't ask TonyTheTiger for clarification, but he's got 90 GA crosses on his userpage. I have just asked Lawikitejana and Happyme22 for clarification. - Dan Dank55 (talk)(mistakes) 00:49, 23 June 2008 (UTC)

Perhaps I need the long answer then, because it seems to me that almost one-third of respondents said thanks, but no thanks. --Malleus Fatuorum (talk) 01:09, 23 June 2008 (UTC)

←Oh...on that point, you're absolutely right Malleus, sorry. "Almost all" was from memory, and was overstating it...not sure why I remembered it that way, I've gotten it right before: all the respondents either wanted more feedback on the article in question, or else there was evidence that they were happy with the process as a whole. That was an important result, because there are people who are fond of saying that people only put up with style guidelines and article reviewers because they have to, that both are crap. That attack had never been taken seriously, and not taking it seriously allowed it to hang in the air and fester (can you fester in the air?) That was one of the points of making this a large random survey. I was quite surprised and happy to see that that really doesn't seem to apply to any of the 31 respondents. You asked about some of the replies at WP:GAU, and I'll reply there. - Dan Dank55 (talk)(mistakes) 02:28, 23 June 2008 (UTC)

• I don't think GA should have a significant prose requirement, and editors (GAU) who want this should be forwarded (with respect, and with offers to help out ourselves if we have the time) to peer review. giggy _(:O) 08:04, 19 June 2008 (UTC)

• The only prose requirement for GA should simply be no blatant spelling or grammatical errors, and basic compliance with the manual of style without going too 'anal-retentive'. "Brilliant prose" and "professional writing" is for FAC. Dr. Cash (talk) 13:58, 20 June 2008 (UTC)
• GAs need to have a significant focus on ensuring errors in spelling/grammar are not included. I always stress as many errors I can find, hoping to prevent these mistakes from appearing in future GANs from the editors. I don't think we want to come as close to "brilliant prose", but the article should be acceptable enough to avoid all spelling and grammar mistakes and hopefully comply with many of the MOS and the respective WikiProject guidelines. Nominating editors should pursue peer review and opinions from fellow editors (especially related WikiProjects) before nominating, and perhaps we should stress this more before articles are nominated. --Nehrams2020 (talk) 04:36, 21 June 2008 (UTC)

Personally, I always take prose into account when reviewing GAs. I find that the more prose copyediting that goes on during GA, the less time required to get from GAN to ACR. If prose isn't taken into account, then you've completely ignored one of the major criterion of the A and FA articles. GA should exemplify our good work, and that means good prose. Cam (Chat) 00:24, 22 June 2008 (UTC)

I've got no doubt that every GA reviewer takes the prose into account. It is, after all, one of the GA criteria. --Malleus Fatuorum (talk) 00:29, 22 June 2008 (UTC)

Yup, it's the quality of prose that counts - everyone should be reviewing for "good" prose. giggy _(:O) 02:02, 22 June 2008 (UTC)

I agree that brilliant prose should be reserved to FAC, but there should be a requirement greater than simply correct spelling and grammar. I won't pass an article for GA if it doesn't flow well. Of course, I always to what I can to help with improving it. LaraLove|Talk 04:25, 25 June 2008 (UTC)

The practice at GAR has generally been to expect a little more than just correct spelling and grammar, but that may partly be because neither Majoreditor nor I like to see GAs with verbose, unclear or generally poor prose. (And we help out too.)

Good prose, yes, brilliant prose, not required. Geometry guy 18:39, 25 June 2008 (UTC)

Next steps for Open Review proposal

Are we ready to move on with this proposal? I realise there is still no consensus for removing/retaining quick-fails (though it seems as though an 'administrative removal' might be a workable compromise), and the technical details need working out (perhaps by consultation with someone who knows something about bot or template writing), but we seem to have agreement in other areas. Perhaps we should put this proposal to the wider GA community for comment, and see what emerges?

The alternative is to continue to thrash out the details here, but we run the danger of this page becoming moribund (and then having to address these issues yet again in the future), and I think we need the input of the editors who would be operating this system... if it gets community approval. Thoughts? EyeSerene^talk 16:53, 3 July 2008 (UTC)

Yes, I agree that we should now draft a succinct summary of this proposal and lay it before the wider community, to see if it receives general consensus, rather than thrashing out minutiae, or letting it becoming moribund. Since we're not in agreement about quickfails, how about presenting the two options (keeping quick fail vs general deprecation but some "admin removal"), and see where general support lies. Gwinva (talk) 22:14, 3 July 2008 (UTC)

I agree that the open review proposal should be presented to the reviewing community (preferably at WP:GAN). I also think there is some consensus here that quick-fails need to be replaced by a less formal procedure, perhaps a purely administrative one, perhaps a matter of discretion. I would hope at least that the bureaucracy (the redirect, the 4 templates etc.) could be ditched. We should try and avoid voting as far as possible: in particular, I think it is much better for all comments (be they supportive or not) are in one section.

Concerning technical issues, I don't see any which significantly affect this proposal, but I have started a subsection below. Geometry guy 20:20, 5 July 2008 (UTC)

Yes, I absolutely agree we should avoid partitioning the debate - it discourages consensus-building, and we need to hear rationales rather than get unsupported conclusions. EyeSerene^talk 10:35, 6 July 2008 (UTC)

Technical issues

I've looked through this page for comments related to technical issues. Here are a few comments.

1. It is very easy to provide a page which lists the articles under review/on hold (would these become equivalent under the open review system? I hope so, as that would be simpler).
2. It is fairly easy to automate the GAN page, as long as comments are not included.
3. Automating the GAN page with comments is harder, because it requires both editors and a bot to be able to update the page. Alternatively, comments could be transcluded from subpages, but this either adds additional complexity, or may lead to a very slow loading GAN page.
4. I do not understand LaraLove's comment: "I link [like] Gwinva's idea, only subpages rather than Categories. That eliminates Dr. Cash's concern. The better organized the better. And any steps we can cut out of the process is a good thing. Bot automation would be wonderful."

There are several comments which say "a bot would be great", but you need to say what you want a bot to do, and what limitations you are willing to accept in return for automation. Geometry guy 20:20, 5 July 2008 (UTC)

Points 1 and 2 are great. 3 doesn't sound fun, but then, not being able to comment at all might be a nuisance. I don't have any good ideas on what to do about it though. —Giggy 02:32, 6 July 2008 (UTC)

I'm guessing the information we'd need to have centralised and easily visible would be: a link to the GA review sub-page, the nom date, the date the review opens, and perhaps the name(s) of the reviewer(s). I suppose it might also be useful to have some facility for requesting additional input. So two questions then: is there anything else we'd need, and can all this be bot-listed based on template parameters? EyeSerene^talk 10:35, 6 July 2008 (UTC)

Link and dates are no problem, and requesting input in the style of the current "status=2nd opinion" is also straightforward with templates and categories. The reviewer name is slightly trickier, but one fairly easy way would be to presume that the (lead) reviewer is the editor who created the /GAn page. A bot can retrieve this information from the edit history of the /GAn page. Geometry guy 13:37, 6 July 2008 (UTC)

That sounds fairly straightforward, though I've noticed that sometimes the nominator creates the review page. Would it be easier for the lead reviewer to add their name to a template when they open the review, or can a bot-updated page include manually updated info without losing it each time the page updates? I'm not fixated on names, BTW, but I think with the mentoring aspect of the above proposal we'd need a way to keep track of who's reviewing what. Maybe there's some other way to do that though? EyeSerene^talk 08:33, 7 July 2008 (UTC)

I added a bolded notice to {{GA nominee}} a while back stating that only reviewers should create the subpage. If it's still happening maybe we need to make this more clear, rather than giving reviewers another hoop to jump through. —Giggy 08:39, 7 July 2008 (UTC)

Yes, that was a good move.

Reviewer names can automatically be added to templates via template substitution (much as dates are automatically added now), but the difficulty is how to read this information and use it. My view is that the edit history is easier to read and at least as reliable.

However, none of these technical issues need to be resolved for open review to proceed. The current mechanisms can comfortably accommodate it, which is why I started a subsection on the technical issues. Geometry guy 21:44, 7 July 2008 (UTC)

Fair enough. I'll get to work on a draft then ;) EyeSerene^talk 08:20, 9 July 2008 (UTC)

Draft proposal to put to GA community

Suggested text below; please comment/amend where necessary. EyeSerene^talk 14:38, 9 July 2008 (UTC)

GA reform working party proposal

First, some background. Following concerns about the reliability and credibility of good articles (raised, for instance, during the 'Green dot' debate), a working party has been examining the way articles are assessed for Good article status, starting with the review process.

The current process has many strengths: collaboration between article reviewers and editors is encouraged, the review is personal, and when a good reviewer works with an inexperienced editor, this informs the editor about the process. Since nomination and review are carried out by individuals, the process can scale to meet growing demand, can minimize bureaucracy and can potentially achieve a fast turnover of articles. The process also has weaknesses. One of the most significant, and the one that attracts much criticism, is a lack of accountability. When the current system is applied well, this doesn't matter, but there are few controls in place to prevent it being applied poorly, or being deliberately abused.

A number of measures were considered and rejected by the working party (discussion here) to address this issue. These included requiring: more than one reviewer per review; voting on reviews; approved reviewers; standardised review templates; specialist reviewers; and review approval by appointees. Although many of these proposals might indeed help to address the concerns, it was felt that they would all diminish the strengths we already have, and may harm the good article process more than they would help it.

One measure, however, emerged with a clear consensus as meeting the need for reform without compromising the strengths of the process. This is set out below to invite inspection and comments from the community of reviewers.

Open review proposal

It is proposed that:

• An article review is opened by an experienced reviewer, who then becomes the 'lead' reviewer for that assessment.
• The review remains open for a minimum period (say three days), during which time any other interested editor can leave comments or join in with the review. However, there is no requirement for extra reviewers to comment.
• After the minimum period, the review may be closed or extended at the lead reviewer's discretion, in line with the GA criteria.
• New or inexperienced reviewers are strongly encouraged to start out by joining in with a few 'live' reviews, where they can see how the process operates and what makes a good GA assessment, before taking on the lead role themselves.
• Where a lead reviewer is unfamiliar with the article subject (for example, for technical or academic articles), advice should be sought from a subject expert (perhaps the article editors or the relevant WikiProject).
• Quick fails and passes are deprecated. No opened review should be closed until a reviewer has left constructive feedback. In the case of a GA fail, this should be some guidance as to how to improve the article; for a GA pass, suggestions should be made for further improvement.
• Articles that are obviously unsuitable for assessment can be administratively removed from the nominations list by an experienced reviewer, with no recorded action taken other than a brief explanation for the removal on the article talk page.
• Review status can be tracked, perhaps on the GAN page, where information about the review (such as the nomination date, opening date, reviewer name etc) along with a link to the article and review sub-page will be listed. It is eventually hoped that this will be bot-generated, but the technical details need more work.

OK, posted this to WT:GAN. Thanks all for the corrections and copyediting! EyeSerene^talk 11:05, 15 July 2008 (UTC)

• I'd suggest this go to a village pump/cent for more eyes. —Giggy 11:14, 15 July 2008 (UTC)
  • Yeah, I did wonder about that. It's GA reviewers that would have to operate the system though, so I don't know if we'd derive any benefit from opening up the discussion that widely. My concern is that we could get derailed by editors who have no knowledge of, or interest in, the GA WikiProject. Maybe I'm being too parochial though? EyeSerene^talk
  • I agree. Keep it off of the village pump. This is an internal GA issue only. We don't need "community-wide" input on this. Dr. Cash (talk) 14:09, 15 July 2008 (UTC)
• Does this proposal applies to all GAs or just new GAs? Wouldn't this approach to solving the systematic bias in GA results in instruction creep? How will sweeps adapt from such changes? We are not thinking deep enough folks. OhanaUnited^{Talk page} 02:43, 16 July 2008 (UTC)
  • I think restarting the sweeps with this new technique, which doesn't change the criteria at all from what I can see, is a ridiculous idea. —Giggy 08:01, 16 July 2008 (UTC)
  • This would (I'm assuming) be for new GAs only, and I don't see it applying to sweeps in any meaningful way. The proposal is mostly intended to address quality issues caused by new or inexperienced reviewers. By definition, the sweep reviewers are trusted, experienced reviewers. Re instruction creep, I agree there's always a danger of that in any process, but this replaces, rather than adds to, the current system. To be honest, if we can get bot operation going, it should even be simpler. EyeSerene^talk 09:00, 16 July 2008 (UTC)

GA reform redux

Hello all! As you may (or may not!) remember, our above GA reform proposal was sent to WT:GAN for community review in mid-July. The discussion generated can be found in the archives here; it's worth a quick re-read, as some useful comments and suggestions were made. However, the discussion was archived within two weeks for lack of activity - in retrospect, with so many editors away on Summer vacation, I suppose this was inevitable.

I was wondering if the contributors to this working party think now might be a good time to revisit the reform proposal? I suggest that, if we're interested in trying to take this further, we could:

• briefly review the discussion and decide if we need to amend anything within the proposals.
• relist the proposal, advertising it more widely across the GA community this time (for example, perhaps including something in the next newsletter?)
• frame the relisting in such a way as to make judging consensus possible - for example, as a support/oppose/neutral type !vote, either for each sub-proposal or the thing as a whole.

I think it's worth taking one more crack at this, but would be interested to hear everyone's thoughts... EyeSerene^talk 13:09, 17 November 2008 (UTC)

Anything that addresses the perceived qualiy of GA is worth considering. So far as I can tell there are really only two significant differences between the proposed approach and the current process.

• Reviews to be kept open for a minimum of three days

• The distinction between experienced and inexperienced reviewers, which needs to be defined. Who decides who's inexperiencd and who isn't? --Malleus Fatuorum (talk) 13:35, 17 November 2008 (UTC)

We also need a way to control the quality of the review. There should be a mechanism that encourages inexperienced reviewers to review to the best of their abilities while those who are classified as experienced reviewers need to understand that these are not lifetime status. OhanaUnited^{Talk page} 13:55, 17 November 2008 (UTC)

Doing something lightweight, such as asking reviewers to look over some information and and then checking their reviews to make sure they don't muff the basics, would probably be beneficial, and we've probably got enough eyes available to maintain lightweight standards. Once you start getting heavyweight ... trying to figure out whether someone's reviews reflect whether they have the best interests of the project at heart, how familiar they are with policies and guidelines, that kind of thing ... then you've got a process that is in many ways similar to RfA. RfA has been very kind to people who create and review articles and who know a lot about content policies this year; anyone who really wants a stamp of approval for their hard work and encyclopedic knowledge of all things GA would probably get a fair reception at RfA, and it would be good for the RfA process as well to keep sending people that way who focus on things content-related. - Dan Dank55 (send/receive) 14:07, 17 November 2008 (UTC)

I really couldn't disagree with you more on the RfA issue Dank55, but best leave that for now. Ohana raises a very important point; it's not the quality of reviewers per se we need to be concerned about, it's the quality of the reviews. --Malleus Fatuorum (talk) 14:59, 17 November 2008 (UTC)

Oh, I missed this when I commented below a moment ago. I can attest, with my own RFA as proof (and I know there have been more), that GA members are not well-received at RFA by everyone. GA project participation can garner a lot of opposition. Additionally, adminiship is not a trophy for good content work. No one looking for a stamp of approval should submit a request to RFA. Quite the opposite, in fact. If you're looking to kill the content creator within you, then you request adminship. لennavecia 16:20, 18 November 2008 (UTC)

(EC) Malleus, I think it should be you who determines when a reviewer has reached the esteemed level of experienced. :)

Seriously, though. As noted in that headache-inducing discussion EyeSerene linked to above, experienced reviewers are nothing special... it's not a hierarchy, just the reviewers that aren't newbs. The recommendation is that someone new to GA reviewing work with some experienced reviewers before attempting to go solo, as many of us did. Everyone go read at least the first part of that archived discussion before discussing it. Seriously. Some may need to read it twice... because we don't want a repeat of that discussion, where everything is having to be explained to death because people are selectively reading. لennavecia 14:10, 17 November 2008 (UTC)

Oh, and on the points of the initial post here, I agree with all three. Personally, I like all of the proposed changes, but clearly (as shown in the archives) they are not easily understood in their current wording. Perhaps we need to expand the points of the proposal a bit before taking this to a larger group for review. Perhaps it's too early in the morning, but my cynical ass sees a serious need for detailed explanations on the semantics to avoid the issues of that previous discussion. لennavecia 14:16, 17 November 2008 (UTC)

Yes, there certainly seemed to be some confusion between quick-failing, failing-after-hold, and failing-with-review, and the suggestion of deprecating quick-fails met a fair bit of resistance. Personally I think replacing them with administrative removal would be good for the project, but if that's going to be a sticking point I could live with dropping that part. Jennavecia's right though - given the amount of talking at cross-purposes, the proposal obviously wasn't clear enough...so maybe someone else can write the next one :P EyeSerene^talk 15:07, 17 November 2008 (UTC)

Oops. I'm going to disagree with myself and agree with Malleus. What I was getting at was that if people want some heavyweight process, we shouldn't attempt that ourselves ... see RfA, both as a cautionary tale and as a place where some people hang out who have good judgment about these things (such as Balloonman). But if someone is aspiring to be a competent GA reviewer, it would be a useless distraction for me to say, "Yes, but have you thought about RfA?" - Dan Dank55 (send/receive) 14:05, 18 November 2008 (UTC)

I'm still not sure I completely follow that, Dan. However, I know I'm in agreement with Malleus, so I probably agree with you as well ;) EyeSerene^talk 15:57, 18 November 2008 (UTC)

I'm with EyeSerene in not understanding what Dan is talking about, but I think we can all agree that Malleus is right. :) لennavecia 16:15, 18 November 2008 (UTC)

I don't understand Dan's point either, but I do agree that I'm right. :-) --Malleus Fatuorum (talk) 16:18, 18 November 2008 (UTC)

Perhaps I should just listen to Malleus :) - Dan Dank55 (send/receive) 16:34, 18 November 2008 (UTC)

My feeling is that the original proposal packaged too many things together. At the very least we need to separate out the issues to allow comment on each.

• Open review. In my view this is the most positive part of the proposal, the idea that GAN reviews can be a collaborative effort. In this setting, the notion of "lead reviewer" is clear: the editor who opens the review is the one who makes the final decision (but with power, responsibility!). This is not a change in process at all: already anyone can comment on a GAN review. However, as a change in culture it could be as significant as the introduction of review subpages: the fact that other impartial reviewers may comment is a disincentive to less-than-impartial or inadequate reviewing.
• Training side-benefit. I don't think it is helpful to define what is an "experienced" or "inexperienced" reviewer. However, some are certainly more experienced than others, and it is surely helpful to encourage less experienced reviewers to comment on (and leave full reviews for) articles where they are not the "lead reviewer". This sharpens the wits of the lead reviewer as well as providing a mentoring opportunity. The regular posts at WT:GAN along the lines of "Just done my first review, whaddya think?" shows there is a need.
• Quick fails and all that. The main objection to abandoning quick fails was that it forces the reviewer to read the entire article in detail. This isn't fair or a good use of resources, so we need to be more flexible about what we ask of the reviewer. Initial comments could be "How come the article has clean-up templates in 7 sections, and 22 citation needed tags?" :-) or "The section on demographics contains unsourced statistics which need to be cited within the next few days, or the article will be failed without further review per 2b". You get the idea, and I think this is the way many reviewers (would like to) approach an article: are there glaring problems, and if so, can they be fixed before I have to read the whole damn thing? These are fails with brief review, but I still believe they should be open, and allow for response.
• Timescales. Closely related to the above, open review requires that reviews be open for at least 2-3 days. This benefits the nominator (a chance to fix those 7 sections or 22 citations, or at least argue that some of them are not valid!) and for the reviewer (a second pair of eyes may confirm or question the analysis). We need to be more flexible about timescales. I've never been a fan of the 7 day hold, although I understand the need to set deadlines, or GAN becomes a sickbed, which would be bad. In my view, once a review is opened, a dialogue between reviewer(s) and nominator(s) has begun, and the article is effectively on hold. At some point the reviewer may want to set a deadline and 7 days is not a bad one, but I don't see why this needs to be so prescriptive.

That's all. Geometry guy 22:10, 18 November 2008 (UTC)

(ec)That's a very clear breakdown of the issues, G-Guy. I think it would be wise to look at each point, so that a workable solution isn't tossed out because people don't like one part of it. The main thrust of the proposal was the concept of open reviews: the others were ways of dressing or operating, which need to be discussed, but not necessarily in concert with the main debate. Gwinva (talk) 23:04, 18 November 2008 (UTC)

Thanks, and I agree. The main thing is to decouple any debate about quick fails from the open review process, I think. Geometry guy 20:29, 19 November 2008 (UTC)

Assorted random thoughts. I was asked to drop by and offer my two bits. Having had a few GAs and having (infrequently) reviewed some GAs, my thinking is as follows: Montanabw^(talk) 23:01, 18 November 2008 (UTC)

1. Quick-failing needs to have pretty explicit criteria (i.e. Stubs, no references, C-class or lower, etc., whatever consensus decides) for the truly "not even close" articles. OTOH, a lot of B-class articles can be whipped into shape with an on-hold. An A-class article I worked on was on for GA was at the receiving end of a quick fail that appeared to be retaliation by a disgruntled editor. It ultimately was resolved in favor of the editors who worked on the article, but not without a lot of unnecessary kafuffel. Having a "floor" for a quick-fail would help avoid this kind of nonsense. Also, with a pretty clear set of guidelines, new reviewers could easily get stated and would do us all a service by going through new nominations and grabbing the clear quick fails, would be a great way for them to get an overview of what's out there, and to save the more complicated cases for those who want more meaty work.
2. It may be too complicated to designate experienced and inexperienced reviewers, but having a clear process by which a newbie could ask for a more experienced person to eyeball something would be helpful. When I have an article I want to pass for GA, I still almost always ask a more experienced reviewer for a second opinion, but I feel bad always bugging the same people.
3. For new and infrequent reviewers, maybe a rewrite of the whole page that describes the process, making it a bit simpler, would help. I find that I am starting from scratch every time I review something...it's actually discouraged me from reviewing because it seems so complicated.
4. Definitely leave reviews open 2-3 days. Now that my real life has cranked up, there are times I simply cannot get on wiki more than every other day. I am sure I am not alone.
5. On hold needs at least 7 days, with easy extension (to 14 and maybe up to 30 days?) if people are actively working on the article. However, there does need to be an outside limit at which point it has to fail for the time being...30 days, perhaps, or maybe less, but something.

That's what I have for now. Hope this helps.Montanabw^(talk) 23:01, 18 November 2008 (UTC)

Failing an article based on its class is a potential pitfall. What if one project says it's B and another says it's Start, which one will you listen to? What if that article is ready but only mislabelled as C-class? Can we have a volunteer system where the experienced reviewers can show the new guys how things are done? It's like job shadowing. And I strongly disagree with 30-day hold. 2 weeks should be more than enough for anyone to stop by 5 minutes a day to slowly improve it back. OhanaUnited^{Talk page} 00:45, 19 November 2008 (UTC)

I see your concerns about class. However, maybe there could be a simple set of criteria for a GA quick fail that would pretty much nail things that are no better than C-Class. For example, few if any footnotes, blatent POV, sounds like promotional material,etc...the obvious stuff, more to prevent hasty quick fails for poor reasons, but also to encourage quick-failing of things that are a long way from ready. (I'm still stinging from the retaliatory quick fail of an A class article, and the DAYS of wrangling before it was reconsidered, easily avoided by a clearer criteria) And maybe 30 days is a bit long, I just threw that out as an idea. Two weeks probably would be fine, absent exceptional circumstances. _Montanabw

Ohana, holds of 30 days happen. They are not an entitlement, nor to be encouraged, but the lead reviewer should have the freedom to manage the review as they think best. In particular they should be free to set a 7 (or 14) day deadline, as you would. But I see no benefit to the project in requiring all reviewers to set only 7 (or 14) day deadlines.

Montanabw, whatever we do must be based only on the good article criteria, not WikiProject assessments, so here I agree with Ohana. Personally I think nominators should always be given a chance to respond before a fail, but that reviewers should not be required to review in detail a clearly substandard article. Geometry guy 20:29, 19 November 2008 (UTC)

Comment from Gosgood

I am responding to EyeSerene's request for comment. My comments pertain to the July 15 version of the proposal.

Overall, I count only six proposals here, not eight. I agree with (1), (2), (3) with amendments. I find that (4) and (5) are sage bits of advice, not proposals. I agree with (6) and (7), so long as they are suitably worded to avoid the pernicious phrases 'quick pass/fail' I think (8) should not be advanced at this time because it is a functional requirement without particular implementation. I think they should be reposted, with votes gathered on the actionable items. So much for the executive summary. Details follow for those who like to delve into such.

1 An article review is opened by an experienced reviewer...

I propose striking the word 'experienced' unless we carefully define what we mean by the term. However, I think undertaking such a definition would open such a large can of worms that it alone would delay the redeployment of this proposal.

In light of that, I suggest, in a proposal preamble, defining a Good Article 'lead reviewer' as any registered editor who is (1) free of content or nomination conflicts of interest regarding the nominated article, who (2) agrees to take the lead in developing for the community an analysis that constructively criticizes both where a nominated article has merit and where it is weak regarding the good article criteria. Concomitant with these criticisms are various suggestions that the reviewer recommends to close the gap, and (3) as lead reviewer has final judgement on whether the article appropriately reflects the good article criteria and is deserving of the marque. The lead reviewer completes his or her effort with that judgement.

By defining the term 'lead reviewer' this way, we bypass the whole experience/inexperience debate and establish the functional deliverable that distinguishes the canonical 'Good Article Reviewer.' — essentially compliance with step three of the good article review process. In 'taking the lead' the reviewer is not obliged to follow a solo course. He or she may solicit input from other reviewers and should not prohibit voluntary commentary from other reviewers.

It's not the business of this proposal to insist on particular presentation formats or reviewer modes of operation. However, as an aside, <shamelessplugalert!>some time ago, Geometry guy fashioned {{GATable}} based on an assembled-by-hand version that I used which facilitates this approach. Current criteria on one side, GAP analysis and remedial steps on the other in a side-by-side format; summary marque on the extreme left. I'm immensely pleased with it, but I'm biased. And I'm not here to insist on its use. Though I think editors should. Really. ;)</shamelessplugalert!>

2. The review remains open for a minimum period (say three days),...

3. After the minimum period, the review may be closed or extended...

Let us say 'three days' and move on. The lead reviewer may adjust to any length of time that particular realities require.

4. New or inexperienced reviewers are strongly encouraged to start out by joining in ...

5. Where a lead reviewer is unfamiliar with the article subject ...

These remarks are good advice, not proposals, as Ruslik observed last July. Since I'm suggesting that a proposal preamble defines the term Good Article 'lead reviewer,' I further suggest incorporating these sage bits of advice as what a good reviewer would do as a matter of course, or what a new reviewer could do to get experience. Since they propose nothing, and give only advice, I recommend omitting them from the proposal itself.

6. Quick fails and passes are deprecated...

7. Articles that are obviously unsuitable for assessment can be administratively removed...

It took awhile for light to dawn over Marblehead; the latter seemed a back door admission to the former's prohibition. I came to realize that we use 'quick fail' to mean two different things, which confuses the issue. On the one hand, 'quick fail' pertains to the review itself. In that context, it signifies a trivial, superficial review that is quick to write. It has an equally pernicious twin: the quick pass. On the other hand, 'quick fail' pertains to failing a review quickly which is not the same as a superficially written fail review. Proposal six dispatches the first flavor of quick fail: the superficial review. Here, I would use stronger language than 'deprecate' I would write 'are not compliant with Good Article reviews (see preamble).' A review that does not highlight variance with good article criteria, and suggestions for bringing an article in line with that criteria is simply not a Good Article review, by definition.

Proposal seven pertains to the second flavor of the term 'quick fail'. There will always be some nominations that fall so wide of mark as to be immediately dismissed as gaffes. However that case still requires a clearly written reason why the article is being removed in a summary manner, for (assuming good faith, here) it is entirely likely that the nominator is inexperienced and has an imperfect grasp of the constituent qualities of a good article. The 'administrator' performing the removal is obligated to furnish, at a very elemental level, the reason why the article misses the mark ('It has one hundred, twenty seven citation tags in seven hundred words of prose. Falls a bit wide of 2b, don't you think?). That is not an easy level at which to write; The written reviews of quick fails of articles would be, primarily, educational tools for inexperienced nominating editors.

Since we habitually miusue 'quick fail' I propose keeping the term out of the proposal. Proposal six, I believe, should simply disallow reviews that are not based on the good article criteria or which lack remedial suggestions. Proposal seven should define the administrative removal of nominations that are essentially gaffes, but the removing reviewer is still obliged to furnish reasoning grounded on the Good Article criteria and pointers on how to bring the article to the threshold of nomination. Since we are dealing with a gaffe, the reviewer is not obliged to furnish detailed analysis. The failing should be obvious, else it is not a gaffe.

8. Review status can be tracked, perhaps on the GAN page... Nice bit of infrastructure, but I don't think the automation is there for the project to commit to the feature, so it should be omitted from the proposal.

That's my two American pennies. Since I'm playing hooky today, I should toddle off now and review an article. What shall it be? I have so much to choose from... Take care. Gosgood (talk) 21:26, 19 November 2008 (UTC)

• While the lead reviewer idea has promise, another alternative is to collect data on who closes reviews. At the end of the month, a one-off review is identified, and it can be verified by looking at that particular review essentially de novo. If everything checks out, add the user to a list of "reviewed" reviewers, and have random reviews every now and then as part of the Sweeps process. Titoxd^{(?!? - cool stuff)} 21:06, 28 November 2008 (UTC)

Next step?

Thank you all for commenting so thoughtfully and constructively. Having flagrantly flouted the MoS in this section header, I'll now try to set out a distillation below. As was suggested, I've de-coupled the open review suggestion from the rest, and left out the fluff - I agree that it's the most important part of the proposal, and doesn't deserve to be scuppered by the rest ;) As usual, please comment, amend etc as necessary. EyeSerene^talk 15:37, 24 November 2008 (UTC)

Looks good. - Dan Dank55 (send/receive) 16:08, 24 November 2008 (UTC)

Clear and understandable. Well done Edmund Patrick – confer 12:12, 25 November 2008 (UTC)

I think the Open review proposal is good.

I think separating the proposals out is excellent.

I think the use of passive voice in the first part the Administrative removal proposal is troublesome, for it is doing what the passive voice is good at doing: bypassing or making obscure who is (or should be) responsible for the action.

I believe this use of the passive voice reflects the fact that there is no formally defined role 'Good Article Program Administrator', though many active participants fill that role in an informal way. As written, the proposal seems to be tacitly assigning the responsibility to a GAP administrator without acknowledging that the office doesn't exist.

I dislike leaving in doubt who is responsible for the action, and would not support the proposal because of its indefinite nature. I would propose that any editor who qualifies as being a lead reviewer — registered, has no conflict of interest with respect to the article, is not a contributor and is cognisant of the task of evaluating articles with respect to the good article criteria — is qualified to perform an administrative removal and the first part of the Administrative removal proposal ought to be reworded in the active voice and task the lead reviewer with the responsibility. That same lead reviewer remains responsible for composing the simple, direct, and compelling explanatory message; it should not be a hard case to make if an article is truly wide-of-mark with respect to the criteria.

Excellent work, otherwise. Take care. Gosgood (talk) 15:03, 25 November 2008 (UTC)

Good work. Comments...

"first reviewer to leave a review". We need to move away from the idea that the reviewer reads the whole article and starts by leaving a complete review, since very few reviewers actually do this. Something along the lines of "start the GA sub-page which initial comments on the article" would work better, I think.

Copyedit?: "The review should remain open for at least 3 days; the period can be extended according to the lead reviewer's judgement".

At the moment "administrative removal" is "quickfail" without the record. Some view this as a good thing (the "no black mark" school). Some view it as a bad thing (the "how does this improve accountability?" school). I'm agnostic here. For me the main point is that it should happen much more rarely. That means lead reviewers need to be encouraged to close reviews after 3 days if they have raised serious concerns which have not been addressed even if they have not provided a complete review. Administrative removal (we may need a better name) should only be for cases like this, where there is no point in opening a review, leaving a few comments and seeing if anything happens. In such cases a brief message explaining which criteria it fails (most of them here!) and why suffices. Geometry guy 20:45, 25 November 2008 (UTC)

OK, thanks. I've tweaked the wording in the first proposal as suggested. Re the two, I think it would in any case be best to offer them for community appraisal on separate occasions (especially since the second clearly needs further work!) EyeSerene^talk 08:35, 26 November 2008 (UTC)

Update: I'll be away for a couple of days - are we OK to at least put the open review proposal to the GA community, leaving the other on the back-burner for now? I'm happy to do this when I get back, unless anyone else has more to add... (or wants to do the honours themselves!) EyeSerene^talk 13:57, 2 December 2008 (UTC)

Yup, that would be fine. Geometry guy 21:09, 3 December 2008 (UTC)

Open review proposal

• The GA review process is started by the first reviewer to leave comments on an article's GA sub-page. By doing so, they become the lead reviewer for that article.

• With the review now open, other GA reviewers are encouraged to participate and leave additional comments on the review sub-page.

• The review should remain open for at least 3 days; the period can be extended according to the lead reviewer's judgement.

• The lead reviewer is responsible for closing the review - either when a reasonable time period has elapsed, or when they believe all the review comments have been satisfactorily addressed. They will then promote or fail the article.

Administrative removal proposal

• If an article has been nominated for review, but clearly has significant issues (per Wikipedia:Reviewing good articles#First things to look for) that would prevent its promotion in the short-term, it should be administratively removed from the WP:GAN page.

• The nominator should be left an explanatory message, informing them why the article was removed and (briefly) what should be done before renominating. The article talk-page GAN template should also be removed. Beyond that, no further action needs to be recorded.