Wikipedia talk:Review aggregators/Archive 1

Archive 1

Similar websites

I added a "Similar websites" section saying that Rotten Tomatoes and Metacritic are well-established. If there is any issue with what I've written, please discuss it here. Erik (talk) 15:00, 11 March 2010 (UTC)

Rotten Tomatoes

• Rotten Tomatoes reported that 40% of 200 sampled critics gave the film positive reviews and that it got a rating average of 5.4 out of 10.

I've serious problems with this wording. Rotten Tomatoes does not really "report", it is not an completely objective evaluation. The critics are selected, the reviews are then interpreted [1] converting it to a 1-10 scale, etc. etc. etc. "Reporting" implies something less subjective and adds unnecessary weight, compared to saying "Rotten Tomatoes gives the film a score of 100%".

Further more it is misleading to say "40% of 200 sampled critics gave the film positive reviews", because it is overly simplistic. It is not like 80 out of 200 literally gave the film a postive review, and the other 120 gave the film a negative review, there is a spectrum of reviews and a review can be both postive and negative about different aspects of a film. The blurb need not go into detail but we can manage a better description than the one previously suggested, it could be better so I'm changing the suggested wording. Please discuss and we can change it more. -- Horkana (talk) 23:31, 24 December 2010 (UTC)

At Metacritic

Why say "At Metacritic" and not also say "At Rotten Tomatoes"? It was probalby me who changed the wording that has been reverted but, I see no reason for this inconsistency, can we not make the two phrases more simliar? Starting the sentence with "At" seems clunky too only in a different way.

If there is a flaw with the rhythm of the sentence, do try to rephrase it in another way and not just revert it, please? -- Horkana (talk) 19:19, 17 January 2011 (UTC)

The re-worded sentence was, "Metacritic assigns a weighted average score out of 100 to reviews from mainstream critics, the film received an average score of 49% based on 37 reviews." Did you want to include a conjunction? I find the original wording to be sufficient because we are not going to list the scores in bullet points. We tend to report Rotten Tomatoes then Metacritic in prose. If we start both sentences the same way, it would be more stilted, even though it makes sense in the bullet points. Erik (talk | contribs) 19:53, 17 January 2011 (UTC)

Now that you point it out, having the conjunction "and" in the middle of my previous wording would have flowed better. Would it be okay with you if I went with that instead?

I suppose I try to write economically and when writing for an encyclopedia not use too many words or overly complicated language, so including "At" seems unnecessary to me. I acknowledge that the wording suggested at the moment is widely used but that doesn't say anything more than it was good enough at the time. Similarly I find it unnecessary to explain "Review aggregation website Rotten Tomatoes" as some editors do, since saying that they calculate a score based on X number of reviews gets across the same message in quite a straighforward way.

I think perhaps what I see as consistent, you might see as stilted? It is good writing to mix it up a litte to stop the prose from being too repetative, and I've seen editors trying to do it in biography articles by mixing an occasional use of Firstname in with Surname, only to see an editor speedily revert it and insist on user only Surname through the entire article. I guess what I'm saying is that although I'd like better prose that I think consistency and slightly duller prose is actually what would be more appropriate for this that is so often claimed to be an encyclopedia. -- Horkana (talk) 01:54, 18 January 2011 (UTC)

Let's avoid bullet points, then. Should we write a brief paragraph using both Rotten Tomatoes and Metacritic for the sample film? That way, we can figure out the best way to transit. In addition, I think that the Metacritic sentence could explain the system better. While "and" fixes it on a technical level, it still seems to define the system on its own, then state the sample film's score. Is there a way to define the system in the process of stating the score? Maybe "Metacritic, which assigns a weighted average score out of 100 to reviews from mainstream critics, gave the film an average score of 49 based on 37 reviews"? I would suggest not using a percent sign to be consistent with the fact Metacritic doesn't use it. Erik (talk | contribs) 14:45, 18 January 2011 (UTC)

My view on it has long been that they both do so much interpreting of the reviews to generate their numbers that it is hardly worth explaining in any detail, but if others want to make it clearer that they add up their reviews in different ways it doesn't make much difference to me so long as they are not too verbose.

I've yet to see a case where the Metacritic score didn't closely align with the Cream of the Crop score, I expect they both seem to be sampling the same 30 or so critics anyhow. (If Rotten Tomatoes would just fix their broken website and not change what different regions see without proper deliniation I'd be glad to give the two ratings for comparision. Maybe I should write to them.)

% "Per cent" and "out of 100" are the same thing, and synonymous too. I think it better serves the readers to present the two as consistently as possible, their similarities are far more than their differences. The new layout much more prominantly shows "out of 100" than before, I hope eventually they will do the decent thing and just use % but perhaps someone feels spelling out the number is more user friendly. -- Horkana (talk) 17:58, 18 January 2011 (UTC)

Statistically accurate

The article says that 10 reviews is not enough to be statistically accurate but as I mentioned above there is so much subjective interpretation in the review aggregation method - not to mention the reviewers - that it hardly seems relevant to talk about statistically accuracy of something not easily quantifiable, like sporting statistics for example.

I tried to rephrase it, noting that Rotten Tomatoes themselves do not give a score unless there are more than 5 reviews. The mention of 10 reviews not being enough is oddly specific. How big a sample space would begin to be accurate enough?

More reviews helps lend greater confidence to the result, that is why it is so useful to give the number of reviews used to come up with the scores. Ultimately the input data is so fuzzy, that even talking about statistical accuracy gives undue weight to the figures. I will try to rephrase this again later if someone else doesn't give it a try first. -- Horkana (talk) 02:43, 25 December 2010 (UTC)

You wrote, "Rotten Tomatoes do not give a score unless they have a sample of more than 5 reviews but even a much larger sample set is not large enough for the score to be statistically accurate." Are you saying that even if Rotten Tomatoes has a set of 100 reviews, that's not even statistically accurate? Erik (talk | contribs) 04:04, 25 December 2010 (UTC)

"if Rotten Tomatoes has a sample of 10 reviews for an independent film, the sample is not large enough for the score to be statistically accurate."

There's something not quite right about this wording. It is irrelevant to say "independant film" this either applies to any film with not a lot of reviews or it doesn't. Considering we are talking about film reviews converted or reducted into numbers, it is quite odd to even talk about "statistically accurate". It seems more reasonable to say with more reviews comes a greater confidence. The number 10 seems arbitrary, even if you accept the premise that more reviews there is nothing to say how many before you should feel confident about the number. Oddly enough 5 reviews is the point at which Rotten Tomatoes give a percentage which suggests they do think 5 reviews is enough to generate a meaninful figure of some sort.
I'm tired, I'll have to come back to this another time, I'm not sure I'm making myself clearer. -- Horkana (talk) 03:34, 29 December 2010 (UTC)

The number 10 is not meant to be a specific one. What I was trying to illustrate is that when a film has so few reviews, additional reviews can affect the Rotten Tomatoes percentage enormously. After a certain number of reviews, the percentage is not going to change as much. Maybe around 35 reviews or so? Statistical speak is harder for me to convey compared to a few years ago. The point of it is that there will be some films that will just not garner many reviews on Rotten Tomatoes, and the percentage is used in the article to reflect the critical consensus. So if the number of reviews is 5 or greater, but still below maybe 35, then the percentage isn't an accurate picture of the film's reception. It's a suggestion to use RT in a smart manner, because there are editors who will add the RT detail merely because the page exists, not because it is useful. Let me know what you think. Erik (talk | contribs) 16:04, 29 December 2010 (UTC)

oKay I dont think we disagree on the intent. I'm not entirely happy with the wording so I'll try rephrasing it agian. -- Horkana (talk) 16:58, 29 December 2010 (UTC)

This point is mostly relevant for older films, recent internet writing about them tends to be largely positive, reviewers don't tend to go back to films they don't like.

Regarding the Top critics, it seems ridiculous to say that is to be judged harshly for being to small. Wikipedia strives to include information from quality sources, Top Critics should be the only sample used!! We need a larger sample of better reviewers, not to drop standards. Best, --Ktlynch (talk) 07:30, 25 August 2011 (UTC)

Critical consensus

There was a discussion a while back at WT:MOSFILM where an editor complained that the Rotten Tomatoes consensus is sometimes written early and not changed when many more reviews become available. This might be worth noting in the limitations of Rotten Tomatoes. -- Horkana (talk) 03:21, 14 November 2010 (UTC)

I'd like to see notes about this added to the article. I will add them myself if I do not recieve any comments. -- 109.77.40.121 (talk) 01:06, 7 December 2012 (UTC)

Metacritic TV

I've also noticed that when Metacritic gives scores for tv shows they are listed as a score for "season 1" although most of the reviews are actually reviews of "episode 1" as opposed to reviews of the whole season when a show is released on DVD/home media. Another details for editors to be careful about and readers to be skeptical. (This reminds me of how Rotten Tomatoes has the problem of the concensus being written based on earlier reviews, so early reviews bias the score/concensus.) -- 109.77.179.64 (talk) 15:07, 11 December 2012 (UTC)

Rotten Tomatoes SSL Support

Please add SSL support to all Wikipedia links to Rotten Tomatoes, since the Rotten Tomatoes webserver supports SSL. For example: https://www.rottentomatoes.com. — Preceding unsigned comment added by 64.128.27.82 (talk) 19:26, 6 August 2012 (UTC)

Why? I don't mind either way and I use HTTPS everywhere so my links are usually HTTPS anyhow but I'm don't understand why this is something editors should make an extra effort to do. Please explain. Also shouldn't there be a bot to do that sort of thing automatically, for all kinds of sites not just RT? -- 109.255.20.91 (talk) 23:56, 5 May 2013 (UTC)

Grammar

There is a mistake I see over and over, it is so frequent I'd nearly need a bot to fix it, or a whole lot of help which is why I'm pointing it out here. This article already includes a recommended wording but many editors do not use it so I need to state here very clearly:

"Review aggregate" is wrong. 

It is simply not grammatically correct, it is nonsese. "Review aggregator Rotten Tomatoes" is okay (although my spellchecker does not like the word aggregator), and my preferred wording is slightly pedantic but unambiguously clear "Review aggregation website Rotten Tomatoes".

So please if you see any articles using "Review aggregate" please jump in and fix it, there is more than I can possibly fix on my own. -- 109.79.221.209 (talk) 22:26, 27 August 2013 (UTC)

Policy discussion

I started a discussion about the use of aggregators as a policy at Wikipedia talk:Neutral point of view#Review score aggregators like Rotten Tomatoes, Metacritic and OpenCritic. SharkD  Talk  03:56, 31 December 2015 (UTC)

Statistical Unreliability and Reviewer Subjectivity

Firstly, individuals who review movies can not be objective. It's impossible. The nature of movies as entertainment means there will be conflicting opinions, no matter how well or badly the film is made. So from the very beginning, a review is purely subjective in nature, reflecting how one single individual feels about a film. When reducing such data to a quantitative level the resulting numbers fail to take into account any biases and other confounding variables that may be in play when the reviewer writes the review. Also, since there is a high level of subjectivity with such data, there would need to be extremely large samples used in order for the data to be generalised to a greater population. Basically, Rotten Tomatos using 14 reviewers is pathetically insufficient to represent a widespread, generalisable view of the film. To reduce the data to quantitative numbers and percentages, there would need to be samples numbering in the thousands to achieve an acceptable level of external reliability. As it stands, the statistics representing the critical reception of a film are greatly flawed and indicate virtually nothing in regards to widespread opinion of the film in question.

As this is supposed to be an online encyclopedia, I see no place for overly subjective sources such as Metacritic and Rotten Tomatos.

A concerned user. 79.66.110.168 (talk) 20:18, 21 January 2012 (UTC)

It just so happens I agree. The false assumption that opinions can be averaged, especially on non-"scored" input, grates on my nerves. Most reviews with grades or numeric scores, ok, maybe they can be "averaged" (yuck) (they really should be binned, and only binned). Reviews which state no strong opinion, or state conflicted opinion internally, resist "scoring" valiantly, and undermine the notion of RT and MC "scores" entirely. --Lexein (talk) 17:02, 17 December 2012 (UTC)

But that is why they draw on a broad range of contributors. 500 people cant all be wrong can they? MisterShiney ✉ 20:52, 17 December 2012 (UTC)

This is a very late posting to this section, but I agree that Metacritic should not be trusted too heavily as a reliable source, as their review system is biased towards certain users and is often considered mathematically unsound. See Metacritic#Criticism for details. -- Matthew - (talk · userpage · contributions) 00:51, 25 June 2016 (UTC)

Clarification on metacritic in articles

Hi. I am wondering if it's okay to have metacritic be referenced under a "critical reception" header and also in the Review Scores box for an album page. I think it's better for an 'at a glance' purpose. Needing clarification. Thanks --Jennica^Talk 00:45, 24 October 2016 (UTC)

Jennica, this was written for film articles. The general consensus has been against score boxes in such articles. However, album articles should be okay. I found this that may be useful for you (and answers your question): Wikipedia:WikiProject Albums/Sources#Aggregates. Erik (talk | contrib) ^{(ping me)} 14:22, 25 October 2016 (UTC)

Phrasing in examples

The examples from a 2008 film listed in this essay describing the use of RT and MC in the "critical response" section reflect language that has become increasingly uncommon in recent film articles (see Star Wars: The Force Awakens and Jurassic World). Instead of stating that a film aggregator "gives" the film a score, many articles now have a tendency to state the film "has" or "received" a rating. Personally, I think the latter is preferred, as it avoids describing RT and MC as entities giving something. This preference seems to be reflected in the number of mainstream film articles using it. While you would think this is a minor concern, apparently it's not to some editors (diff1, diff2). I understand that something like this isn't worth edit-warring over, but it would be nice for us to agree on a general, recommended format.

Here's the proposal:

• On review aggregator Rotten Tomatoes, the film has a <percentage> approval rating based on <number> reviews, with an average rating of <number> out of 10. The website's critical consensus states, "<consensus summary>".
• On Metacritic, the film has a score of <number> out of 100 based on <number> critics, indicating "<consensus summary>".

Like all guidelines, this essay's example format would just be a recommendation and not a requirement. There would be no need to jump from one article to the next, changing the critical response sections to match these examples. However, having something here may assist in high-traffic articles where minor disputes over phrasing may arise. --GoneIn60 (talk) 02:21, 2 August 2016 (UTC)

I am partial to the following wording:

The review aggregator website Rotten Tomatoes reported a <percentage> approval rating with an average rating of <number>/10 based on <number> reviews. The website's consensus reads, "<consensus summary>"

Metacritic, which uses a weighted average, assigned a score of <number> out of 100 based on <number> critics, indicating "<consensus summary>".

- Favre1fan93 (talk) 03:47, 2 August 2016 (UTC)

• Comment I am not very keen on the "approval rating" lingo for Rotten Tomatoes, since if you are not familiar with Rotten Tomatoes it is not immediately clear how the percentage should be interpreted. The Tomatometer score is basically the percentage of reviews that RT deem to be positive, so maybe it is more to the point to say "Review aggregator Rotten Tomatoes surveyed 209 film reviews and judged 40% of them to be positive, with an average rating of 5.4 out of 10." I think that sums up what each number means. With Metacritic I think the proposed revision is better, not least for the fact that the example currently gives the score as a percentage, when it is actually just a score out that is arbitrarily normalized to 100 (and more akin to RT's average rating rather than its percentage). Betty Logan (talk) 23:44, 2 August 2016 (UTC)

I agree that the wording around the numbers must be absolutely clear; a reader unfamiliar with either site (or simply unaware/uninformed of what the numbers actually mean) shouldn't have to look elsewhere for that understanding. The examples on the essay page aren't quite that. My preferred wording back in the day went something like:

• Metacritic, which assigns a normalised rating out of 100 to reviews from mainstream critics, reported that reviews were ["generally x"] with a weighted average score of [x] based on [y] reviews.
• As of [date], [percentage] of [y] critics listed by review aggregator Rotten Tomatoes had given the film a positive review, with an average rating of [z].

While Metacritic's two-or-so word summary is probably OK to quote directly, I'd ditch the same for Rotten Tomatoes', which is arguably more subjective, recommending instead that if it's to be included at all, it should be paraphrased with appropriate context. Thus, we end up with something like we have at Changeling (film)#Critical response, which I think hits the requirements without looking too boilerplate:

• Critics generally judged Changeling well-acted and beautifully shot, but considered the compelling story to have been told too conventionally.^[1] Metacritic, which assigns a normalized rating out of 100 to reviews from mainstream critics, reported that reviews were "generally favorable," with an average score of 63 based on 38 reviews.^[2] As of June 10, 2014, 63% of 196 critics listed by review aggregator Rotten Tomatoes had given the film a positive review, with an average rating of 6.3/10.^[4] The film's reception in several European countries was more favorable,^[4] and in the United Kingdom 83% of critics listed by Rotten Tomatoes gave Changeling a positive review^[5]

The important thing to note is that in an ideal world the use of these numbers is contextualised, or at the very least not relied upon as the only determinant of a film's critical reception (Rotten Tomatoes itself sometimes publishes articles that can provide that context, as in the example above). Steve ^T • C 21:19, 3 August 2016 (UTC)

This has been bothering me for a while, but using a conjunction (and) rather than a preposition (with) would be the proper way to connect two ideas ("it is this and that", not "it is this, with also that"). Also, when reporting review aggregators, we can't simply say that "critics said" or "critics judged", even if it's hedged with "generally". These review aggregators have weaknesses, which I have described in detail in this thread. When Rotten Tomatoes or Metacritic post a consensus, it is only the consensus of the sources they have polled, and they do not poll a representative sample (non-English sources and non-American sources are frequently, if not completely, absent, even for non-American and non-English films). NinjaRobotPirate (talk) 18:15, 4 August 2016 (UTC)

I'm looking at this again because some editors are following the examples here and then writing "On Cinemascore" which is nonsense. Even writing "On the Cinemascore website" is missing the point, Cinemascore is an actual survey company (see also Rentrak) that directly surveys audiences, as opposed to just another web based poll of people who might not even have seen the poll.
There is a more fundamental thing about this wording that bothers me: It is bizarre that some editors still insist on claiming that "out of one hundred" and "percent" do not mean exactly the same thing. When you google search the Google generated content does not, make that absurd distinction and gives you a percent for both. Some editors still hold on to the idea that differences between RT and MT are somehow more than their huge similarities and we ended up with this unhelpful distinction. (You say it's apples and oranges, but I say it's all comparing fruit, by a slightly different algorithm.) I still hold the opinion that the wording for RT and MC should be the same or very close to the same but I never did get consensus for that even in the examples in this limited essay. Not that it matters anymore, the Google generated quick content is what more people will see.

Way way back when this wording was suggested, and largely ignored, there were a lot skeptics who didn't even want to use Rotten Tomatoes or Metacritic (the selection bias of reviews when it comes to older films is just one of many problems). Thanks to this essay/guideline we at least got to the point where the number of reviews used to create the score are actually listed, and the description is somewhat consistent. When you get stuck into these discussions about all the flaws of Rotten Tomatoes it is difficult not to want to be precise and we all end up arguing about what precise things really matter.
The reason I reject the phrasing reported is that both sites take a review and interpret the stars or letter ratings and then turn that into a number quantity to plug into their algorithm but it is entirely subjective. It is not clearly defined, it is not like the score in a sports game, and I still have reservations about the use of the word "report" which suggest objective reporting of facts when there is so much subjective interpretation at so many stages along the way.

Addressing some of the specific suggestions:
On review aggregator. The preposistion "on" makes sense only if we are saying "on the website Rotten Tomatoes". If it was a film censor, rather than a film review aggregator, we wouldn't say "on film censor". Yeah, pedantic I know but if you are suggestion boilerplate text the grammar should be the best it can be.

As of [date] has been suggested before but in most cases the date of when Rotten Tomatoes generated their score, or more accurately the time it was written into a Wikipedia article, is not useful context. The number of reviews provides some context, enough that a reader can tell a film isn't getting a super high score because hardly any reviews are in yet. In cases where a film has been reviewed long after it was released, or a film was rereleased or retroactively became better regarded (Shawshank) a more descriptive wording is needed anyway.

NinjaRobotPirate mentions the "consensus", which again is more interpretation by the site's in question and there have been cases where Rotten Tomatoes decided the consensus early on, and many more reviews came in after that.

There used to be a Wikitemplate for Rotten Tomatoes, maybe there still is. I never liked the choice of wording but not sure why it stopped being used.

This is just one example of the many cases were Wikipedia fails to achieve consistency, and even the best new proposal means a whole lot more inconsistency and only a small chance older articles will ever be updated to follow new suggestions. Even then it is a lot of the same few editors who write the early drafts of the film articles and that is what sticks and has more influence than this guideline. Previous efforts only got as far as this article, and didn't get enough support to become a more solid part of WP:MOSFILM and without that we continue to get an inconsistent mish mash. Good luck to the brave soul who tries to get a consensus on this bikeshed. -- 109.77.244.201 (talk) 15:33, 31 March 2017 (UTC)

Place last

I revised the "Place last" passage because it contained advice that contradicts the general practices we have. While it can be suitable to put Rotten Tomatoes and Metacritic last, to establish that as some kind of rule is a bit much. I think what matters is to have prose upfront so readers can immediately get a sense of what critics think of a film. (This is supported by the actual guidelines.) Sources like the Los Angeles Times can be used to this effect. Aggregate scores can help fill out the big picture. I also removed the part about "commercial entities" because it is irrelevant; many sources both reliable and unreliable will have some sort of commercial backing. Are we to dismiss Box Office Mojo as a source for box office statistics because it's commercial? Like it or not, aggregate scores are one aspect that readers look forward to in a film article. We can talk about presentability, but a source's commercial nature is not relevant. Erik (talk | contribs) 13:52, 3 July 2013 (UTC)

We start articles with a summary, most writing introduces what they are talking about and then go into further detail. In most cases it makes sense to put RT/MC first. There are still some (but not many) cases where it might make sense to put it elsewhere, such as films made before 2000 that Rotten Tomatoes only reviewed retrospectively and particularly in the cases where films may not have been as popular at the time but became more popular over time (The Shawshank Redemption?) or got better reviews after a special edition rerelease. In effect the article already says this but without the added emphasis on where exactly Rotten Tomatoes should be placed. -- 109.76.174.27 (talk) 15:01, 11 July 2013 (UTC)

Maybe the wrong place for this discussion, but I've noticed a disturbing trend of pages spammed with Rotten Tomatoes information in the Lead section *and* later headings like Reception. This frequently includes duplication of the RT blurb, eg. "It has a 20% approval rating on Rotten Tomatoes, which called it "only moderately amusing."" Even if a few words *could be* representative of a critical consensus, this elevates anonymous copywriters above the many critics with real names and citations whose opinions the aggregate supposedly represents. Whatever justification there is for the ubiquity of the Tomatometer on Wikipedia, the averages don't describe films, they describe reviews; the blurbs describe the averages; neither should appear in the Lead section. Paulspyder (talk) 06:36, 10 October 2018 (UTC)

Overlink

I want to make a statement about WP:OVERLINK here before I try to change the essay. The Overlink guidelines make it pretty clear that less is more, and that having many links in sentence doesn't improve clarity but often distracts readers from the content. With this in mind I would say that while it worthwile linking primary keywords such as Rotten Tomatoes or Metacritic so that readers can follow those keywords if they need more information, but it is an unhelpful distraction and a clear example of overlink to link other words such as review aggregator or weighted average. Some editors have not yet come to understand WP:OVERLINK but I think over time and with hindsight it will become as clear as MOS:BOLD and editors will eventually learn that less is more. -- 109.78.247.34 (talk) 00:20, 26 August 2018 (UTC)

I don't mind de-linking "review aggregator" and "weighted average" since both are pretty overt in what they mean. The links for RT and MC will probably have "review aggregator" linked in a more revelant sense. "Weighted average" not be as obvious, but I don't think linking to an encyclopedic article is tremendously useful. Curious to know other editors' thoughts. Erik (talk | contrib) ^{(ping me)} 14:09, 26 August 2018 (UTC)

"review aggregator" links do definitely seem like overkill. I'm torn on "weighted average" because I'm not sure that's a term laypeople would necessarily understand without some background. Of course, the generic article which is linked to doesn't do much in terms of explaining how the review aggregator sites are using the concept. —Joeyconnick (talk) 20:18, 26 August 2018 (UTC)

"Weighted average" should definitely be linked because it is a technical term that not everyone will be familiar with. Betty Logan (talk) 22:33, 26 August 2018 (UTC)

Betty I understand why you would say that but from reading WP:OVERLINK I think it is far more helpful to readers of an average film article to only link Rotten Tomatoes and Metacritic where users should get all the extra details they might need if they aren't already familiar with review aggregators. Extra links are a distraction. Less is more, linking the biggest keyword is better than linking several words, even if those words seem important, in the general case they are a distraction. At first I found WP:OVERLINK surprising and maybe a bit too much, but then again I also thought MOS:BOLD was harsh at first, but in hindsight Wikipedia is better for it. In most cases I think one single wikilink per sentence is best, and although there are a few exceptions I don't think this is one of them. -- 109.78.247.34 (talk) 01:18, 27 August 2018 (UTC)

I don't think readers should have to click twice to get up the page for a technical term. It's simpler to cut out the middle man. Betty Logan (talk) 01:27, 27 August 2018 (UTC)

The point WP:OVERLINK makes is that very few readers click on either link. -- 109.78.247.34 (talk) 02:00, 27 August 2018 (UTC)

OVERLINK makes an argument for avoiding repeated links and links to self-explanatory and obvious terms. MOS:UNDERLINK on the other hand states that links should be created for technical terms, so I think the MOS supports a link in this particular case. Betty Logan (talk) 02:55, 27 August 2018 (UTC)

Average is a generally understood term (even then there is the question which type of average). Weighted average is a little bit more specific, but it is not anything special, it is not a technical term and it is not a wiktionary:term of art. Anyone who would need explanation in this context of review aggregators, is unlikely to want the level of detail on the page weighted average, and it is only distracts from the links to the pages where they would get the most context relevant explanation of weighted averages, which would be the links to Rotten Tomatoes and Metacritic.

Shame we can't get specific statistics about the links in film articles that people actually click on. -- 109.79.79.43 (talk) 19:04, 2 September 2018 (UTC)

Joeyconnick the Metacritic article links weighted average in the intro and explain the specific context of weighted averages as used by Metacritic it in the article text. Do you think the Metacritic article needs improvement? -- 109.78.247.34 (talk) 01:21, 27 August 2018 (UTC)

As seen here, I agree with Betty Logan. The IP needs to stop going around removing these helpful links and claiming that he or she is adhering to WP:OVERLINK. And the IP especially should not be edit warring over it as though there is consensus for the removals. Flyer22 Reborn (talk) 02:53, 21 December 2018 (UTC)

I also agree with Betty, if anything people would understand Rotten Tomatoes or Metacritic more than weighted average, not that I’m arguing for delinking those. yorkshiresky (talk) 07:40, 22 December 2018 (UTC)

I've been referred back to this discussion. I don't think liking either of these terms is actually helpful to readers, when Rotten Tomatoes and Metacritic are already linked. At most I see a weak consensus here in favor of keeping a link for weighted average, I don't see a local consensus that WP:OVERLINK is wrong. I'm asking WP:MOS for more information about Overlink. -- 109.76.143.176 (talk) 19:52, 31 December 2018 (UTC)

Note: The IP has taken the matter to Wikipedia talk:Manual of Style/Linking#Overlink. I weighed in there as well. Flyer22 Reborn (talk) 22:39, 31 December 2018 (UTC)

ASOF

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

The Rambling Man has suggested that the wording for review aggregators like Rotten Tomatoes should follow WP:ASOF due to the likelihood that the numbers will become out of date. I don't know if this has been discussed here before, but I was hoping to get some more thoughts from the community regarding this before going ahead and following this advice, especially as I think there should be general continuity between the different mentions across film and TV articles. Do others think this is something we should be doing? - adamstom97 (talk) 21:47, 19 November 2018 (UTC)

To expand on that, I was performing a GAN review and discovered that one source (Rotten Tomatoes) was way different from what was claimed in the article. This is a classic scenario for the use of WP:ASOF to ensure our readers understand why the details in the article (which sometimes claim to be Wikipedia's best) differ from the sources they reference. It's nothing special, it's a Wikipedia-wide editing guideline, and should be used for precisely these situations. It doesn't need to be rolled out across every article in every circumstance, but in situations where it's deemed necessary, e.g. when review numbers and statistics change significantly over an eight-month period, the Wikipedia-wide guideline should be followed. After all, it's the addition of four words: "As of X Y" (where X is month and Y is year). And nothing more. And all it does is improve verifiability of facts for our readers. The Rambling Man (talk) 21:53, 19 November 2018 (UTC)

Seems reasonable to me. Basically anything that is related to a time and place should use it, as it gives a more correct context to the following fact. --Gonnym (talk) 21:55, 19 November 2018 (UTC)

Just expanding on my personal thoughts, I don't believe that there are any instances where we can be sure the RT score won't be changing, so if others think this change can be good in some instances then I believe it is only logical to apply it to all articles citing RT scores. - adamstom97 (talk) 21:57, 19 November 2018 (UTC)

Indeed, but there's no need to wait for a universal project-related mandate to implement an existing Wikipedia-wide editing guideline. The Rambling Man (talk) 22:03, 19 November 2018 (UTC)

It was very good of The Rambling Man to bring this up but I think it is important to note that we are experiencing a temporary but massive distortion. Rotten Tomatoes approved about 200 more critics in August 2018, which has changed many scores. This has been good for smaller genre films, giving them the 5 or more critics needed to have a Rotten Tomatoes score. Although Rotten Tomatoes scores do change a bit gradually over time I think what we are seeing at the moment is particularly unusual. (Metacritic scores barely change at all by comparison).
Verifiability is important to editors but I don't think readers are as worried. So long as the number of reviews used to create the score are listed, I think that gives enough context for readers to understand that a score might only be high because there were not many reviews yet. For editors we can check access dates.
On a more fundamental note I wouldn't want to encourage readers to put any more undue trust or emphasis on Rotten Tomatoes than we do already. RT provides is a quick overview but despite the simplistic system people take it way to seriously. (I don't think I have to explain in detail to the kinds of film fans that are reading here, what happens when reviews are polarized and they love or hate a film, which RT simply isn't designed for).
The ASOF template is good when editors can be convinced to use it (and so many mentions of RT fail to even include the number of reviews counted), but it is overkill for Metacritic and probably not worth it for Rotten Tomatoes. -- 109.76.239.61 (talk) 23:42, 19 November 2018 (UTC)

I think the point is that it is website-agnostic. Whether it's Rotten Tomatoes or any other dynamically updated website, the WP:ASOF approach (i.e. adding FOUR EXTRA WORDS which are factually correct) is simply normal and useful for our readers. There's no such thing as "overkill" when it comes to providing expected verifiable results to our readers. Why is the film project so different to all the other projects where data can change from month to month? The Rambling Man (talk) 23:47, 19 November 2018 (UTC)

• Comment the use of the ASOF template will add four words to each article, and nothing more. And those four words will be to make the article more verifiable to our readers. Can someone explain why we should not comply with a Wikipedia-wide editing guideline by adding this? The Rambling Man (talk) 23:49, 19 November 2018 (UTC)
  • I don't think using ASOF is any great burden for the most active editors working on the latest greatest articles. I think it might be worthwhile to use ASOF more often, in some cases but I don't think changes to aggregator scores are important, so long as we can see if the score are generally positive, mixed or negative. I don't think readers are all that concerned about that level of verifiability or particularly bothered by a difference of a few percentage points as a Rotten Tomatoes score settles down. For editors the reference access dates should give us that extra detail anyway.
    I've made the point I wanted to make about Rotten Tomatoes being more screwed up than usual at the moment, and that we take Rotten Tomatoes way too seriously. Other opinions exist. Best of luck with the rest of the discussion. -- 109.76.239.61 (talk) 00:27, 20 November 2018 (UTC)

The Rotten Tomatoes score is usually updated by editors (IPs or registered, but more so IPs). Plus, if a score was last changed by Rotten Tomatoes two years ago, we will get some editors changing the "As of" date to be current because the score is still the same two years later. So if the article states "As of 2016," we will get editors changing the date to "As of 2018." So taking all of that into account, I don't see the need to have "As of" at every film article. But I don't strongly oppose it either. Flyer22 Reborn (talk) 02:43, 20 November 2018 (UTC)

I'm not sure if you got the point. I was reviewing an article where the scores in the article were not verifiable by the source at Rotten Tomatoes because the numbers had change substantially. That is precisely what WP:ASOF is about. Of course the score is usually updated by editors, who else would do that? And if the score is the same, no problem, and the as of date can be updated, or not, it doesn't matter, but at least it's verifiable. The Rambling Man (talk) 08:31, 20 November 2018 (UTC)

I get the point. I'm just not big on stating "As of." And I've seen "as of" considered out of date by editors or readers and then removed. This is why I stated that "So if the article states 'As of 2016,' we will get editors changing the date to 'As of 2018.'" In some cases on Wikipedia (meaning beyond film articles), an "as of" change is incorrect because the sources are about 2016 rather than 2018. Flyer22 Reborn (talk) 03:03, 21 December 2018 (UTC)

• Support the use of Template:As of (or the same information conveyed without a template.) per TRM. Airbornemihir (talk) 21:16, 8 December 2018 (UTC)

The discussion above is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

RfC: Should the "As of" template, or some similar wording indicating that the score may have changed over time, be used for review aggregators in articles?

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

As seen at Wikipedia talk:Review aggregators#ASOF, there was little discussion on whether or not the {{As of}} template, or some similar wording indicating that the score may have changed over time, should be used for review aggregators. This RfC is to gauge wider opinion on the matter. I'll alert WP:Film, MOS:Film, WP:TV, MOS:TV, WP:Video games, WP:Manual of Style/Video games and WP:WikiProject Music and WP:Manual of Style/Music to this RfC since film, television, video game articles and music articles, especially the first three, commonly use review aggregators. If someone feels that another WikiProject or MOS page should be notified, feel free to do so. Flyer22 Reborn (talk) 00:33, 24 January 2019 (UTC)

• I am fine with this being brought to a larger audience. @Adamstom.97, The Rambling Man, Gonnym, and Airbornemihir: (sorry can't ping IPs, will leave a talk page message) for completeness, although I would assume the closer would take the above discussion into account. AIRcorn (talk) 00:58, 24 January 2019 (UTC)

Survey

• Depends. As I've stated above, the "As of" template or "As of" wording without the template can also be outdated. If a score was last changed by Rotten Tomatoes two years ago, we will get some editors changing the "As of" date to be current because the score is still the same two years later. So if the article states "As of 2017," we will get editors changing the date to "As of 2019." I'm aware of WP:Dated, and, from what I've seen, the "As of" wording is usually unnecessary. Flyer22 Reborn (talk) 00:49, 24 January 2019 (UTC)
• Unnecessary As long as the aggregator score is immediately cited with the proper accessdate, the As Of is unnecessary. And I think its fairly obvious that during the first week or two of release of a reviewed product that the score will shift a lot, that no reader expect an "As of" when that score could change later in the day. Its key to have the citation with the right access date though. --Masem (t) 01:26, 24 January 2019 (UTC)
• Comment – Clearly, there is a benefit to using the {{As of}} template in situations where the content may change, especially when it is likely to change. The problem, as Masem points out, is in the first days and weeks following a product's release. Reviews are pouring in and aggregator scores are changing quickly. Even when you break it down by day using the template's available parameters, it fails to cover minute-by-minute and hour-by-hour changes, rendering it useless during this time. So common sense would tell us to recommend its use after the dust settles a bit and the score isn't rapidly changing, perhaps after the first month – sooner or later depending on the type of product. Since this is an essay, sure go ahead and throw a recommendation in there referencing the WP:ASOF guideline if that's the consensus. But if the motivation is to offer advice to the general editing community (such as at MOS:FILM), well we really need to be having a discussion at a more relevant talk page. My 2¢ for what it's worth. --GoneIn60 (talk) 04:47, 24 January 2019 (UTC)
• Unnecessary per Flyer22 and Masem. My issue with this, as others have sort of addressed, is, at this time, aggregators such as Rotten Tomatoes and Metacritic do not indicate on their sites when the last time they updated. So we will get in a situation where editors will be attempting to make the "As of" template date to be current, which doesn't really need to be done. For example, let's take a look at something like The King's Speech which released in 2010. Upon the film's release, your going to be getting a flurry of reviews added to the aggregator, so I could see a use of "As of" here. But fast forward 5-6 years, those reviews have stopped coming. So "As of" isn't needed here in this instance. As long as the accessdate parameter in the reference has been updated to the last time an editor checked the data, that's fine. Currently the "accessdate" on The King's Speech is March 2018. I have no issue with this. And while writing this up, I went to check the data to make sure that was still the case, and it was, so no need to update the accessdate. - Favre1fan93 (talk) 18:17, 25 January 2019 (UTC)
• Use. Arguing in this specific situation against ASOF is an argument against ASOF entirely, so those objecting to its use here should probably consider whether or not WP:ASOF etc should be removed in toto. And I say this only as someone who found data well out of date when performing a review. Had "as of" been there, no problem. And this sudden belief that users update "as of" and somehow don't update "accessdate" is bizarre. The Rambling Man (talk) 20:57, 25 January 2019 (UTC)
• Unnecessary usually, with limited exceptions. I think "As of" is perhaps fine for the two weeks after a piece of media is released and is the same as those mostly-unhelpful "this is a current event, article content is volatile, check your newspaper" templates. After that, the score is generally pretty stable; if it adjusts 1 or 2 points, it's not a problem that Wikipedia doesn't real-time mirror it, any more than WP not including new info from later editions of a book or the like. SnowFire (talk) 03:03, 27 January 2019 (UTC)
• Use. I personally have seen no arguments that convince me this is even worth discussing, given that WP:As of is an editing guideline. Airbornemihir (talk) 06:04, 13 February 2019 (UTC)
• Unnecessary per Masem and Favre1fan93. - Brojam (talk) 01:56, 10 March 2019 (UTC)

Discussion

By the way, I'm not really sure what's being debated here. We know there are scenarios where it's pointless to add the template as described above, but that's only temporary. Also, any editor wishing to add it has the backing of the WP:ASOF guideline, regardless of when they choose to. I'm not sure how we can argue against that, especially in an essay, unless I'm missing something. --GoneIn60 (talk) 05:00, 24 January 2019 (UTC)

GoneIn60, the reason I started this RfC is because it's best not to have anyone going around enforcing this change on articles based on the previous small participation. That section was closed as "you should do this." We already have or had an IP going around messing with wikilinks partly based on little participation -- participation that doesn't even have consensus for what the IP was (and may still be) doing. Plus, WP:ASOF is a guideline, not a policy. And it doesn't state that "As of" must be used. An editor can't just go around enforcing "As of" wording. Flyer22 Reborn (talk) 23:15, 25 January 2019 (UTC)

I do see that WP:ASOF currently states in its introduction that "Every individual, non-contiguous statement in an article that may date quickly should be so tagged." I'll see about changing "should" to "may be." The guideline does have a section below the introduction on usage. Flyer22 Reborn (talk) 23:20, 25 January 2019 (UTC)

I hate the "As Of" template because it treats Wikipedia like a newspaper, television, web site, or any other commercial enterprise preoccupied with the latest, what's currently popular, the newest new thing—terms usually not associated with encyclopedias. It encourages IP users, who don't know the rules and don't want to learn, to treat it exactly that way. That encourages, among other things, a graffiti mentality, vandalism, the addition of useless material to articles. It piles work onto editors who already have plenty to do. I'm not a reporter.
Vmavanti (talk) 21:32, 26 January 2019 (UTC)

In any case, this is the talk page of an essay that had little participation. It doesn't hold any weight. If there is an IP going around using this as their justification, they need to be instructed/disciplined like we would any problematic editor. I won't weigh in on the "As of" template here, but if an editor uses that guideline as their justification, well then that definitely carries more weight than anything being discussed here. This is simply the wrong venue. --GoneIn60 (talk) 04:29, 28 January 2019 (UTC)

The discussion above is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

Audience reviews

The article talks about critics reviews, but I don't see anything stated clearly about mentioning audience reviews in reception section. "reception" or "reviews" itself does not suggest it should be limited to critics only. Describing movie reception based only on an opinion of 20 professional critics, comparing to thousands made by audience, seems a bit biased (and biased works in both directions). At least rotten tomatoes include audience score just next to critics score to give some balance on that. And this becomes much more notable in cases where critics opinion is much different to audience opinion (one of the latest examples are The Orville [65/94] and STD [82/48]). Such fact itself provides additional information about what the reception of that media was amongst different audiences (critics are also audience).

So I suggest allowing mentions of audience scores when those are notably or significantly different from critics scores. Often these will also serve as a confirmation of statements like "later the movie received cult status" which are not rare, but are quite hard to confirm or verify on their own.

109.172.70.126 (talk) vladimir

As seen at MOS:Film, we allow audience reception. Sometimes this will be in the "Critical reception" section if the material is not so big that it requires a separate "Audience response" section. Flyer22 Reborn (talk) 05:41, 15 May 2019 (UTC)

But, as also noted MOS:Film, we typically don't include the "user ratings submitted to websites such as the Internet Movie Database or Rotten Tomatoes" because they are "vulnerable to vote stacking and demographic skew." We have included them in the case of Star Wars: The Last Jedi, but that's because the audience scores received so much attention in reliable sources. Flyer22 Reborn (talk) 05:46, 15 May 2019 (UTC)

We do already allow "audience [reception] notably or significantly different from critic scores", where the words notably or significantly is or should be taken to mean "the audience reception was commented on by reliable sources". --Izno (talk) 12:30, 15 May 2019 (UTC)

I was going to elaborate on what Izno said, instead I'll just point you to the existing rules and reasons that might help make it clearer. In short we do not allow WP:UGC User Generated Content particularly IMDB votes, or any other user voted web polls which are too easily rigged, and they are not reliable sources WP:RS. -- 109.79.86.175 (talk) 06:46, 21 June 2019 (UTC)

Tense

The examples given here are written in differing tense: "Rotten Tomatoes gives the film a score...." "At Metacritic...the film received an average score..."

IMO, the past tense wording is more accurate. RT gave a score, an editor added it to the Wikipedia article and that's what the score was. It may or may not still be that score. In truth, the numbers do change.

In the particular instance being discussed, our article said "On Rotten Tomatoes, the series has an 82% rating with an average score of 7.69 out of 10 based on 62 reviews."[2] However, at that point the series did not have these scores. Yes, at some point in the past it likely did. So, I updated the scores, reporting what scores the show had.[3]

Yes, the score difference here -- for a popular and current show -- was rather small and likely off by a sew days at most. In other instances, I have seen films where our article reported that a film "holds" or "has" a score that was several years old.

As I see it, we have three basic options:

1) Write in present tense and include an access date in the cite, depending on readers to check the cite to see if the scores are out-of-date.

2) Write in past tense and include an access date in the cite. Another editor objects to this saying it seems to imply the scores are wrong.

3) Write in past tense, include an access date in the cite and refer to the the date inline: "As of April 2020, Rotten Tomatoes gave the film..." At the article where this is being discussed, another editor rejected this as inconsistent with other articles and unnecessary, given the access date in the cite.

The relevant standard would seem to be MOS:DATED: "...terms such as now, currently, to date, so far, soon, and recently should usually be avoided in favor of phrases such as during the 1990s, since 2010, and in August 1969."

Other editors already involved in this question -- in one way or another -- include YoungForever, Bignole and Drovethrughosts. If I have missed anyone, please ping them here. If I have misinterpreted your opinon, please clarify -- I do hate speaking for others. - SummerPhD^v2.0 00:51, 11 April 2020 (UTC)

The film guidelines were updated (maybe a year ago, maybe less) to recommend that editors use "As of" to make the dates explicitly clear in the text. I wasn't convinced and I don't see many people following that recommendation. Specifying "As of" still seems like overkill. Less is more, as a reader I just want to know the scores (sometimes might I want a bit more, such as how many reviews, because 100% based on 5 reviews is not particularly convincing) and as an editor the access-date is enough for me to know the score isn't too badly out of date. But there seemed to be enough consensus to get the guidelines changed anyway, even if people aren't actively trying to enforce it.

I take your point about past tense, "gave" a score. MOS:TENSE Wikipedia often uses present continuous, which in some ways is simpler, at least that is what I think people had in mind when using the phrasing "gives", or "holds" or "has". Editors have long had very different ideas about how much detail to include (some very verbose), and how to phrase the scores from RT and MC, but there were always lots of inconsistent wordings that no one could seem to agree on. It doesn't seem like the advice in this essay was ever followed by all that many, updating it isn't likely to have a huge impact either, but after all this time it would be fair to try and update the example text in this article to recommend a different wording. The difficult part would be to get a good idea of what best represents the defacto standard that people actually use, or get some consensus that the most active editors could agree to use in future. It seems like it will be difficult to please everyone. Greater consistency would be good but past efforts at consensus didn't seem to get much buy in from the Television project, or others, and many active editors simply kept doing their own thing. -- 109.76.217.83 (talk) 07:04, 16 April 2020 (UTC)

It was discussed and reiterated again recently that editors should be using "AS OF" but MOS:FILM has since 2017 included the example text [4]

As of May 2015, 50% of the 68 reviews compiled by Rotten Tomatoes are positive, and have an average score of 5.2 out of 10.

... although i don't think I've seen that specific phrasing "compiled" actually used in real articles. -- 109.76.217.83 (talk) 07:10, 16 April 2020 (UTC)

There are so many different potential wordings, it would be a start to even get a consensus exclude a few things. Editors should avoid using site specific jargon such as fresh/rotten, Tomatoemeter, and Metascore (although this may already be covered by existing guidelines, it seems obvious that an encyclopedia should avoid unnecessary industry specific jargon), I hope at a minimum we could agree on that. Furthermore, Rotten Tomatoes and Metacritic do a certain amount of interpretation when they consider if a review is positive or negative, and when they convert that into a number, so I think it is better to avoid words such as "reports" or "compiles" that imply a false sense of objectivity, they don't report the score like in a football game. Again a though, it seems like it would be very difficult to come up with a consensus wording that people would agree to use (and updating older articles to use any revised wording will be a mammoth task). -- 109.76.216.10 (talk) 15:16, 28 April 2020 (UTC)

Also see above "Metacritic percent" and also MOS:PERCENT. Both Metacritic and Rotten Tomatoes scores are out of one hundred, which is literally what percent means! Editors have for "reasons" mostly chosen to write Rotten Tomatoes using % the percentage symbol and the Metacritic score as "out of 100". Even if you could get the film project that both RT and MC are in fact percentages, you will still get a few people looking at the general guideline MOS:PERCENT and insisting that "percent" should be fully written out as a word. So much inconsistency, so difficult to gain a consensus, let alone get people to follow it. -- 109.78.197.202 (talk) 23:18, 8 May 2020 (UTC)

Rotten Tomatoes template

I've started a discussion about adjusting the Rotten Tomatoes template here: Template talk:Rotten Tomatoes#Specifying reviews page. Editors are invited to comment. Erik (talk | contrib) ^{(ping me)} 16:38, 4 February 2021 (UTC)

Specific wording

It should be made clear that this is an essay and that there is no requirement to use the specific wording of the scores from the review aggregators as covered in the essays. Nor should it be considered the "ideal" wording. Erik (talk | contrib) ^{(ping me)} 19:25, 7 March 2021 (UTC)