Wikipedia:Tertiary-source fallacy

This is an essay on WP:No original research, and WP:Identifying reliable sources.

It contains the advice or opinions of one or more Wikipedia contributors. This page is not an encyclopedia article, nor is it one of Wikipedia's policies or guidelines, as it has not been thoroughly vetted by the community. Some essays represent widespread norms; others only represent minority viewpoints.

Shortcut

WP:TSF

The tertiary-source fallacy (TSF), dictionary fallacy, encyclopedia fallacy, or style-guide fallacy is the idea that tertiary sources—such as encyclopedias, dictionaries, and sometimes review articles—are more reliable than primary and secondary sources, and therefore that they trump other arguments and evidence. Often this is paired with an argument that a given tertiary source provides the only correct version or interpretation of the facts. However, tertiary sources are of highly variable quality, and are not uniformly "better" or "more reliable" than primary or secondary sources. There are important reasons for this.

It is not fallacious to cite a tertiary source in an article or offer it as evidence in a discussion. But advancing its view as if it ended the discussion, as if other facts and reason cannot surmount this source, is fallacious. Specifically, it is a form of the argument to authority fallacy.

Dictionaries

Modern dictionaries are primarily descriptivist works, not prescriptivist ones like those of the 19th century and earlier. They do not create spelling, capitalization, or meaning, as if written by the gods and handed to us as holy truths. Rather, they observe and record usage – ever-shifting – in reputable publications. They do this in piecemeal fashion, very slowly, and in an under-staffed manner. Like most tertiary sources, some of what they contain is incomplete, a little of it is mistaken, and much more of it is obsolete by the time it sees publication.

The fact that a dictionary prefers one spelling over another doesn't mean that one spelling is preferable. It indicates nothing but the preferences of one publication's editors. A dictionary that provides one particular spelling or capitalization but omits another one that is nevertheless well attested in high-quality works elsewhere cannot magically make the other variation go away. It's simply an incomplete dictionary. A dictionary's general purpose is providing simplified, "as concise as possible" definitions of how a term is used in everyday English. A dictionary cannot be used to prove that a term's narrow meaning in a specific field doesn't exist or isn't what it is, just because the dictionary doesn't contain it or defines it differently in the context of the average person's use of the language.

One even has to know the biases of the publisher. For example, the American Heritage Dictionary was created as a neo-prescriptivism work as a direct negative reaction to Webster's Third New International Dictionary, the most linguistically descriptive dictionary at the time. In short, traditionalists got very angry that it included things like ain't as "real words", and set out to create their own anti-Webster's to reject acceptance of non-mainstream American English. And it remained in that kind of mode for several decades. AHD, under better editorship two generations later, is a more respected work today. But this serves to illustrate that "published in a dictionary" doesn't really mean much, and that an old version of a tertiary work is effectively a primary source – like old news, it is too close to, too involved in, what it was purporting to neutrally record. Notably, AHD still isn't actually neutral, infusing its description with prescriptivism: it conducts an annual "Usage Panel" poll, of hand-selected American editors, authors, journalists, English professors, and other such persons on hundreds of usage questions, and uses the results of this – a set of highly entrenched prescriptions – to decide how to write the dictionary's notes on what is and isn't proper usage. It's unclear how other dictionaries are even arriving at their determinations, but it's probably a similar processes. Style manuals are even more iffy, often editorially dominated by a single person.

Style manuals, including usage dictionaries

These are much less reliable than general-audience dictionaries, and are in fact opinion pieces. They are primary sources that represent the opinion of their organizational publisher or sometimes just their individual author. Style guides are not written by dispassionate parties but quite impassioned ones, usually for a specific micro-market (a particular news agency, book publisher, journal, professional association, or government agency/ministry) with little independent editorial oversight. Such works have an explicit agenda to set "rules" – a prescriptive and subjective exercise. Many of them have a palpable nationalistic bent; exaggerating and even inventing differences between American, Canadian, British, or Australian usage helps sell the books and their successive editions, and to reinforce what the work advises as a kind of minor patriotic duty. (This idea dates back to Noah Webster's dictionary of 1828, which standardized modern American English spelling, sometimes with the explicit goal of differentiating American from British spelling.)

Style guides are not written by general linguistic authorities, but by newspaper editors, journalism professors, English teachers, law dictionary editors, processors of university theses and dissertations, and other specialists from narrow fields, though some are written by dictionary editors who have linguistics training. They come from a professional background of sharply limited approach to the language, and of (most often – dictionarians excepted) denialism of variation in favor of insistence on a particular ruleset – on pain of rejection of one's submitted work. Most of this is nothing at all like an encyclopedic approach to language and its usage, but is a throwback to the earliest notions of prescriptive lexicography and grammar. A few specific individuals have a strong personal effect on a whole range of such publications. For example, most of The Chicago Manual of Style's grammar and vocabulary material, Black's Law Dictionary, Garner's Modern English Usage, The Oxford Dictionary of American Usage and Style, Garner's Dictionary of Legal Usage, The Chicago Guide to Grammar, Usage, and Punctuation; The Elements of Legal Style, The Redbook: A Manual on Legal Style, and half a dozen other style guides are all by or principally by the same person, Bryan A. Garner (a law teacher). Less discouragingly, though no less narrowly, New Hart's Rules and Fowler's Dictionary of Modern English Usage, along with New Oxford Dictionary for Writers and Editors and New Oxford Dictionary for Scientific Writers and Editors, have for successive editions been the work of editors or chief editors of The Oxford English Dictionary. At any give time, a tiny handful of individuals and two publishers totally control the majority of mainstream English-language style manuals, and they do so on sharply divided but artificial "British versus American English" lines. Oxford University Press in particular profits from this both ways, since they get to sell competing sets of US and UK stylebooks on a nationalized basis, plus sell "serious writers" both collections of such books.

At a fundamental level, style guides lack independence from the source material, and are instead deeply bound up in controlling, shaping, and prescriptively attempting to define English usage, rather than dispassionately describing it.

Encyclopedias

Encyclopedic works suffer similar limitations, and more besides. They entail a great deal more subjective judgement in their assembly, as to what they include, what they omit, and how they interpret and present what the sources are telling their authors. Most challenges we face as encyclopedists at Wikipedia are also faced by those at Britannica and other encyclopedia publishers, but with a much smaller community of support and a much less public system of checks and balances.

Some other encyclopedia cannot be used to prove that Wikipedia is wrong when we draw on reliable, current secondary sources. At most, it tells us that editors of another work (at some probably indeterminate point) assessed different sources and came to a different conclusion – that we may need to examine more sources and the quality of those we've already found.

Topical encyclopedias

Virtually any subject of note has at least one book (or, today, online database) about it claiming to be an encyclopedia, though many of them are actually jargon usage dictionaries. Even among those that really are encyclopedias, their quality varies wildly. On pop-culture topics, they are generally written by amateurs – fans – who have no credentials to speak of. (This does not mean they're necessarily completely unreliable. Someone obsessed with Star Trek for 35 years may in fact actually be the world's foremost authority on the franchise. But we have to research the reputability of the publication and author. The fact that it was published and has "Encyclopedia" in its title means nothing.)

Among alleged encyclopedias for various technical and scientific fields, they range from unreliable wikis to single-author works that robotically summarize terminology in a one-off volume that never sees an update, to in-depth, multi-author ongoing projects with an eminent editorial board, like Encyclopaedia Iranica. They must be assessed on a case-by-case basis, as to the nature, depth, reputability, and currency of the work. Even when found to be reliable, they are just one source, and still just a tertiary one.

Topical encyclopedias can take various special forms, such as sectionalized histories of particular fields, biographical "dictionaries", geographical gazetteers, historical timelines, and others. The layout doesn't matter; we care about the quality and kind of research and sources that produced it, and the reputation of the authors(s) and publisher – and especially of the work itself within the field to which it pertains.

As with a general dictionary, no topical usage dictionary for any field, providing an over-simplified gloss, can be used to disprove better, more in-depth secondary sources from a particular discipline that provide a more specific definition, a newer one, or an additional one. (However, a current, high-quality tertiary source of this sort could be more reliable on a particular point, especially if it cites recent peer-reviewed material, than a contrary but secondary source that is considerably older.)

When and how tertiary sources are useful

Tertiary sources like dictionaries, encyclopedias, and style guides are only of much use in helping settle Wikipedia content and presentation disputes when all of the major ones (for the general public and/or for a particular topic) are consulted and their aggregate view is examined and used.

If almost all dictionaries prefer the spellings pedology or paedology (for the study of children), usually clearly identifying the former as primarily American and the latter as mostly British, while only one even suggests the spelling paidology is attested (and it doesn't include any usage notes), we can then be quite confident in what information we should present. If we consulted nothing but that last dictionary, we might come up with (and publish) the incorrect idea that all three spellings are well-attested and interchangeable.

If 90% of encyclopedias, biographical dictionaries, histories of science and philosophy, and similar works give a historical figure's birthdate as c. 52 BC and only a handful vary from this (e.g., with 52 BC, c. 52–51 BC, 52–51 BC, 51 BC, or c. 51 BC), we can be confident that we're in the clear to use "c. 52 BC" and perhaps relegate any doubt about this to a footnote. If only about half of them are this certain, we can instead firmly resolve to hedge with "c. 52–51 BC". If, however, we only consulted one such work and it said "51 BC", we would be on very shaky footing using that value, and may well be perpetuating a claim that most scholars have rejected.

The tertiary-source fallacy can be disruptive

It's clearly disruptive editing and gaming the system to willfully engage in the fallacy that the tertiary source you like overrides other evidence, to push a viewpoint in a content dispute. Yet this behavior can be observed on Wikipedia quite frequently. Call it out as unreasonable when you encounter it.

If someone has pointed you to this page and claimed you are engaging in this fallacy, ask yourself some questions: Are you are presenting a particular source's take on the subject because it agrees with your beliefs and preferences on the matter? Are you asserting that work's view in the face of contradictory evidence from other sources, especially secondary ones? Are you misusing a generalist source to reject a more pertinent specialized definition/interpretation? (Conversely, are you trying to rely on a narrowly specialized or biased and prescriptive source that is not appropriate for a more general context, a broader usage, or more descriptive material?) Are you ignoring others' reasoned arguments in a "bible-thumping" manner because you've found a book that says something different from what all their sources and policy arguments conclude? Are you trying to come to any kind of analytic, evaluative, interpretative, or synthetic conclusion based on your tertiary source?

The TSF can be unhelpful to consensus formation even when used innocently. Another question to ask yourself: Have you taken the time to examine numerous such works to see whether a general consensus emerges from them as a group? If you have not done this kind of homework, but are presenting the one source you found as the truth rather than as just one source to consider among others that need to be identified and examined, then you are making a mistake.