User:Cscott/Ideas/Some thoughts on Global Templates, so called

Lots of folks are talking about "Global Templates" these days, and the need for the Wikimedia movement to better support them. Let's start by pointing out the ambiguity embedded in the name of this discussion, to wit: both "Global" and "Template" are used to mean some rather different things by different parties.

Let's list a number of ways that "Global" is used, to start:

• Global: on a database/infrastructure level, there should be a way for an invocation to include a "template" from a different wikiproject. This is referred to as "scary transclusion" in the MediaWiki source code, but we'll just call it "cross-wiki transclusion" here.
• Global: the languages used to write "templates" should be language-independent/allow localization of (some combination of, depending on the speaker) code/documentation/module names/parameter names/strings embedded in modules/test cases. We'll call all of this "localization" here, but it covers a wide arc.
• Global: there should be a community of template authors that includes folks on different wikis.

Note that these three definitions are all orthogonal! We can have cross-wiki transclusion without solving any localization issues or building a nonlocal community; we can internationalize the languages used to write templates/modules without doing anything about cross-wiki transclusion or community; and we can build a global community of template authors (say, on commons, supported by annual conferences and similar) while not doing anything about localization of the templates themselves or about cross-wiki transclusion. Thinking positively, working on these aspects independently is still useful in isolation—for example, if you "just" localize the implementation language then you can use localized templates on wikiprojects which are already global, like commons or the annual Wikimania sites. The "global" efforts can reinforce each other—and it would be wise to gently harmonize them—but we don't need to wait until we can do everything at once to get started.

Turning to "Templates":

• There at at least four (!) different template implementations used in core; in rough order of "computational power"/complexity: "regular wikitext" transclusions, ParserFunctions, Wikifunctions, Scribunto modules (perhaps you can quibble over the ordering of the final two). Putting aside Wikifunctions, Scribunto is probably the only solution which in principle has the power to replace all the others, but (a) this has never been actually attempted, and (b) Scribunto hasn't been meaningfully updated since Lua 5.1 (first released in 2006, end of life in Feb 2012). We're going to use "templates" in the rest of this document for clarity, but you can mentally substitute "transclusions/parser functions/wikifunctions/modules" every time it appears.
• There are many different uses for templates, and different folks have different ideas about which "template" use cases should be "global":
  • "Simple computation", like a template to convert miles to kilometers, or compute the current age of a person from their birth year.
  • "Wikidata processing": this ranges from "fetch the current population of the city from Wikidata" to "generate the infobox for this city entirely from Wikidata properties" to "content generation" (next!).
  • "Content generation": generate an entire article/paragraph/navbox using Natural Language Processing (NLP) tools.
  • "Syntax workarounds": ranging from common escaping mechanisms like {{!}} to workarounds used to make wikitext syntax work better with RTL handling of punctuation; this (surprisingly?) interacts with and constrains uses of template-like mechanisms; for example editors would generally like to have citations mark specifically the phrase or range of article text which is supported by the citation, but the wikitext syntax most-cleanly supports only an "at a point" reference mark. Changing the syntax mechanisms for "templates" (heredocs are one attempt at this) in turn enables/facilitates different sorts of uses of templates.
  • "Macros": simplifying common texts or rote formatting, overlapping slightly with...
  • "Style consistency": making article formatting or thumbnail formatting or citation formatting (etc) consistent across a wiki by using a template for every use. The web team has recently begun using Extension:WikimediaMessages as a "poor man's Global Templates" for this purpose.
  • Navbox/infobox: overlaps with many of the other cases, and perhaps involves tooling of its own. Infoboxes/navboxes are really a form of knowledge categorization, and involve inheritance and other mechanisms to structure the information about a topic.
  • Page layout: Again, there is some overlap with "macros" and "style consistency", but "templates" are the only mechanism currently available for "combining disparate pieces of content into a single page" (more). Pull quotes, asides, figures, and tables have a relationship to the main content but in some sense stand outside it (not to mention navbox/infobox, already discussed, and annotations (next!) like lead image, page summary, and the lead paragraph).
  • Semantic tagging/annotations: Template syntax and names can be used to add semantic information to a page. The "lead image" can be marked with a template; extracting the "navbox" and "infobox" from the page content is aided by knowing the names used for these templates on the page; the "citation" templates structure information in a more meaningful way than raw <ref> tags, etc. Many of the workflow templates (next!) could be considered part of this category as well.
  • Work flow: Wikimedia is not just an "encyclopedia/dictionary/source" project, it is also the means by which the community organizes itself to do this work, and templates are an important part of the mechanisms used for that work. Everything from {{citation-needed}} to voting on administrators is organized with templates and these workflows are also "global": to some degree they are common to all of our projects and in other ways communities differ (and need to differ) in the ways they organize themselves.
• More broadly, "templates" are used to describe the extension mechanism: the essential relief valve which lets the community experiment, develop, and at times outrun the foundation. In practice this runs the gamut from Gadgets (implemented in client-side JavaScript), Modules (server-side Lua), Bots (a variety of platforms, but server-side Python using Pywikibot/mwparserfromhell is very popular), to Extensions (server-side PHP). In all of these "deployed on WMF servers"/"maintained by the foundation"/"integrated into mediawiki-core" is the eventual goal for "experiments which succeed". The reader will surely notice that the implementation languages vary for each of these extension mechanisms, and unfortunately the APIs to common mediawiki tasks vary greatly as well. There is no inherent consistency between the mechanisms used to (say) "get the article title" between the gadget, module, bot, or extension APIs, nor in the object type such a request would return (text string, dbkey string, or objects of various kinds). Change management and maintenance are key concerns, handled to some degree by modern version control and code review systems -- but these are not available to gadgets and modules, and probably not consistently handled for bots either. These concerns are exacerbated in the "global" context -- how do I know as an editor on (say) English Wikipedia that the "template" edit I'm about to make isn't going to break something on Spanish Wiktionary? And how does the reader of Spanish Wiktionary know who to contact when they see their content is suddenly broken? And, in "solving" this problem, how do we maintain the essential "ability to experiment with low overhead and without staff involvement" which is the whole point of an "extension mechanism"? We need to preserve the ability for local change/innovation/adaptation even as we provide new opportunities for "global" coherence.
• Finally, taking the translator perspective, as Amir does: take the simple task of translating content which appears on wiki X to wiki Y. Translating the non-template content is straightforward -- but what do we do, in the translation context, with the "templates"? This implicates all the different use cases for "template" above, and perhaps requires various solutions for different use cases, but fundamentally this is a "knowledge translation" and "workflow" question; not just "is there an equivalent for this template" or "can I make the new article visually display the same as the old article" but "how do we manage cross-language information (as expressed incidentally via the means of templates)"? In order to translate {{citation-needed}} we need to know how the workflow on that wiki is organized, and how do we represent that? What about templates that create categories or taxonomies, and what if my local wiki/culture has a different taxonomy? How do I even start to categorize various templates according to the list of use types above, where does that data live?

As with "global", we can solve various part of global "templates" in isolation, but it is probably very important for speakers to be clear about exactly which meaning(s) of "template" they mean, because it is pretty unlikely that any work is going to apply to all "template" systems and all meanings of/use cases for "templates".

Types

When programmers, the parsing team, the Visual Editor team, linguists, and Wikidata folks are brought together by this discussion, they will all talk about "types". But again, caution is warranted, because these four communities mean different things by this term:

• For programmers, "type" usually means a basic computer data type, like integer or floating point number, boolean, character or string; and then complex data types usually formed as compounds or arrays of such types, and often named or abstracted. "A user object contains a username of type string, a password stored as a binary hash, the date the account was created stored as an ISO-8601 date string..."
• For the parsing team (and some template editors!), types are the "shape" of the thing generated by the template or provided as arguments. The basic types currently in MediaWiki (values output from templates, parser functions, and extensions) are "raw string", "wikitext string", and "html string" (not always well distinguished in practice), with incomplete support for complex objects encoded as a "JSON string". The content transform team would like to clarify and optimize these types and more clearly describe how they are composed; adding things like "block" and "inline" contexts to the type system, borrowing from the HTML tree building specifications in order to better define the composition of such fragments into an eventual HTML document. Ideally the "types" of arguments to transclusions should match up with the types of transclusion output (with implicit conversions between types, perhaps) to allow transclusions to be composed.
• For the Visual Editor team, types are "the sort of UX widget needed to represent this value". The TemplateData specification lists some of these. Distinctions are made between "string" (multiline string) and "line" (short text field) as well as distinguishing things like "wiki-page-name" and "wiki-user-name" which can have relatively complex UX affordances, despite all of these being "just (raw) strings" from the parsing or programmer perspective.
• For linguists, types in pure lambda calculus are are a way to encode grammar and construct language. This can be somewhat misleading, because these types "look like" types that a programmer in a functional programming language might recognize, but the values are all "meaning units". Here are some type constructor functions from Grammatical Framework and note that we have grammar "types" like "number less than a thousand". These are very relevant for generating grammatical sentences, and in some sense knowing the grammatical gender and number of (say) an infobox value may be helpful in ensuring agreement with a corresponding label for that value, but grammatical categories like "non-inflecting particle/phrasal verb" are generally not what non-linguists think about when filling in a template argument or editing an article.
• Wikidata and the semantic web have an ontology with allows arbitrary and flexible relationships between objects via properties, which can be thought of as types or inheritance. The White House is an instance of mansion, civic building, tourist attraction, and official residence, but it is also part of the White House Complex, inspired by Leinster House, owned by the federal goverment, located on street Pennsylvania Avenue, etc. One can imagine writing a "type" system as a SPARQL query: allow only values here which are "instance of civic building".

We should be careful not to assume our definition of "type" is universal, nor that our notions of what "type" distinctions are useful map cleanly to the sorts of types useful to a different community or use case. If the argument to a template is "a number", then the phrase "about a million" (a string not a number to a programmer type) might well be a valid input, depending on your perspective.

Theory of change

Embedded in many discussions is also an accompanying "theory of change": assuming we want "global templates" (using the speaker's own preferred definition) and that we don't currently have "global templates", how do we get there from here? There seem to be three main theories of change, which in turn influence a speaker's perspective on their idea of "global templates" and its feasibility:

• The innovator. Build a new thing from scratch, and because it is the best thing, users will chose to use it; alternatively, "because we don't know what the right thing to build is, we have to start from a clean slate". Sometimes this is explicitly a "cathedral" model of development, sometimes the rationale is feasibility aka "minimum viable product" -- if we try to build something which solves everyone's problems at once and tie our hands with backwards-compatibility, we may never finish. "With our limited resources we can only afford to build the new thing." For the innovator, persuasion is appropriate, but adopting the new thing needn't be "easy" because the obvious technical superiority of the new thing will cause adopters to flock to it despite any difficulties.
• The moderate. "If you build it they will come." The Foundation's responsibility is to build the new thing, and the community will vote with their feet to adopt it, or not. Explicit foundation encouragement to replace the old thing with the new would be risky, may backfire, and isn't resourced anyway. "We don't know what the community wants." The innovator is sure that the new thing will persuade users, the moderate is going to wait and see, and doesn't think that persuasion is appropriate: adoption decides merit, so if you lobby for adoption you are obscuring the merit signal. The merit of the thing is fundamentally unknowable a priori, but the process works if you keep rolling the dice until you get a hit.
• The incrementalist. "The only way to build a new thing is from an existing old thing." Backwards-compatibility is the only metric for success, and ideas which require a break from the past are infeasible. Unlike the innovator, the incrementalist prefers to pave the cow paths; persuasion isn't necessary because the incrementalist is following, not leading. Adoption must be easy otherwise there is no merit.
• The politician. Thinks in terms of carrots and sticks and believes technical merit is subordinate to power/community dynamics. In the best case, wields mostly carrots; in the worst, mostly sticks: "well, we're turning XYZ off at the end of this month, so they'll either convert or their things will break" or (more implicitly) "we can't afford to maintain the old thing any more".

These are stated in a rather extreme manner, of course; any actual person will have more nuance to their views. However, locating proposals among these axes can be useful for helping determine how success is to be measured and what resources are required "after the project is done" ie, deployed and usable. The theory of change often frames the question: are we trying to build the best new thing possible, any thing (and we'll see if its a good thing later, and if not iterate), the best low-cost improvement, the thing the community wants most, or something decided via another (good or bad) process which the community ultimately will have to use? Will we only succeed if enough people use it? If everyone uses it?

Hopefully this document as it stands can help support clear communication and help folks clarify what, exactly, they are talking about when the discussion turns to "global templates". In another document I will try to lay out some of my own thoughts about possible solutions and my own theory of change.