Template:Database report/doc

This template summons a bot to update the page with the results of the provided database query. It must be "closed" with {{Database report end}}; the text between these two templates will be replaced by the bot-generated table. It is possible to have multiple reports in one page.

The table is updated by the bot whenever you click the "Update the table now" button, and can be configured to automatically update periodically by setting |interval=N (one update every N days).

Some formatting options are provided which enable page titles to be wikilinked and article excerpts to be shown. For complex use cases, the row_template or postprocess_js options can be used which allow you to format the query output in arbitrary ways using wikimarkup, Lua or JavaScript.

This template is currently powered by SDZeroBot (source code). It populates Category:SDZeroBot database report subscriptions.

---

Don't know SQL? No worries. SDZeroBot's AutoSQL will help you get the SQL equivalent of a query written in plain English. It doesn't work all the time, sadly – in which case you can ask for help at WP:RAQ.

Example

{{Database report
|sql         = 

    SELECT page_title, rev_timestamp AS "Last revid", page_latest, page_len, actor_name AS Creator, user_editcount

    FROM pagetriage_page

    JOIN page on page_id = ptrp_page_id

    JOIN revision ON page_id = rev_page AND rev_parent_id = 0

    JOIN actor ON rev_actor = actor_id

    LEFT JOIN user ON user_id = actor_user

    WHERE page_namespace = 0

    AND page_is_redirect = 0

    AND ptrp_reviewed = 0
|wikilinks   = 1, 5:2 <!-- Link 1st column to mainspace, 5th column to user namespace (ns 2) -->
|excerpts    = 
|comments    = 
|widths      = 2:9em, 3:15em <!-- Set 2nd column width as 9em, 3rd column width to 15em -->
|interval    = 7 <!-- Update once every 7 days -->
|table_style = 
|pagination  = 
|max_pages   = 
}}
... (This will be overwritten by the bot) ...
{{Database report end}}

Parameters

sql

The SQL query used to generate the report. Consider using Quarry to test your SQL first. For queries with large outputs, consider using a LIMIT clause to limit the output size, or use pagination. IMPORTANT: The query must execute within 10 minutes. The bot uses a server-side timeout – MariaDB's max_statement_time variable to prevent overload on the DB. This should only contain 1 SQL statement. Do not combine multiple statements with semicolons.

If the SQL needs to contain vertical pipes (|) or other characters that may get parsed as wikitext (like {{ and }}), wrap the SQL with <nowiki>...</nowiki>.

This is the only required parameter.

Database table references: page · revision · text · actor · user · main database layout manual page

wikilinks

Wikilink page titles. The syntax is best explained with some examples:

• wikilinks = 4 Links titles in the 4th column. Use this style if the column contains the full page name.
• wikilinks = 4:2 Links titles in 4th column to user (ns:2) namespace. Use this style if the column contains the page name without namespace.
  • See Template:Namespaces for namespace numbers. Common ones are: 0 (article), 2 (user), 4 (Wikipedia), 6 (file), 10 (template), 14 (category), 118 (draft). For respective talk namespaces, add 1 to those numbers.
• wikilinks = 5:2:show Links titles in 4th column to user (ns:2) namespace, and additionally adds the namespace prefix to the displayed text.
• wikilinks = 4:c3 Links the titles in 4th column using namespace number given in the third column. Applicable for queries like SELECT page_namespace, page_title from ... that have both page title and namespace but in different columns.
• wikilinks = 4:c3:show Same as above, but additionally put the namespace prefix in the displayed text.
• wikilinks = 2:2:show, 3:0 Links multiple columns! The configuration of each column (using one of the above styles) is comma-separated.
• wikilinks = 1:0, 2:2:show, 3:4 Adds links to three different columns: 1st column to mainspace, 2nd column to userspace making the "User:" prefix visible, and titles in 3rd column to WP space (with the namespace prefix piped).

comments

Comma-separated list of column numbers that contain edit summary comments or log action comments. Specifying this makes the bot escape the text so that edit summaries like "added {{infobox person}}" or "removed [[File:Example.jpg]]" are properly displayed without the template being transluded or images showing up on the report page.

widths

Explicitly specify the column widths. Eg. widths = 2:10em, 4:20px forces the 2nd column to have a width of 10em and the 4th column to get a width of 20px.

table_style

The style attribute applied to the table element. By default this is overflow-wrap: anywhere which causes better overflow behaviour. To suppress the default, use overflow-wrap: normal.

table_class

The class attribute applied to the table element. By default this is wikitable sortable.

excerpts

Show an excerpt of articles. Excerpt config takes the form:

  srcColumnNumber : destColumnNumber : namespace : charLimit : charHardLimit

• Apart from srcColumnNumber everything is optional. However, optional parameters can only be used if the ones before were used, that is: to mention namespace, you also need to mention destColumnNumber first.
  • srcColumnNumber: (required) The column number containing page names whose excerpts are to be shown
  • destColumnNumber: The column number in which to place the excerpt. The original contents of that column will be shifted rightwards. If unspecified this takes the value (srcColumnNumber + 1)
  • namespace: The namespace number to use for page names in srcColumnNumber. If unspecified, it's taken as 0 – which means that the page name in srcColumnNumber is treated as the full page name (appropriate for mainspace). Use c4 to get the namespace number from 4th column. This can be used as input to {{Namespace name}}, to return the actual name: "{{Namespace name|1}}:" will return "Talk:" and "{{Namespace name|10}}:" will return "Template:".
  • charLimit: A soft limit for size of excerpt by number of characters. This is a soft limit, the excerpt can be longer to avoid cutting mid-sentence. (Default: 250)
  • charHardLimit: A hard char limit for size of excerpt. Sentences may be cut in the middle (but in that case will end in ellipsis). (Default: 500)
• Most of the time, excerpt columns will mangle the layout of the table. Always use the widths parameter to tune the column widths.
• Since excerpts add columns to the table, note that the column numbers used by all other parameters (widths, wikilinks, comments, remove_underscores) take into account the added columns.
• Excerpts are only intended for articles and drafts. The Node.js code used for generating them is here – only the initial few lines of the first paragraph are kept with references, markup, images, infoboxes etc are removed.
• Excerpts for multiple columns are supported – in which case this parameter should be comma-separated list of excerpt configs.

remove_underscores

A comma-separated list of column numbers in which underscores are to be replaced with spaces. This is intended for columns containing page titles, since the database stores titles with underscores (_) in place of spaces. You can prettify it a bit by using this parameter. This is automatically done for columns with wikilinks or excerpts.

interval

(No default) Numerical value. Number of days to wait between automatic updates. Minimum allowed value: 1. If unspecified, no periodic updates are done.

pagination

(No default) Number of results to include in a page. Further results are saved to paginated subpages. For example, if |pagination=1000 is set and there are 4200 results, 1–1000 are saved to the page containing the query. 1001–2000 are saved to the /2 subpage, and so on. If unspecified, all results are saved on a single page. See also max_pages.

Note: Pagination cannot used when there are multiple reports on a page.

max_pages

(Default: 5). For use with pagination only. Max number of report pages to create. This can be set upto 20.

hide

Hide a column. Normally columns can be hidden by just not including it in the SELECT statement. However, some columns like namespace number are necessary for generation of wikilinks but undesirable in final output. Specify the column number(s) of such column(s) here, comma-separated if multiple.

row_template

Generates a template call rather than a table row; the template must then generate the table row in turn or the table header/footer need to be suppressed using skip_table parameter. Values in the row are passed to the template as unnamed parameters (|1=, |2=, etc).

• To have the template substituted instead of transcluded, put "subst:" before the template name.
• To use a Lua module instead of a template, put "#invoke:" before the module name.

row_template_named_params

While using row_template, instead of using unnamed parameters, use the column names as parameters. This is a boolean field - any value will work.

skip_table

Suppresses the table markup completely when using row_template. Any value will work.

header_template

Generates a template call rather than a table header; the template must then generate the table header. No parameters are passed to this template. The template can be substituted as well - just specify "subst:" before the template name.

If this is used without using row_template as well, then the header template will be placed over the table, and table headers will still be generated.

footer_template

Generates a footer calling the given template. For use with skip_table. This could be used along with header_template for collapsing (collapse top/collapse bottom) or creating columns (div col/div col end) in the generated content. No parameters are passed to the template. It can be substituted instead by specifying "subst:" before the template name. For quick reference: |} is {{end}}.

postprocess_js

Experimental

Custom postprocessing code, in JavaScript, which is executed by the bot on the raw db output. Format:

function postprocess(rows) {
  // `rows` is an array.
  // Each element in `rows` is an object with column names as keys, and with values stringified
  // Eg. for SELECT page_namespace, page_title FROM page LIMIT 2
  // the `rows` could be:
  // [
  //    { page_namespace: '0', page_title: 'Main Page' },
  //    { page_namespace: '1', page_title: 'Main Page' },
  // ]
  // Write code to process rows here.
  // Return it after modifications:
  return rows;
}

The postprocess() function is applied on the query result before it is modified by other formatting options like wikilinks. It can be used to introduce new derived rows or columns (eg. a "Total" row at the bottom), dynamically modify rows based on content of other rows, etc. If the code contains wikimarkup-like syntax that could confuse the parser, wrap the whole code within <nowiki>...</nowiki>.

The JS code is run on Node.js v18.14.2 sandboxed to use upto 16 MB of memory and 1.5 seconds of execution time. No network or filesystem access is allowed – with one exception.

await bot.request('https://...') provides readonly access to various Wikimedia APIs. Supported endpoints: wiki.riteme.site/api/rest_v1, wikimedia.riteme.site/api/rest_v1, wiki.riteme.site/w/rest.php, en.wikipedia/org/w/api.php, api.wikimedia.org and query.wikidata.org. These can be used for fetching pageview data (from REST API), ORES scores (from Lift Wing), and so on.

For the Action API, await bot.api({action: 'query', ...}) can be alternatively used. Parameters action: 'query', format: 'json' and formatversion: 2 are set by default.

silent

Suppress visible output from this template. Only the table generated by the bot will be visible. The last update timestamp and query runtime are also not saved. This means that if the query result is unchanged, periodic updates won't reflect in the page history.

This is a boolean field - any value will work.

SQL formatting tips

• Datetime columns are stored in YYYYMMDDHHmmss format. Select them as DATE_FORMAT(rev_timestamp, '%Y-%m-%d %H:%i') or DATE_FORMAT(rev_timestamp, '%Y-%m-%d').
• For generating a serial number column, select ROW_NUMBER() OVER(ORDER BY (SELECT 1)) AS "No.". For example, see WP:Database reports/Fully salted titles with unusually long expiries.
• To use this with {{static row numbers}}, put header_template = static row numbers and table_class = wikitable sortable static-row-numbers static-row-header-text.
• Applying table_style overflow-wrap: anywhere; word-break: break-word can sometimes help avoid columns becoming too wide due to presence of long non-breaking text. This may cause even the headers to break mid-word, use widths to avoid that.
• To display full wikilinked page names where namespace can vary (as in SELECT page_namespace, page_title FROM ...), use |wikilinks=2:c1:show and then hide the namespace column from display (|hide=1).
• This template also plays well with {{Namespace name}}, which allows you to avoid extra queries: for example, "{{Namespace name|1}}:" will return "Talk:" and "{{Namespace name|2302}}:" will return "Gadget definition:".

---

The design of the bot and template were inspired from User:ListeriaBot and {{Wikidata list}}.