Wikipedia talk:Lua/Archive 10

This is an archive of past discussions on Wikipedia:Lua. Do not edit the contents of this page. If you wish to start a new discussion or revive an old one, please do so on the current talk page.

Archive 5

←

Archive 8

Reading the default collapse state

The collapsible option template has two places where it reports the default collapse state for the documented template. Currently, those defaults are set by hand and require changes anytime the default is changed. It occurred to me that it may be possible to have a module read the code and determine the default automatically. One way would be to look for {{{state|FOO}}} and report that FOO was the default. Another way would be to read the classes from {{ {{BASEPAGENAME}} }}. Note that Template:Collapsible option should never be used in articlespace and should never be used more than once per page, so we should be able to do something that is more computationally expensive. Another helpful thing would be if it could detect that there is no {{{state in the template, this would mean that Template:Collapsible option is reporting false information. Is any of this feasible? Thanks! Plastikspork _―Œ^(talk) 14:18, 18 December 2020 (UTC)

It looks like

{{#invoke:string|sub|{{ {{BASEPAGENAME}} }}|{{#expr:{{#invoke:string|find|{{ {{BASEPAGENAME}} }}|mw-collapsible}}+15}}|{{#expr:{{#invoke:string|find|{{ {{BASEPAGENAME}} }}|navbox-inner}}-2}}}}

works for many pages, but with three transclusions, so I would imagine that this could be cut down to one for detecting the default, or two for checking to see if |state= does anything. Thanks again! Plastikspork _―Œ^(talk) 14:36, 18 December 2020 (UTC)

@Plastikspork: Another approach is to rely on the parsing the wikitext of the template rather than the rendered output, producing something like

{{#invoke:String|match|{{#invoke:Page|getContent|{{FULLBASEPAGENAME}}|as=raw}}|{{(((}}state{{!}}([^{{)}}]-){{)))}}|nomatch=none}}

, which should return the default state value if the default is specified and "none" if no state parameter could be found. * Pppery * _{it has begun...} 18:29, 18 December 2020 (UTC)

@Plastikspork: Your "It looks like..." sample is trying to work on the HTML source of a page. However, Lua cannot get access to that. Your code is working on the wikitext of BASEPAGENAME. I don't think any attempt to parse the wikitext in a one-liner of template code is going to work because it's very likely that in the many places {{Collapsible option}} is used ("121052 transclusion(s)"), there will be variations in the use of whitespace and other quirks. For example, the wikitext of Template:Solar System includes state = {{{state|{{{1<includeonly>|autocollapse</includeonly>}}}}}}. I think you would need a module to read the wikitext of the template, find state if present, then make some guesses. Johnuniq (talk) 22:58, 18 December 2020 (UTC)

data calling data

el.wiktionary, Sarri calling Houston: I have a little problem. (and other bigger ones too). LuaSchool.episode2.
Suppose i am in a module which makes title-links. I am gathering data for titles like local topics_2 = { this , that } I have to write c. 200 of those (with keys and other 'fields' is the word?). They all, already exist in el:wikt:Module:labels/data as in the word_cat How can I call them? I have tried and failed with

local topics_2 = { require("Module:labels/data")['word_cat'] = true }

I know that it is terribly wrong. If it is not possible, no problem: I will rewrite them. Thank you! Sarri.greek (talk) 18:21, 15 October 2020 (UTC)

Rewrite that table so that it looks like this (note the comma at the end of each line:

 return {
	['αγγλισμός'] = { link = 'αγγλισμός', linkshow = 'αγγλισμός', word_cat = 'Αγγλισμοί', key = 'αγγλισμοι', cat_group = 'style' },
	['αεροπορικός όρος'] = { link = 'αεροπορία', linkshow = 'αεροπορικός όρος', word_cat = 'Αεροπορικοί όροι', key = 'αεροπορικοιοροι', cat_group = 'topics' },
	['αθλητισμός'] = { link = 'αθλητισμός', linkshow = 'αθλητισμός', word_cat = 'Αθλητισμός', key = 'αθλητισμοσ', cat_group = 'topics' },
	['ακουστική'] = { link = 'ακουστική', linkshow = 'ακουστική', word_cat = 'Ακουστική', key = 'ακουστικη', cat_group = 'topics' },
	-- etc
	}

Delete local label = {} and return label. Then, in (I presume) Module:labels, do this:

local topics_2 = mw.loadData ('Module:labels/data')

Then, to fetch word_cat from the αθλητισμός key/value pair:

some_variable = topics_2['αθλητισμός']['word_cat']

Because Module:labels/data is just data, it only needs to be loaded once per page; require() will reload the data for every instance of the labels module.

—Trappist the monk (talk) 19:38, 15 October 2020 (UTC)

Lifesaver @Trappist the monk:. I will work on it, and also add it at my school notes! Sarri.greek (talk) 20:35, 15 October 2020 (UTC)

The 'return' at el:wikt:Module:labels/data spoils el:wikt:Module:labels so I could not add it. But anyway, I gather from your notes that it is better if i rewrite all such labels from scratch at the testing module. Thank you anyway @Trappist the monk:. --Sarri.greek (talk) 21:42, 15 October 2020 (UTC)

Are you sure? In this version it looks like you got part way there and declared failure before completing the job. But I can't be sure because the table that you modified with all of those commas is commented out. So it you were trying the experiment with that version of the module and expecting it to work, of course you will be disappointed. The other thing that is wrong is that each k/v pair where you added a comma, you did not remove the label at the start of every k/v pair. This is wrong:

label['αγγλισμός'] = { link = 'αγγλισμός', linkshow = 'αγγλισμός', word_cat = 'Αγγλισμοί', key = 'αγγλισμοι', cat_group = 'style' },

it should look like this:

['αγγλισμός'] = { link = 'αγγλισμός', linkshow = 'αγγλισμός', word_cat = 'Αγγλισμοί', key = 'αγγλισμοι', cat_group = 'style' },

—Trappist the monk (talk) 22:03, 15 October 2020 (UTC)

I am a terrible student: I forgot to delete the word label. Also, There was one cat_group = 'xxx' missing somewhere, this also created a problem. Which means, I gather, that I must have exactly all the keywords at all the rows. Thank you, @Trappist the monk:, again. Sarri.greek (talk) 22:39, 15 October 2020 (UTC)

I am testing it for matching, but i am not very lucky -because me and Lua met by chance, only from luck do i get things right-. I have to study first some Lua. May I ask, if we need to call the very first thing, the one which used to called 'label', by what name do we call it? All the others have a name (link, linkshow etc). It is 3:30 in the night, (or morning), i give up Sarri.greek (talk) 00:36, 16 October 2020 (UTC)

@Sarri.greek: I recommend linking to a talk page at el:wikt where you briefly describe the big picture. Then I could have a look at the overall design and may have some ideas. It's not very productive to examine each tiny problem that arises when the overall task is unknown. You may have several jobs in mind but you should start by describing one of them, preferably with example wikitext that would go on a page, along with the expected output that would be generated. Add a brief explanation of what it's all about and how the output would be calculated. The description can be rough—it does not need to be a full specification, just an idea. Johnuniq (talk) 01:09, 16 October 2020 (UTC)

@Trappist the monk:, thank you for teaching me, I have made lots of data pages as you have shown me. Have a nice holiday, and thank you all Sarri.greek (talk) 11:51, 21 December 2020 (UTC)

file upload date?

Anybody knows if there is a way to look up file upload date or page creation date for a file page from Lua? For either current page or for named page? Thanks --Jarekt (talk) 01:19, 2 January 2021 (UTC)

@Jarekt: I can't see a way at present, but I'll think about it. My initial thought was that the creation date is in the Page information page associated with every page, so if we could read the content of that page, we could search for the date, but {{#invoke:Page|getContent|Special:PageInfo/WP:AFC}} produces:

which is disappointing. It's the usual problem that Scribunto doesn't provide access to the Wikipedia API, which is the normal way of retrieving that sort of stored info about a page. Maybe somebody else has cracked the problem? --RexxS (talk) 02:29, 2 January 2021 (UTC)

I don't think it's possible. As you know, JavaScript could be used to access the API if that was any use. Johnuniq (talk) 03:32, 2 January 2021 (UTC)

I did not think it was possible either, but maybe someone found a way. I just discovered that someone's solution to a similar problem, of accessing creation date, was creating all kind of havoc on Commons in c:Category:Pages with script errors and in c:Category:Pages with malformed coordinate tags. The crazy approach was to use frame:expandTemplate function to expand the filepage as if it was a template, stuff the wikitext between <html></html>, use HTML parser to parse it and in the end access the date element, (see see here in function dateWorkCreated). The approach strangely works for some pages, but gives bizarre errors for others. --Jarekt (talk) 03:58, 2 January 2021 (UTC)

That's showing a great deal of initiative. The problem with using undocumented tricks is that it is bound to fail somewhere or sometime. A MediaWiki update could make the system fail completely. Johnuniq (talk) 04:29, 2 January 2021 (UTC)

It's a hack and indirect, but you could have an external bot that maintains a list of creation dates and stores them in files in your user-space or project-space, then have the template parse that. It wouldn't cover newly-created pages/files and it would give false results in cases of deletion-and-recreation or file-revision-deletion and re-upload, but it might serve the need. You could do the initial run against an archive or mirror, but you'd need to get bot-approval group to do the runs that update the logs. It's a hack that shouldn't be needed. davidwr/(talk)/(contribs) 🎄 18:18, 2 January 2021 (UTC)

Another idea: Have the template output something like

{{Hover title|date pending|A bot should fill in the correct date by [current timestamp plus small delay], purge the page to check for updates}}[[category:pages that need page creation dates inserted by a bot]]

then have a bot monitor that category and replace the whole thing with [actual date goes here][[category:pages processed by bot-name|timestamp goes here]] within minutes. Have the same bot re-process all pages in its "pages processed by" categories that were last processed more than, say, 30 days ago. davidwr/(talk)/(contribs) 🎄 18:26, 2 January 2021 (UTC)

wikidata artwork item description

I am looking for Lua code for creation of basic wikidata artwork item descriptions. For example if we have instance of (P31)=painting (Q3305213) and creator (P170)=Gustave Courbet (Q34618) than the function would return "painting by Gustave Courbet" for English and "tableau de Jules Breton" in French. It should not be hard to write one that works for 10-20 languages and similar number of most often found artwork types, but I was checking is someone already wrote one of those. --Jarekt (talk) 02:07, 10 January 2021 (UTC)

This might be very hard for slavic laguages for example "obraz Gustave’a Courbeta" in Polish. There is required to create genitive form of names, and there are countless rules for it. Paweł Ziemian (talk) 10:44, 10 January 2021 (UTC)

Paweł, You are right genitive form of names in Polish and other Slavic languages complicate any algorithmic solutions. I have seen 3 approaches, all of them quite bad:

incorrectly use nominative form instead of genitive one, like in case here
do not add automatic descriptions to languages you do not handle
Contort the phrasing so nominative case can be used: instead of "painting by X" use "painting (author: X)"

As a Polish speaker I sometimes have hard time figuring out what a genitive form be for names rarely seen in Polish language. I even looked up official Polish guide for declension of foreign name and it is quite a mess with some changed and some remaining in nominative form, unlikely to handled by algorithm. --Jarekt (talk) 23:17, 10 January 2021 (UTC)

@Jarekt: this is probably the sort of issue that the folks working on meta:Abstract Wikipedia might already have solutions for. They probably don't have solutions written in Lua, but if they have underlying algorithms for dealing with languages that decline their nouns, etc. it might be possible to leverage those for Lua. It's worth checking with the Abstract team members if they have anything you could make use of. Cheers --RexxS (talk) 12:11, 10 January 2021 (UTC)

RexxS, thanks I never heard of meta:Abstract Wikipedia but they are dealing with issues Commons was struggling for last 15 years. --Jarekt (talk) 23:21, 10 January 2021 (UTC)

`gsub` problem

Please see these two pages first:

code
sandbox

As you can see from the code, I am printing the text to the sandbox before calling gsub and after calling gsub.

The problem here is that __M_CONTENT__ is not replaced by escape_replacement(content). Any ideas? Or is it a Scribunto bug?

– Ase1este^t@lk_c0ntribs 11:38, 19 January 2021 (UTC)

~~Nvm, found the problem right after posting this. 😓~~ – Ase1este^t@lk_c0ntribs 11:43, 19 January 2021 (UTC)

code, debug output. It is still a problem...

Looking at one of the replacement result: (text is input, result is output)

["result"] = "{{Userboxgroup\ |title=[[Wikipedia:WikiProject |Projects]]\ |collapsed=true\ |content=__M_CONTENT__}}", ["text"] = "{{Userboxgroup\ |title=[[Wikipedia:WikiProject |Projects]]\ |collapsed=true\ |content=__M_CONTENT__}}"

The input and the output are the same. It seems like gsub is simply not replacing even though __M_CONTENT__ is in the original string.

However, sometimes the replacement works, as shown below: (text is input, result is output)

["content"] = "{{User Wikipedian for |year=2019 |month=7 |day=22 |link=true |sc=true }}\ {{User Wiki age |year=2019 |month=7 |day=22 }}\ {{User suck |2019 }}\ \ {{User numberofusers2}}\ {{User 1ofActive}}\ \ {{User edit summary |{{PAGENAME}} |left-text=99.{{Overline |9 }}% }}\ \ ", ["result"] = "{{Userboxgroup\ |title=[[Statistics]]\ |collapsed=true\ |content={{User Wikipedian for |year=2019 |month=7 |day=22 |link=true |sc=true }}\ {{User Wiki age |year=2019 |month=7 |day=22 }}\ {{User suck |2019 }}\ \ {{User numberofusers2}}\ {{User 1ofActive}}\ \ {{User edit summary |{{PAGENAME}} |left-text=99.{{Overline |9 }}% }}\ \ }}", ["text"] = "{{Userboxgroup\ |title=[[Statistics]]\ |collapsed=true\ |content=__M_CONTENT__}}"

The situation is the same for other 'magic words'.

This is very confusing.

– Ase1este^t@lk_c0ntribs 14:49, 19 January 2021 (UTC)

Step one: create a simple testcase that shows the problem; simpler the better. A wall of text won't help others help you.

Just to confirm that gsub can replace __M_CONTENT__, in the debug console I wrote:

=string.gsub("content that has __M_CONTENT__ magic word", "__M_CONTENT__", "bob")

which returned this:

content that has bob magic word 1

—Trappist the monk (talk) 15:41, 19 January 2021 (UTC)

Finally, I think I know the true cause now. It is because of escape_replacement. Because it directly returns the result from gsub, it returns two results.

When the code is like this: mw.ustring.gsub('some', 's', escape_replacement('a')), it evaluates to mw.ustring.gsub('some', 's', 'a', 0). The last 0 tells gsub to replace nothing. Therefore, it looks like gsub is replacing nothing.

Looks like I am not familiar enough with multiple return results.

– Ase1este^t@lk_c0ntribs 01:44, 20 January 2021 (UTC)

If there's any chance of multiple results being returned, it's good for your sanity to assign the returned value that you want to a temporary variable like local esc_rep = escape_replacement('a') and then use that variable: mw.ustring.gsub('some', 's', esc_rep). Of course it's less efficient, and the script kiddies will moan about "slow code", but when you come back to it months later to debug an issue, or if somebody else needs to see what your code is doing, it makes life much easier. --RexxS (talk) 02:28, 20 January 2021 (UTC)

if all you want is to suppress the 2nd return value, a simple alternative to an extra parameter, is to enclose the call in parentheses, which will swallow all but the 1st return value, like so:mw.ustring.gsub('some', 's', ( escape_replacement('a') ) ). This extra parentheses prolly deserve a comment, so just counting characters, you may not gain much over an extra variable. Peace. קיפודנחש (aka kipod) (talk) 04:17, 20 January 2021 (UTC)

Would something like _,result = string.gsub(line,'something','other') or _,result = string.gsub(line,'something','other',1) be of any use.. - Matroc (talk) 23:55, 1 February 2021 (UTC)

Infobox script errors

There are several lua script errors at ln: ln:Richard_Ali_A_Mutu, ln:Alfred_Liyolo, ln:Yoane_ya_La_Fontaine, ln:Flavour_N'abania, ln:Bokungu_(eténi), ln:Sato_ya_Beaumaris, ln:MoziloCMS, ln:Marie Misamu, ln:Mabánda_ma_nsé_mpé_antaktiki_ya_Falansia I have no idea about LUA and am more or less the only regular cleaner on ln: . Could someone who knows something about it please take a look? Thank you Bombo (talk) 13:10, 10 February 2021 (UTC)

@Bombo: To create an infobox, we currently use a template that calls a Lua module, so both the template and the module are needed if you want to import them into another language Wikipedia.

The problem is that the Lua module Module:Infobox in use on the Lingala Wikipedia is taken from the one on the English Wikipedia, but the template "Modèle:Infobox Biographie2" has been taken from the French Wikipedia and it expects a different fr:Module:Infobox from the English one. They are not compatible. You added the ln:Infobox Biographie2 template to the article but didn't change the corresponding module.

You need to make a decision whether to use the English en:Template:Infobox person with English en:Module:Infobox or use the French fr:Template:Infobox Biographie2 with the French fr:Module:Infobox. Obviously you can't have a mixture because the modules have the same name. Be careful if you replace the English module with the French one, as that will cause errors if you have any uses of English infobox templates. I hope that gives you an idea of what needs to be done. --RexxS (talk) 14:46, 10 February 2021 (UTC)

Module:Message box

It would be great if we could somehow make {{tmbox}} have an opt-in collapsible nature to it. I tried playing around with the lua of the message box for a few hours now, but I can't seem to do it myself. For clarity, I am talking about a situation where we could input:

Extended content

{{tmbox|type=speedy|collapse=yes|headerstyle=font-size: larger;|headertext=Before requesting any edits to this protected article, please familiarise yourself with [[WP:Reliable sources|reliable sourcing]] requirements.|image=[[Image:Stop_hand.svg|40px|]]|text=Before posting an edit request on this talk page, please read the '''[[WP:Reliable sources|reliable sourcing]]''' and '''[[WP:Original research|original research]]''' policies. These policies require that information in Wikipedia articles be supported by citations from reliable independent sources, and disallow your personal views, observations, interpretations, analyses, or anecdotes from being used.
<div style="clear:both;"></div>
Only content verified by subject experts and other reliable sources may be included, and uncited material may be removed without notice. If your complaint is about an assertion made in the article, check first to see if your proposed change is supported by reliable sources. '''''If it is not, it is highly unlikely that your request will be granted.''''' Checking the archives for previous discussions may provide more information. Requests which do not provide citations from reliable sources, or rely on unreliable sources, may be subject to closure without any other response.}}

To get this. –MJL ‐Talk‐^☖ 04:45, 14 February 2021 (UTC)

Mobile does something very similar today. This is something that should get on the pile for making message box 2.0 (i.e., div-based rather than table-based). For now you should add your own collapsible element if there is some component that you don't think should display by default. In general, I would prefer very much not to add an 'opt-in'. --Izno (talk) 05:31, 14 February 2021 (UTC)

@Izno: the only way for me to add a collapsible is to replace {{tmbox}} with the <table>...</table> tags. –MJL ‐Talk‐^☖ 16:24, 14 February 2021 (UTC)

How so? Just don't attempt to collapse the top element. Add some divs with the appropriate classes internal to the element in question. In general, as I said, I think this is a bad idea right now, and it might be more of a "never a good idea" too. --Izno (talk) 18:25, 14 February 2021 (UTC)

Newbie question

So I am teaching myself Lua by jumping right in and creating modules. One thing that I'm failing to understand is when you can pass an arg to the function via #invoke, how do you do that from another module?

I'll just show you. I wrote Module:Road data/size with the intention that it could be used by multiple modules. Its purpose is to tell other modules what size images called by it should be through the style argument. It works great by invoking it:

{{#invoke:Road data/size|size}} → 24px

{{#invoke:Road data/size|banner|style=infobox}} → Script error: The function "banner" does not exist.

How do I call that size function with the style argument from another module? I have variables set up for the two functions in the module:

local sizeModule = require " Module:Road data/size"
local shieldSize = sizeModule.size(style)
local bannerSize = sizeModule.banner(style)

I figured out that you have to put style in brackets otherwise it thinks you're trying to pass the whole function and not the result. I feel like I'm close, but I need a hand. –Fredddie ™ 09:35, 14 February 2021 (UTC)

One way is to split the functions so, for example, this:

function p.banner(frame)
	local args = getArgs(frame)
	return p._main(args)
end

becomes:

function p._banner(args)
	return p._main(args)
end

function p.banner(frame)
	local args = getArgs(frame)
	return p._banner (args)
end

The other module builds an args{} table before it calls p._banner(args).

You might want to get in the habit of adding

require('Module:No globals')

to your modules. Is it really necessary to tostring() the strings that p._main() returns? Is p._main() an exported function? If not, then

local function _main(args)

is better.

—Trappist the monk (talk) 12:22, 14 February 2021 (UTC)

Since you are already using the arguments module, just do exactly what it tells you at Module:Arguments/doc#Recommended practice (and some more that I have added that hopefully makes it easier). --Izno (talk) 18:37, 14 February 2021 (UTC)

Module:Repr

Hi all, I've just finished writing Module:Repr, a module for generating string representations of Lua object. The repr function works a lot like mw.dumpObject, except that you can customise the whitespace and indentation, and the output is meant to be mostly valid Lua if you copied and pasted it back into a module. It takes its inspiration from Python's repr function, and is based on https://github.com/Ozzypig/repr (with heavy modification and test cases). Also there is an invocationRepr function for generating string representations of function invocations. If you think the module would be helpful for you, please give it a try, and let me know what you think. — Mr. Stradivarius ^{♪ talk ♪} 10:50, 27 February 2021 (UTC)

Is this not Module:Dump? --Izno (talk) 20:04, 27 February 2021 (UTC)

It's similar to Module:Dump's _dump function, but with some extra features: the option to choose whether or not to pretty-print tables, some extra formatting options, and the ability to generate string representations of function invocations. And it doesn't come with the "variable = " part at the start. On the other hand, it is not meant for being output on wiki pages directly, so it doesn't contain features for formatting the result in a <pre>...</pre> block etc. — Mr. Stradivarius ^{♪ talk ♪} 00:47, 28 February 2021 (UTC)

property used more than once on a Wikidata item

I have a module Module:Wdtablerow/listed buildings which pulls data from WD including property P87 (lines 89-100). Now I've discovered that at least one of the Wikidata items e.g. Old Mining Building (Q26262664) I am pulling this property from has two instances of P87. The module only pulls the first use of P87. How do I call all the uses of P87? Nthep (talk) 14:19, 28 February 2021 (UTC)

I presume that you mean P84? Because you are using Module:Wd to interface to Wikidata, isn't your question best asked at Module talk:Wd?

—Trappist the monk (talk) 14:53, 28 February 2021 (UTC)

I did and I will. Nthep (talk) 16:10, 28 February 2021 (UTC)

Module:Infobox

I recently updated bs.wiki's version of Module:Infobox to Module:Infobox with some argument mapping, and I noticed some strange behavior when embedding infoboxes. See Korisnik:Srđan/ugraditest and User:Srđan/test1. There seem to be two extra empty HTML rows between the fields "Instrumenti" and "Rođenje" (analogous to "Instruments" and "Born") whereas on en.wiki it looks like something (perhaps lines 15–62?) is applying style="display:none" to one of them, but I can't for the life of me figure out why it isn't doing that on bs.wiki. Could someone take a look, please? – Srđan (talk) 17:35, 7 March 2021 (UTC)

Infobox styling

Infobox of mni.wikipedia is align on left as in ꯏꯝꯐꯥꯜ can anyone help to align it(infobox) on right side of a page as in other wikis.ꯂꯨꯋꯥꯡ (ꯆꯥ) 16:12, 15 March 2021 (UTC)

@Luwanglinux: Your Common.css does not have any content. --Izno (talk) 16:31, 15 March 2021 (UTC)

(edit conflict)

Not likely a Lua problem. I think that at en.wiki, this bit of css (particularly line 343) is what renders infoboxes at the right side of the displayed page. There is an equivalent page at mni.wiki but it is empty: mni:MediaWiki:Common.css.

—Trappist the monk (talk) 16:33, 15 March 2021 (UTC)

Can you pliz help with this interface problemꯂꯨꯋꯥꯡ (ꯆꯥ) 16:41, 15 March 2021 (UTC)

I don't have the necessary permission to edit interface pages at mni.wiki so you will have to find someone who does. The simple test is to copy or import en:MediaWiki:Common.css to mni:MediaWiki:Common.css.

—Trappist the monk (talk) 16:48, 15 March 2021 (UTC)

Template error

Hi guys here I am again seeking for a solution ..there seems to be a bug in speciesbox or taxobox in mni.wikipedia,showing annoying ";" template which can't be created either like this mni:ꯀꯕꯣꯀꯩ and this mni:ꯅꯣꯡꯁꯥ ꯂꯨꯋꯥꯡ (ꯆꯥ) 13:37, 25 March 2021 (UTC)

You will probably want to ask at that module talk page, since I know those modules are generally well-watched. Izno (talk) 18:41, 25 March 2021 (UTC)

Izno's advice is good. Meanwhile, the mess at the top of the infobox includes "solid red; error:colour" and the article is in categories mni:Category:Taxoboxes with the error color and mni:Category:Taxobox articles possibly missing a taxonbar which needs investigation. I put a simplified infobox in User:Johnuniq/sandbox (permalink). It shows what should be at the top of the infobox. Very weirdly, it uses a template called ; which cannot be viewed but which can be edited (see links in sandbox). When I tried that trick to edit the semicolon template at mni, I seemed to get the main page. Johnuniq (talk) 00:25, 26 March 2021 (UTC)

Template:; is really weird! I'm tempted to nominate it for deletion just to prevent future headaches, but based on the history it probably has been functional at some point. I'm currently running a quarry to see how many transclusions it has, since what links here is broken, but the template links table is notoriously slow. --Trialpears (talk) 01:29, 26 March 2021 (UTC)

I was able to Special:WhatLinksHere to work by visiting https://wiki.riteme.site/w/index.php?title=Special%3AWhatLinksHere&target=Template%3A%3B&namespace=. Unfortunately, it has 434,035 transclusions. For context, the page is inaccessible due to phab:T238285 * Pppery * _{it has begun...} 01:32, 26 March 2021 (UTC)

Looks like deletion isn't the answer then. Perhaps moving it to Template:Semicolon would be a good solution to mitigate the problem until it's properly solved. --Trialpears (talk) 01:43, 26 March 2021 (UTC)

As noted, edirect Template:Semicolon exists and works as a method to display Template:;. Apparently it's a method to avoid a semicolon at the beginning of a line which would be markup. But how many times is a semicolon at the beginning of a line wanted!? It appears to be due to the very mysterious {{Taxobox}}. Johnuniq (talk) 02:20, 26 March 2021 (UTC)

I'm too irritated by the limitations of "what links here" to track down how {{;}} is used but I'll record an item I found. Template:Wd/doc displays ;_ with an abbreviation thingamy that shows "semicolon space" (it looks like that page uses an underscore to show a visible space); wikitext: {{dfn|;_|title=semicolon space}}. I tried previewing a plain semicolon and it breaks so {{;}} or equivalent is needed there. Johnuniq (talk) 02:34, 26 March 2021 (UTC)

The template can be viewed here. Cirrus search suggests only 16 direct uses, including transclusions, but it may be wrong. Petscan thinks there are 417397 transclusions, including indirect ones. Perhaps {{;}} is used once in a widely-used template, which would explain why it is protected. Certes (talk) 10:47, 26 March 2021 (UTC)

My count gives 13,244 direct uses which is slightly more. I'm very tempted just to move it as the current title is very broken and I can't imagine people thinking it isn't an improvement. --Trialpears (talk) 13:04, 26 March 2021 (UTC)

I suspect 99% of uses could be subst'd. The documentation says it's used to avoid confusion with a <dt> tag, by which I think they mean the wikitext for a description list, but that's only a problem at the start of a line. Of course, many templates containing {{;}} might be used at the start of a line, either now or in future. Certes (talk) 13:13, 26 March 2021 (UTC)

13k indeed. We should probably subst its use in any templates. Probably in most talk pages it's a typo for {{;)}}.

The particular template is similar to another one I shipped to TFD recently which I would guess has since been kept. It had similar "accessibility" issues (in the non-technical sense). Izno (talk) 20:45, 26 March 2021 (UTC)

89 possible uses in template/module space. Izno (talk) 20:47, 26 March 2021 (UTC)

Date formatting outside citations

Following up from this VPT thread, I've come across several instances where dates have been used in templates (like {{Spoken Wikipedia}}) and I've wanted to format those dates appropriately according to whether an article is tagged with {{use mdy dates}} or {{use dmy dates}}. To do this, The Earwig mentioned that some of the code in the CS1 module would have to be moved. Would anyone who's familiar with module coding be willing to help? {{u|Sdkb}} ^talk 05:39, 31 March 2021 (UTC)

That would require great care to avoid stupendously high overhead when previewing or publishing an edit. I would need to remind myself to be sure but ISTR that the CS1 module loads the wikitext of the current page then parses the wikitext to decide whether to use dmy or mdy dates. If (that's if) that is a good idea, it should be done once only per page, not once for every template where someone thinks it would be nice to have auto-formatting dates. My hunch is that this should be done with some software enhancement. I have occasionally wondered whether Scribunto (Lua) could allow a per-page global table that could be written and read using mw calls. That is the sort of thing that would make the idea of automatic dmy/mdy format choice feasible. Johnuniq (talk) 06:53, 31 March 2021 (UTC)

Hmm, good to know. I'm not quite sure what makes it possible to do for references but so expensive to do outside of them; I guess there's something? {{u|Sdkb}} ^talk 10:04, 31 March 2021 (UTC)

I'm saying that if a reference reads and parses the whole page to get the date format, that information should be available to other modules/templates so they do not have to also read and parse the whole page. There is a way that could be arranged that would involve writing a single module to read and parse the whole page and return a table of what it finds, including the date format. The reference module would then need to use that central module by using mw.loadData which loads the table once per page no matter how many time the central module is loaded. Then another template could (via a module) also mw.loadData the table. Only the first load would involve reading and parsing the whole page. That is how references work now, but the information is part of the reference system and (I imagine, I haven't looked) is not readily available for another template. I think the first step might be a central discussion about the desirability of the scheme if it is technically feasible. For example, your link to the VPT thread did not show much enthusiasm and a comment said "automatic date formatting was abolished". Johnuniq (talk) 02:50, 1 April 2021 (UTC)

Johnuniq, ah, got it, thanks for the explanation. Redrose64, do you have a link to that discussion by any chance? And is what you're referring to the same thing I'm talking about here, or was it the kind of thing where users could set date format preferences? {{u|Sdkb}} ^talk 03:04, 1 April 2021 (UTC)

this is a clever idea, but i do not think it's a good one. currently, the doc page mentions mw.ext.cattools (which is not yet available on wikimedia wikis), which supposedly will allow asking if current page belongs to some category. it should be straightforward to teach these templates (e.g., {{use mdy dates}}) some known category, that any module will be able to test for. it is not a good practice to read content pages from scribuntu.

(offtopic disclosure: i do it myself, not for content pages but for template pages, because "templatedata" team and "scribuntu" team do not speak with each other. reading templatedata is something that modules want to do often, but unfortunately mw.templatedata scribuntu extension was not yet written. see phab:T107119).

peace - קיפודנחש (aka kipod) (talk) 00:46, 8 April 2021 (UTC)

Get User info through Lua

I'm trying to make a template that takes a username, checks how many edits it has made, and outputs it. I couldn't do that using normal templates and I am wondering whether there's a way of doing this through Lua. --Aknell4 (talk) 19:18, 11 April 2021 (UTC)

In theory you could parse the wikitext of Wikipedia:List of Wikipedians by number of edits (which could be done either in Lua directly or in Wikitext using Module:Page and Module:String), but otherwise this isn't possible with Lua either. * Pppery * _{it has begun...} 21:42, 11 April 2021 (UTC)

Fetching info for a diff ID

I'm trying to get the author username and date from a diff ID number for use in a template. Is there some Lua module or API that could achieve this? If not, is there some way I could fetch a revision of a page and do some string parsing on it to extract this information? Srey Sros^talk 01:56, 5 May 2021 (UTC)

I'm pretty sure Lua cannot do this but the API can. The API would require JavaScript which is not callable from a template. In other words, I don't think you can get a template to do this. If no further comments occur here, you might try WP:VPT. Johnuniq (talk) 03:58, 5 May 2021 (UTC)

table.sort

I just received a note on my talk page that the players in Template:Atlanta Dream current roster were not entirely sorted by number, with 40 before 25 and 32. at first, I thought that ipairs() wasn't processing the table in order, so I tried this which had no impact. after that, I made this change which calls the table.sort() twice instead of once and the problem was fixed. is there something wrong with my sort function or is this a known bug? thank you. Frietjes (talk) 16:25, 20 May 2021 (UTC)

@Frietjes: The problem may lie in (a['last'] or 'ZZZZ') < (b['last'] .. 'ZZZZ'). If x['last'] == y['last'] (both "Williams", say) then this is true for both x<y and y<x, because "Williams" < "WilliamsZZZZ". (Did you mean b['last'] or 'ZZZZ'?) However, that comparison should only occur when the numbers are equal, which they're not. Certes (talk) 16:51, 20 May 2021 (UTC)

Certes, thank you for finding that bug. as you said, that comparison should only occur when the numbers are equal. but, when I fix that bug in the sandbox, they are now sorting correctly (see Template:Atlanta Dream current roster/sandbox) with only one table.sort()? Frietjes (talk) 17:08, 20 May 2021 (UTC)

An aside, I often create a fixed length sort key to use in sorting records/entries (ie. format numbers with leading zeroes, upper/lower case name or not depending on case sensitivity desires, code expansions, leading look ups and type of entry etc.). Since I know the lengths or each position of the key I can manipulate the original sort key accordingly to create new ones for sorting. I often used this to produce author, title, keyword indexes for manuals, catalogs, books etc. This may or not be another approach to consider. - Matroc (talk) 05:13, 25 May 2021 (UTC)

Italic dab not working with titles that have disambiguation with parentheses

{{Italic dab}} via Module:Italic title does not work on titles that have disambiguation with parentheses such as The Ghost Talks (Randall and Hopkirk (Deceased)). In this example, "Randall and Hopkirk (Deceased)" is the whole disambiguation.

From what I can tell, the logic is found in line 82-85: prefix, parentheses = mw.ustring.match( title.text, '^(.+) %(([^%(%)]+)%)$' ) I've tried my best, but I can't find the correct pattern to fix this. Can anyone help with this? Gonnym (talk) 13:07, 26 May 2021 (UTC)

You could try a pattern like '^(.-) %((.+)%)$'. However, as this is simpler than the original, I suspect that it may break some use cases that the original author thought of and I missed. Certes (talk) 14:03, 26 May 2021 (UTC)

Haven't tested it, but I bet that would match e.g. "(B) (C)" in "A (B) (C)", which I don't think is desirable. OTOH, if we made the latter capture non-greedy, it may only match "Deceased)". Perhaps we should simply instruct editors to manually specify the part to italicize in DISPLAYTITLE in cases like this. Nardog (talk) 14:21, 26 May 2021 (UTC)

But [[A (B) (C)|]] turns into "A", so maybe this isn't so bad? Nardog (talk) 14:24, 26 May 2021 (UTC)

Lua's special construct %b works, though it captures the parentheses: "^(.+) (%b())$". They can then be removed with parentheses = parentheses:sub(2, -2). — Eru·tuon 03:01, 27 May 2021 (UTC)

@Erutuon: that works great! Thanks! --Gonnym (talk) 13:47, 27 May 2021 (UTC)

Lua error: not enough memory.

Any guidance related to sorting out "Lua error: not enough memory" on the page COVID-19 pandemic in India? The error started since this edit on 24 May 2021 and currently appears after citation 59. Every transclusion, template used seems to be causing the error from that point onwards. (Also seeing the same at the commons category for the same article "Category:COVID-19 pandemic in India") DTM (talk) 15:34, 26 May 2021 (UTC)

Removing {{Update|date=April 2021}} was the cause of all those Lua erros. I've put it back and they have gone. I've no idea why the error is occurring but think an unnecessary template is better than all the errors while we try and work it out. — Jts1882 | talk 16:26, 26 May 2021 (UTC)

Jts1882, thanks for the fix (hack/jugaad) for now. DTM (talk) 08:31, 27 May 2021 (UTC)

frame:callParserFunction('#time', 'j M y', frame:callParserFunction('REVISIONTIMESTAMP', page))

Is there a better way to write that with native Lua—rather than parser—functions? — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 (talk) 08:12, 27 May 2021 (UTC)

I don't think so. You could find an alternative to the #time but it would be ugly. However, REVISIONTIMESTAMP requires calling the parser function and I don't think there is any alternative to frame:callParserFunction. It has low overhead so I wouldn't worry about it. Johnuniq (talk) 04:01, 28 May 2021 (UTC)

The lua version of time is mw.language():formatnum(). You could either use mw.language.getContentLanguage():formatnum() to use the language of the wikipedia in question, here it is english. Or you could do mw.language.new(languagecode):formatnum() for an specific language. There is not a pure lua alternative for revisiontimestamp.--Snævar (talk) 20:41, 1 June 2021 (UTC)

Need help with Module:Coordinates

I run into a problem with this module on bnwiki. In short: interlanguage links will move to on the top of the page. If we do nothing it will broke coordinates display on bnwiki. MW developer recommended to use indicator extension. So, i applied this to bnwiki, it fixes the problem. But i run into following problem: coord2text function stopped working with display=title (it works with display=title,inline or display=inline but not with display=title). coord2text should extract the longitudine and latitudine from coord template. It shouldn't display coordinates at the top of the page. But right now on bnwiki:

{{#invoke:স্থানাঙ্ক/খেলাঘর|coord2text|{{Coord|57|18|22|N|4|27|32|E|display=title}}|lat}} gives nothing & shows coordinates at the top of the page
{{#invoke:স্থানাঙ্ক/খেলাঘর|coord2text|{{Coord|57|18|22|N|4|27|32|E|display=inline,title}}|lat}} gives 57.30611 & shows coordinates at the top of the page

Both example should give 57.30611 and shouldn't display coordinates at the top of the page. I found similar problem with bn:মডিউল:অবস্থান_মানচিত্র/খেলাঘর#L-165 (Module:Location map, testcases).

Coord2text has documentation (explanatory text) that says that it only takes one direction at once, so either longitude or latitude, not both. So, that is working as intended. You are supposed to call coord2text twice, once for latitude and once for longitude. As for the other problem, why don't you just do display=inline, that works, why do you need to specify display=title or display=inline,title ? --Snævar (talk) 06:46, 7 June 2021 (UTC)

আফতাবুজ্জামান would know about lat/long but I agree there needs to be an explanation for where the request would be used—why would you need to extract the lat or long from {{coord}} with display=title? Johnuniq (talk) 09:03, 7 June 2021 (UTC)

This concerns mw:Help:Page status indicators. The interesting feature is that the wikitext affects the display without being present in the final page (like references). Previewing either of the following lines in a sandbox shows them to have an effect although the invoke outputs only a number, so the original wikitext is not present in the rendered page.

{{#invoke:string|len|<ref>Hello</ref>}}
{{#invoke:string|len|<indicator name="coordinates">Hello</indicator>}}

For the first line, a reference is shown and the invoke displays 31 (the length of the ref strip marker). For the second line, "Hello" is shown at the top of the page in the indicator area and 37 is displayed (the length of the indicator strip marker). Regarding the original post, the problem is that {{coord2text}} receives only a strip marker when display=title is used. Currently, enwiki does not use the indicator method and the following shows that much more than a strip marker is returned.

{{#invoke:string|len|{{Coord|57|18|22|N|4|27|32|E|display=title}}}} → 827

The number displayed is currently 794 because the enwiki {{coord}} outputs 794 characters which includes the latitude and longitude to be displayed at the top of the page. Johnuniq (talk) 09:03, 7 June 2021 (UTC)

@আফতাবুজ্জামান, @Johnuniq, @Snævar, I don't know if it's related to your issue howsoever, but look how a module was edited on Korean Wikipedia. SGrabarczuk (WMF) (talk) 11:30, 7 June 2021 (UTC)

SGrabarczuk (WMF), Thanks but it doesn't work. @Johnuniq, @Snævar, i mainly need it for bn:Module:অবস্থান মানচিত্র#L-164 (Module:Location map). Both module uses same regex for coord2text, so i thought if one gets fixed, i could apply that to other one. For a use case consider this example, copy this and preview anywhere: {{Location map| Iraq | coordinates = {{coord|36|20|00|N|43|24|00|E|format=dms|display=title}} }} (it works here). Now preview the same code anywhere on bnwiki(we have redirects, don't worry about English name in the template.) result is it doesn't work. live example (see bottom right, there supposed to be a map). --আফতাবুজ্জামান (talk) 19:28, 7 June 2021 (UTC)

I left a long thought at bn:User talk:Johnuniq#2021 problem. Johnuniq (talk) 00:41, 8 June 2021 (UTC)

Why can't I pass this string as an argument?

Script error: No such module "Table graph".

The args seem to getting through fine to the function:

Script error: No such module "Table graph"..

And it works when the argument is called explicitly: Special:Permalink/1028302707

— 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 ☎ 05:10, 13 June 2021 (UTC)

It's because Template:Table graph inserts spaces before and after parameters 1, 2 and 3. That means the module receives " pie chart " as parameter 1. Johnuniq (talk) 07:00, 13 June 2021 (UTC)

Right! I never know when those spaces get counted and when they don't in template language.

Thanks, Johnuniq. — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 ☎ 08:03, 13 June 2021 (UTC)

Can this be made leaner and meaner?

Is there anything I can do make this lighter and avoid the break at row 218? Cheers. — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 (talk) 22:44, 25 May 2021 (UTC)

Looking at the HTML source of the sandbox shows Expensive parser function count: 477/500. I suspect you are breaking at row 218 to avoid passing the limit of 500. Also, Lua time usage: 7.063/10.000 seconds which suggests that you'd hit that limit after about 300 rows. Have a look at mw:Extension:Scribunto/Lua reference manual to see which expensive functions you might be able to call less frequently. Certes (talk) 14:13, 26 May 2021 (UTC)

The expensive parser functions for each gadget page are mw.title.new() and mRedirect.luaIsRedirect(). The first is necessary to get the content, while commenting out the latter doesn't seem to make a different and halves the expensive function count.

The current 217 limit seeems to be due to the characters in gadget 218: User:קיפודנחש/TemplateParamWizard.js. This causes a bad argument in gsub error.

The main problem, though, is time and I don't see how this can be avoided as each page content must be retrieved. The only solution I can see is to invoke the module several times from the template using something like this:

    local rowsToGet = frame:getParent().args['number-rows'] or 200  -- use this to determine number of rows to get (default 200)
    local rowOffset = frame:getParent().args['rows-offset'] or 0   -- use this offset to allow multiple calls to build larger table   
    ...
		count = count + 1
		if count > rowOffset then
	    ...
		end
		if count > rowsToGet + rowOffset -1 then  return mytable end
    ...

This would would need a change to how the table was built, as the header would need to be handled by the template rather than the module and the module would just return blocks of rows. — Jts1882 | talk 15:08, 26 May 2021 (UTC)

That doesn't work, as the 10-second Lua time limit is shared across all invocations. The only way around it I can think of is to move parts of the table to subtemplates updated using {{template cache}} * Pppery * _{it has begun...} 15:18, 26 May 2021 (UTC)

The only way around it I can think of is to move parts of the table to subtemplates updated using {{template cache}}.

Like that?

<includeonly>
{{User_scripts_table/1}}
{{User_scripts_table/2}}
{{User_scripts_table/3}}
{{User_scripts_table/4}}
</includeonly>
<noinclude>{{doc}}</noinclude>

<includeonly>{{#invoke:User scripts table | main | rows = {{{1|rows}}} | offset = {{{2|offset}}} }}</includeonly><noinclude>{{doc}}</noinclude>

{{template cache | User scripts table/main | 250 | 0}}
<includeonly>
<!--Template cache top-->
<!--Template cache bottom-->
</includeonly><noinclude>{{doc}}</noinclude>

etc... — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 (talk) 02:46, 27 May 2021 (UTC)

Yep, that's the idea. I've made a minor fix to Template:User scripts table/main, but otherwise it should work as soon as PearBOT runs at 03:30 UTC (in 33 minutes) * Pppery * _{it has begun...} 02:57, 27 May 2021 (UTC)

That specific attempt failed since {{template cache}} doesn't work properly when inside <includeonly>...</includeonly> tags. I've made some changes to the subtemplates to fix it, which should update properly in another hour. * Pppery * _{it has begun...} 03:44, 27 May 2021 (UTC)

Amazing! I noticed {{User_scripts_table/1}} broke at 154, though (even missing the last two cells). Does it too have a limit? (No biggie, though; I can cut it in 150 row chunks instead.) — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 (talk) 05:42, 27 May 2021 (UTC)

Actual non-portal uses of {{template cache}}!? That's coo! The bot really doesn't do anything sophisticated and just substitute the template given to it. If it hits template limits it will get the same issues as if a normal user hits template limits. I'm also unsure how easy it is to fix the includeonly issue, it comes from the bot checking the transclusions of template cache to determine which pages uses the feature. Since the template isn't actually transcluded at the main template if it's within includeonly the bot doesn't find it. If there is actual demand for the feature I'm happy to improve it. --Trialpears (talk) 07:08, 27 May 2021 (UTC)

The thing inside <includeonly>...</includeonly> tags that caused things to break was the "Template cache top" and "Template cache bottom" comments, which confused PearBOT since subst: markup doesn't get expanded there, not the {{template cache}} call itself. * Pppery * _{it has begun...} 12:25, 27 May 2021 (UTC)

@Trialpears: Two questions:

Do I need to set a frequency for updates, or does it default to the hourly schedule? I'm noticing caches are not being reupdated after the first one.
How can we prevent the timestamp outside the comment from being transcluded with the cache? (e.g. here).

— 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 (talk) 09:51, 28 May 2021 (UTC)

Guarapiranga Without a frequency it defaults to a daily schedule and if |noupdated=yes it noincludes the notice. I really did a bad job documenting this since not even I knew this without reading the code. --Trialpears (talk) 10:45, 28 May 2021 (UTC)

That's perfect, Trialpears. Once a day is good enough. You said in the summary the nouodate is not working but it is. Perhaps you're looking at the subtemplate page, confusing noinclude with includeonly. Having the update timestamp in the template page and not in the transclusion is perfect. Cheers. — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 (talk) 10:56, 28 May 2021 (UTC)

Yes I thought it would remove it entirely but using noinclude is significantly more clever. I don't think I've actually looked at the code since 2019. --Trialpears (talk) 10:59, 28 May 2021 (UTC)

I've investigated and fixed the breakage at line 154, which had to do with User:Steel359/Protection js having wikitext that had an unclosed comment at the end and has nothing to do with template limits. This means you could probably go back to 200 rows per page if you wanted to. * Pppery * _{it has begun...} 18:21, 28 May 2021 (UTC)

I would wrap an template invocation around the original table, that would allow you to get rid of mw.sting.gmatch (in line 20), which is one of two most timeconsuming functions in the module, mostly due to how often it is called. The module would then understand it as the table is a bunch of parameters, where each pipe signal sepeates each of the parameters. If I remember correctly, that pipe sign would need to be re-added in, the existing gsub function should be reused for that. The module would also regain the time used by getContent using that method, as it would no longer be necessary. Then, you need to remove Module:Redirect or mw.title.new().exists. Although it is a nice feature, there just is no room for it.--Snævar (talk) 17:27, 26 May 2021 (UTC)

mw.title.new isn't expensive in this case because it isn't called with a page ID. Strangely, titleObj.redirectTarget is not expensive, so that can be used instead of Module:Redirect or titleObj.isRedirect. And instead of checking the page exists, it works to just pcall the function that gets the page content. I made these changes to the module and it manages to load more lines of the table. — Eru·tuon 19:52, 26 May 2021 (UTC)

Wow, that was a master class! Thanks. Your improvements are most welcome, Erutuon. Curiously, I managed to run it up to 300 rows with them once, but when I went for more, the sun melted the wax on my wings, and I fell flat back to your 260. Cheers. — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 (talk) 00:58, 27 May 2021 (UTC)

I would wrap an template invocation around the original table

Thanks for the suggestion, Snævar, but the source table is generated by a bot I don't own, but I will take your suggestion to this other little project. Of course, I'll have to understand what you mean first: do you mean use mw.text.gsplit instead? — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 (talk) 01:03, 27 May 2021 (UTC)

the header would need to be handled by the template rather than the module

Done Yeah, makes much more sense. I didn't understand how I to do that before. Now I do. — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 (talk) 01:51, 27 May 2021 (UTC)

After all that {{template cache}} engineering, it turned out the simple solution was ditching Module:Transcluder, and using the getContext() function instead. I was already pattern processing the excerpt after fetching the content, so might as well. Now the module processes all rows (nearly a 1000) in less than half the time limit (4.814s). The problem with Module:Transcluder is that it makes abundant use of the ustring library, which according to Johnuniq and Mr. Stradivarius is a lot slower. Thank you all for the help. There's still some pattern polishing to beat the descriptions into shape, if anyone feels like some regex cubing. — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 (talk) 16:39, 31 May 2021 (UTC)

I haven't examined Module:User scripts table but noticed the excerpt code. It might not matter much if most pages are fairly short, but it could be worth truncating the string before calling gsub so they operate on a shorter string. Also, it is conceivable that some of the non-greedy patterns like .-}} could be replaced with a much more efficient greedy variation such as [^}]*}}. That would require hoping that the skipped text did not contain a single }. That only matters if the text being searched is reasonably long. Johnuniq (talk) 07:39, 1 June 2021 (UTC)

Thanks, Johnuniq. I just tried pre-truncating (as well as post-truncating), but got quite a bit gibberish (presumably from unfinished wikicode).

[^}]*} is faster than .-}? Interesting; didn't know that. I've changed =+.-=+ to =+[^=]+=+ and __.-__ to __[^_]+__, but, say, %[%[%s*[Ii]mage:[^%]]*%]%]([\n%[]) didn't work in place of %[%[%s*[Ii]mage:.-%]%]([\n%[]) bc those file/image/media wikilinks have wikilinks with []s inside them. Any workarounds? — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 ☎ 08:33, 1 June 2021 (UTC)

Sorry, don't have room to think about that at the moment. Re speed, it all depends and testing would be required to determine whether there was any difference (and there would be no practical difference in short strings). My guess is that .-}} would require Lua to take the next character (.) then check if the next two characters match }}. Then repeat for each following character until finding a match or end of string. However, [^}]*}} probably loops adding non-right-brace characters [^}] until it hits a brace or end of string, and then checking if }} follows. Johnuniq (talk) 09:54, 1 June 2021 (UTC)

If .-x and [^x]*x are mathematically equivalent, wouldn't the pattern-matching machine use the same algorithm for both, whichever is most efficient? — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 ☎ 12:15, 4 June 2021 (UTC)

Some regular expression engines might optimize this sort of thing, but Lua string patterns are pretty minimalist and they are basically executed as a set of instructions without much transformation. FWIW here's the code that handles the - repetition operator for the string patterns. — Eru·tuon 22:30, 4 June 2021 (UTC)

Thanks, Erutuon. I made the changes recommended by Johnuniq; indeed it reduced processing time by about a third. The regexing is throwing the baby out with the bathwater, however (it was already before today's changes; look at the top table entry row, for instance). I tested out disabling a number of gsubs but couldn't figure out the guilty one. — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 ☎ 06:52, 8 June 2021 (UTC)

@Guarapiranga: FWIW, the only items on the Lua profile report (which can be printed to the JavaScript console with this code in your common.js) that are over 1 second are

    ?                                                               2880 ms       47.1%
    Scribunto_LuaSandboxCallback::redirectTarget                    2360 ms       38.6%

? is a catch-all for any function that Lua couldn't get the name of (like an anonymous function), but string.gsub isn't on the list (it was at 40 ms before my recent edit) so if it's not included in ? then it's not the problem. That would make sense to me because I've generally found it very fast at processing large amounts of text. — Eru·tuon 21:15, 8 June 2021 (UTC)

Thanks, Erutuon. I didn't mean that the gsubs were impacting the performance, but that they were too greedy, throwing away too much text. Your latest change fixed that, including one of the entries in the table that was breaking it. It isn't anymore. Thanks again. — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 ☎ 21:45, 8 June 2021 (UTC)

@Erutuon: I added your code to my common.js, but can't find its effect. Should I see the NewPP data at the bottom of the parsed page? Also, could that code add a progress bar next to the x/y data in the NewPP limit report? Cheers. — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 ☎ 00:59, 16 June 2021 (UTC)

@Guarapiranga: You will see it if you open up the JavaScript console. In Windows, that's done with Ctrl+Shift+K (Firefox) or Ctrl+Shift+J (Chrome). The console is mostly plain text, so I'm not sure it's possible to make a progress bar in it. — Eru·tuon 20:22, 16 June 2021 (UTC)

Ah, I see! Is that to find it easier than in the source window? As for the plain text... You're showing your youth Erutuon; you're clearly not from the day and age of DOS and ASCII art 😀 Cheers. — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 ☎ 22:03, 16 June 2021 (UTC)

True... and I just remembered that wget and other terminal programs have progress indicators. The advantage of the JavaScript console is you don't have to preview the page (which can take a long time). A table somewhere on the page would be even nicer, but I didn't want to have to figure out the layout of it. — Eru·tuon 22:32, 16 June 2021 (UTC)

Can Lua getContent from external webpages?

I'm afraid I know the answer to this question, but my search of the archives didn't confirm it. — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 ☎ 08:07, 29 June 2021 (UTC)

No. This is a feature, not a bug. Johnuniq (talk) 08:58, 29 June 2021 (UTC)

Depending on the use case: Create a bot that gets the content from the external site and uploads to Commons tabular data (or Wikidata). Then Lua displays the data in the article via a template. Example {{NUMBEROF}} (the data is from the WMF API but same idea, external to enwiki); created by Johnuniq and myself. -- GreenC 18:44, 29 June 2021 (UTC)

Create a bot

That's above my pay grade ;D

Then Lua displays the data in the article via a template.

About that... I asked here whether module:Tabular data could (be made to) filter (search) by two (or more) fields. Radio silence as of yet. Do you know? — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 ☎ 21:53, 29 June 2021 (UTC)

The functions in module:Tabular data are fairly simple. What exactly do you want, to return the values in both columns if one or both have a search term or get the whole row if two column searches are both matches? It might be difficult to make a general purpose search function, so it might be simpler to make a custom module that does exactly what your template needs. — Jts1882 | talk 07:27, 30 June 2021 (UTC)

What exactly do you want, to return the values in both columns if one or both have a search term

No, to return a field value when two other fields are matched. The typical example, as I put there, is some stat for some entity at some time (e.g. population of Afghanistan in 1950). This is the most common use of tabular data, no? — 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚 ☎ 10:45, 30 June 2021 (UTC)

Looks like p._lookup() could be modified to use two search criteria by adding handling for |search_value2= and |search_column2=. — Jts1882 | talk 12:50, 30 June 2021 (UTC)

👍

it does not seem Module:Tabular data is something you want to use. its main purpose is to allow templates access to commons table data, and to generate wiki code for wikitables, consuming commons tables. for your module, i think it makes more sense to learn from it, and where sensible imitate it, rather than use it as a library. i think this is what Jts1882 meant when they said "The functions in module:Tabular data are fairly simple" read them twice, and you'll figure out how to do what you want to do. just my $.02. peace. קיפודנחש (aka kipod) (talk) 00:07, 20 July 2021 (UTC)

Jts1882 already created a mod on Module:Tabular data to filter on two columns, קיפודנחש. — Guarapiranga ☎ 04:32, 20 July 2021 (UTC)

this makes perfect sense if you want to extend the way a template can "query" the table. however, to extract data from commons table when you write a module (unlike "template"), the simplest way is to get it directly, and process it using your own code - you can start by copying one of the functions in that module, and change it so it will do exactly what you want. iiuc, the only "secret" you need is data = mw.ext.data.get(table on commons). you can look at the structure of the table you are pulling by examining the table's source, and extract from it using straight lua. peace - קיפודנחש (aka kipod) (talk) 04:47, 20 July 2021 (UTC)

I see. Ok, I'll give it a go. Cheers. — Guarapiranga ☎ 08:09, 20 July 2021 (UTC)

Here's @Jts1882's function; I can't see how I could've written it any better (other than perhaps replacing string.format for ustring.format). Might as well use that in the template:

function p._lookup2(args)

	--local data = args.data or mw.ext.data.get(args[1])
	local page = mw.text.trim(args[1])                 -- "UN:Total population, both sexes combined.tab" -- set page name explicitly for testing
    local data = args.data or mw.ext.data.get(page)                    
		

	local searchValue = args.search_value
	local searchPattern = args.search_pattern
	local searchColumnName = args.search_column
	local searchValue2 = args.search_value2
	local searchPattern2 = args.search_pattern2
	local searchColumnName2 = args.search_column2
	
	local searchColumnIdx
	for i, field in ipairs(data.schema.fields) do
		if field.name == searchColumnName then
			searchColumnIdx = i
		end
		if searchColumnIdx then
			break
		end
	end
	assert(searchColumnIdx, mw.ustring.format("Search column “%s” not found.", searchColumnName))

	local searchColumnIdx2
	for i, field in ipairs(data.schema.fields) do
		if field.name == searchColumnName2 then
			searchColumnIdx2 = i
		end
		if searchColumnIdx2 then
			break
		end
	end
	assert(searchColumnIdx2, mw.ustring.format("Search column “%s” not found.", searchColumnName2))
	
	local occurrence = tonumber(args.occurrence) or 1
	
	local numMatchingRecords = 0
	for i = (occurrence < 0 and #data.data or 1),
		(occurrence < 0 and 1 or #data.data),
		(occurrence < 0 and -1 or 1) do
		local record = data.data[i]
		
		if (searchValue and tostring(record[searchColumnIdx]) == searchValue) 
		   and (searchValue2 and tostring(record[searchColumnIdx2]) == searchValue2) then
			-- or (searchPattern and mw.ustring.match(tostring(record[searchColumnIdx]), searchPattern)) then
			
			if (1==1) then return data.data[i][3] end  -- just return single occurence 
			
			numMatchingRecords = numMatchingRecords + 1
			if numMatchingRecords == math.abs(occurrence) then
				local args = mw.clone(args)
				args.data = data
				args.output_row = i
				return p._cell(args)
			end
		end
	end
end

— Guarapiranga ☎ 10:26, 21 July 2021 (UTC)

Lua module to indent reference differently if it's displaying inside the references list

Hello,

I must admit I'm extremely bad at programming, so please keep that in mind. I want to make a Lua module which will indent a reference differently if it's displaying inside the references list, as opposed to the hover reference that appears when mousing over a reference in the article's body. How can I do this? So far, I've done this:

css.identifier = null
multiref.parameters_except_ref_one = null

function("GetOtherReferences")
	multiref.parameters_except_ref_one = (get all unnamed parameters except unnamed parameter 1 from Template:Multiref)
end

function("IndentReferences")
	css.identifier = (get identifiers of the elements which the reference is inside)
	if css.identifier == reflist then
		multiref.parameters_except_ref_one (indent them)
	end
end

Unfortunately I don't know how to achieve the bracketed text. Can someone guide me without giving the answer away so that hopefully I can try to improve at least a little?

However, I'm not sure if the code I've written is even any good. If we need to start over from the beginning, please let me know.

Thanks, DesertPipeline (talk) 08:23, 15 August 2021 (UTC)

This is very ambitious and may not be achievable in any efficient manner. Before investing much more effort, I suggest outlining a proposal somewhere (WP:VPR?) and explaining, perhaps with a dummy setup in a sandbox, what is proposed. Johnuniq (talk) 10:36, 15 August 2021 (UTC)

User:Johnuniq: I was thinking I could do it by simply switching between two versions of output depending on whether or not it's inside a reflist class element. The output when inside a reflist class element would just wrap everything except the first reference or text in <span style="margin-left: (whatever amount looks correct)">. Am I incorrect in thinking that'll work? DesertPipeline (talk) 10:44, 15 August 2021 (UTC)

I haven't thought about it and don't know. If you want opinions on the css side of it, ask at WP:VPT. However they would probably also want to know what the proposal is all about (what articles; what change; who would be affected). Johnuniq (talk) 10:48, 15 August 2021 (UTC)

User:Johnuniq: The module would be for Template:Multiref, so that when viewing the reference list, all paragraphs are indented the same (as in, every paragraph after the first has the same left margin as the first, which is further out than the rest usually because there's the reference number and up arrow to click to where the reference is in the article body), and when viewing the reference in popup which appears when mousing over a reference number, none of them are indented. DesertPipeline (talk) 10:57, 15 August 2021 (UTC)

The vertical alignment looks a bit off in both the rendered reflist and the popup to me. That kind of thing is not something I work on but it does make me wonder if {{pb}} used to join the items might work better (but I'm not going to investigate that). At any rate, setting up a sandbox with a clear demonstration of the issue and asking what can be done at WP:VPT looks best. Johnuniq (talk) 23:43, 15 August 2021 (UTC)

User:Johnuniq: Does {{pb}} need to be closed? I've had a lot of trouble getting that stuff to work while ensuring there are no extra linebreaks and no lack-of-linebreaks when there should be. It was quite frustrating.

Also, do you not think it would be a better idea for me to have help writing this module here? Why do you think that going to the technical village pump will be a better idea? I'm happy with whichever venue is more likely to get this working correctly; do you think that's here or somewhere else?

Considering my lack of skill with programming, do you think it might be better if someone else did this? My only concern there is that nobody but me might be interested in doing it; but if I do it, I'm going to need a lot of help along the way.

Do you know, at least, how I can get a Lua module to generate wikitext? I suppose that's my biggest question right now. Then I need to figure out how to detect the CSS class the template is currently rendering in. DesertPipeline (talk) 06:37, 16 August 2021 (UTC)

Re pb, visit the link to see the documentation. At Special:ExpandTemplates put "A" in "Context title" and "{{pb}}" in "Input wikitext" then press OK. That will show what that template would expand to if used at A.

This is the right page for Lua help. I recommended VPT because before attacking a problem with Lua, it would be desirable to see if there is a better approach. The people there know all about css and would be interested in fixing a vertical alignment problem.

A module generally returns a function which is called by a template or directly with #invoke. The function returns a string of wikitext which replaces the template or #invoke where the module was called. Unless the module does something clever, any template returned in the wikitext will not be expanded—the module is supposed to return simple wikitext. People generally find an example and try to understand it. I can only find complex examples at the moment and don't think the recommended Module:Example is very helpful. You might try examining Template:Football manager history and Module:Football manager history (while ignoring the details of the logic as irrelevant). A module has no knowledge of where it is being expanded and you won't be able to detect the css class. Johnuniq (talk) 07:06, 16 August 2021 (UTC)

User:Johnuniq: It's not vertical alignment that I'm trying to change; it's the left margin of all paragraphs after the first, so they all line up with each other in the reference list (as if there were a "[ref num] ^" beside each paragraph). Since you say I can't detect what the template is inside with Lua, are there any ways I can achieve what I'd like to do? DesertPipeline (talk) 07:54, 16 August 2021 (UTC)

That's why I keep saying to ask at VPT. Johnuniq (talk) 09:23, 16 August 2021 (UTC)

I'm not sure I understand what is wanted properly, but it seems like <templatestyles> might provide a solution. Move the styling in the template, i.e. style=margin-left:1em;, to Template:Multiref/styles.css and qualify the indent according to the container. — Jts1882 | talk 11:03, 16 August 2021 (UTC)

User:Jts1882: Interesting. Do you know how I "qualify the indent according to the container", though? I'm not sure how to do that. DesertPipeline (talk) 11:13, 16 August 2021 (UTC)

Wait, hold on. Won't that apply the style to the entire template output? I need every paragraph except the first to be indented. Might I need to use a Lua module still for doing that, as well as using templatestyles? DesertPipeline (talk) 11:15, 16 August 2021 (UTC)

I've added templatestyles for the styling to {{Multiref}}, along with an extra div element with class multiref. See {{Multiref/styles.css}}. It looks like it does what you want (indent in the reference list, no indent in the popup). It might be better to have two different indents. — Jts1882 | talk 11:54, 16 August 2021 (UTC)

User:Jts1882: Let's test this in Template:Multiref/sandbox2 for now. I've moved that there; I also changed div to span, as div caused a problem with the linebreaks. However, it seems the margin thing isn't working. Any idea why? DesertPipeline (talk) 12:24, 16 August 2021 (UTC)

PageInfo

hi all.

i expected pageinfo to be available to lua, but could not find anything in the docs.

looked mainly in the "title" object, which is where i expected to find it.

USE CASE:

i was asked to help with creating a template for templates' documentation page, which will list the WD entities used by it.

this data appears in PageInfo, and i thought it's the right place to get it, but i can't figure out how to get pageinfo.

any suggestions?

peace קיפודנחש (aka kipod) (talk) 23:39, 14 September 2021 (UTC)

I don't think Lua can get this but I don't know. A definitive answer might be available at a Wikidata noticeboard. Johnuniq (talk) 23:45, 14 September 2021 (UTC)

I'm not sure exactly what you're trying to do, but you may find something useful in the mw.wikibase extension libraries, documented here. Certes (talk) 00:53, 15 September 2021 (UTC)

i will try to rephrase, in the hope of being clearer:

what i want, given a template, is to list the wikibase entities used by this template. i do not believe wikibase can tell me that, because this list is not a property of any specific data item; it's a property of the template itself.

when clicking "page information" in the template page, the entities/properties/whatever of wikibase which are used by the template are listed. i did not yet verify that this list is indeed exhaustive, and i don't really know how it's "compiled", but this list is better than anything else i can think of.

what i want is getting the content of this list into a "service template" (to distinguish from "the template" discussed here), that can be used in the documentation page of "the template", listing the WB entities/properties/whatever used by "the template".

hope this clarifies a bit.

as far as i understand, this list is generated by mw "parser" when "compiling" the template.

the template is local to a specific project, and i do not believe this information can be retrieved by any mw.wikibase API, and i do not think this question belongs on WD noticeboard.

"pageinfo" page contains other nuggets of information which would be useful, and i was hoping/expecting the whole shebang to be accessible via some title object function/query. peace - קיפודנחש (aka kipod) (talk) 15:31, 15 September 2021 (UTC)

I suspect that the wikidata usage info is the result of some code somewhere at MediaWiki drilling down through templates and sub-templates and modules. For example, {{cite q}} info lists a lot of wikidata stuff but the template itself, has none. When you edit a template, a similar list is available under the 'Wikidata entities used in this page' drop-down (below the edit box). My guess is that the data you seek are not available unless you use some sort of external tool that reads the info page and from that writes something that a module and template can use, or you write something that reads the wikitext of each template and module transcluded into the template of interest and builds a list that way – I can imagine that to be a difficult thing to do if calls into wikidata are created dynamically.

—Trappist the monk (talk) 16:11, 15 September 2021 (UTC)

Module help request

You are invited to join the discussion at Wikipedia:Village pump (technical) § Ref parameter. {{u|Sdkb}} ^talk 07:57, 1 November 2021 (UTC)

is there any way to discard loaded data?

There are 87,000+ taxonomy templates. Module:Autotaxobox walks through those templates from an arbitrary starting point and produces a taxonomic list from which that starting point descends. I've been wondering about replacing those 87,000+ templates with a series of data modules primarily so that other wikipedias don't have to import (and maintain) 87,000+ templates. But...

The data from those 87,000+ templates would require a 12.5MB data module; much too large (I think that there is a 2MB limit). So, to make things more manageable, I separated the data into separate data modules organized alphabetically by first letter of the taxon name. I further split lists that have more than 2500 taxon names into sublists of 1000 taxon names.

I have written a sandbox module that walks through the module data set in a way that mimics Module:Autotaxobox. You can see the result at Module talk:Sandbox/trappist the monk/taxonomy. My list is the left-hand list; the pretty table is from Module:Autotaxobox.

As my code walks the linked list of taxon names, it loads data using mw.loadData() as necessary to get the next taxon name data. If you preview Module talk:Sandbox/trappist the monk/taxonomy and open the lua logs dropdown in Parser profiling data, you can see which modules were loaded. The list also shows the number of times that a particular data module was needed. Most are loaded and used for only one taxon name. After that use the loaded module continues to needlessly consume a rather large chunk of memory which could be better used for other things.

Is there any way to discard (unload) a loaded data module? I tried using require() without much noticeable gain.

—Trappist the monk (talk) 15:07, 16 October 2021 (UTC)

Just verifying, when you say that require() wasn't a noticeable gain, how did you set it up?

local p = {}
function p.get_list()
	return list{...}
end
return p
:

local p = {}
local list = {...}
function p.get_taxon(taxon)
	return list[taxon]
end
return p
:

I'm asking to know if the second option wasn't a help as it would seem it doesn't load the complete list, but I haven't tested it out. Gonnym (talk) 15:35, 16 October 2021 (UTC)

mw.loadData() statements at Module:Sandbox/trappist the monk/taxonomy lines 176 and 191 changed to:

ok, taxonomy_t = pcall (require, 'Module:Sandbox/trappist the monk/taxonomy ' .. suffix);

The whole data module is loaded.

Lua memory usage for Module talk:Sandbox/trappist the monk/taxonomy:

using require(): 22,996,176/52,428,800 bytes

using mw.loadData(): 25,679,112/52,428,800 bytes

—Trappist the monk (talk) 15:56, 16 October 2021 (UTC)

Ok, so in your change the require still loads the complete list which it doesn't need. I'll try and free time tomorrow to create a working example with the above code and see if it changes anything when only accessing the single taxon. Gonnym (talk) 16:06, 16 October 2021 (UTC)

It does. Pretty sure that's a requirement. I don't think that there is a way to get a table entry from a table that hasn't been loaded ...

—Trappist the monk (talk) 16:31, 16 October 2021 (UTC)

The question (for me) is what gets loaded when you require a module but retrieve only one entry vs loading a complete table. If you already know the answer and you think it's a waste of time then I'll drop that. Gonnym (talk) 16:44, 16 October 2021 (UTC)

I'm not sure that we're communicating. For the example at Module talk:Sandbox/trappist the monk/taxonomy, the first thing that Module:Sandbox/trappist the monk/taxonomy needs is the 'Felis' taxon data. That taxon's data are located at line 240 in Module:Sandbox/trappist the monk/taxonomy F. To get those data, Module:Sandbox/trappist the monk/taxonomy must read the whole of Module:Sandbox/trappist the monk/taxonomy F so that it can have access to the individual items in:

['Felis']={rank='genus', parent='Felinae', link='Felis'}

To walk the taxonomy tree, Module:Sandbox/trappist the monk/taxonomy needs the values assigned to rank and to parent.

—Trappist the monk (talk) 17:18, 16 October 2021 (UTC)

I might look at this in more detail later but here are some initial thoughts. First, if a module will probably only be used once on a page, use require, not loadData. The latter has quite a lot of overhead that is only useful if the data is loaded at least three or four times while the page is rendered. Second, if the question concerns whether memory can be freed, the answer is yes—set the variable concerned to nil. I don't know if it's allowed in Scribunto or whether it would be useful, but the Lua way to unload a loaded module is to execute package.loaded[m] = nil ~~where m is the result of m = require('example')~~. Finally, it might be worth investigating whether a structured data table at Commons could efficiently handle 12.5MB—example available on request. BTW, I believe the 2MB limit is the accumulated total of how much wikitext is transcluded into the page. A module has a 50MB limit of how much memory it can occupy at any instant. Johnuniq (talk) 23:01, 16 October 2021 (UTC)

Ding! Ding! Ding! Ding! require() and package.loaded[<module name>]=nil for the win! Memory use for the example at Module talk:Sandbox/trappist the monk/taxonomy was 22,968,874 now is 13,669,700! And, Module:Sandbox/trappist the monk/taxonomy can now render Template:Taxonomy key/testcases without running out of memory (26,958,802/52,428,800 bytes) – I did not publish the version of Module:Autotaxobox/sandbox that uses my sandbox.

The 2MB limit I was talking about was the wikitext limit. Yeah, if you know ahead of time that you will be needing a particular module multiple times then you should use mw.loadData(). But, for this use-case, because the module has to crawl along a linked list and it doesn't know what module it will need next until it gets to the next node, the time expense incurred by reloading a previously loaded module is an acceptable trade-off.

Structured data tables at Commons are on my list of things to investigate; just haven't got there yet... I expect that such a table will be rather larger than the lua wikitext tables because JSON...

Thank you.

—Trappist the monk (talk) 00:02, 17 October 2021 (UTC)

If you later want an example of using structured data, see User:Johnuniq/index#NUMBEROF. The advantage of that is that all the data is kept at a central location and never has to be duplicated at different Wikipedias. However, now that I think about it, structured data might be unhelpful because each table at Commons has to be loaded as a Lua table by a module and then analyzed line-by-line. I haven't looked at what you're trying to do yet but my first thoughts would be that you need an index table that might be periodically generated by a maintenance module. Perhaps the index could speed up knowing what to load. While I'm reminiscing, I have to mention that I once experimented with a suggestion on a mailing list by Tim Starling. As he predicted, it turns out that it is much faster to store a large amount of data in a huge string, then use a binary search to find what is needed. That is super ugly compared with what we do, namely have a large Lua table with a key that virtually instantly finds wanted items. However, simply loading the large Lua table into Lua takes a lot of time and overhead and it is much faster to do a binary search of one huge string, then parse the result. Actually just doing a string.find would probably be fast enough but I didn't try that. Johnuniq (talk) 02:24, 17 October 2021 (UTC)

Ah, on looking again I see that you are correct—m is the name of the module, say 'Module:Example'. Johnuniq (talk) 05:44, 17 October 2021 (UTC)

@Trappist the monk: I realize I am a bit late to this conversation but I believe you are trying incorrectly re-architech WP:TX.

I cannot say I know much about the WP:TX system (or taxons themselves really) but my understanding is, it is a large collection of data that is represented as a hierarchical graph that needs to be able to be traversed quickly (i.e., during the rendering of article pages). Currently the nodes in this graph are stored as a tree of templates under Special:PrefixIndex/Template:Taxonomy/ and using the parameter |machine code= and set of templates starting at Special:PrefixIndex/Template:Don't edit this line to fetch specific fields. In 2016, this system was updated to allow for certain optimizations in the graph traversal using Module:Autotaxobox to shortcut and simplify some things but it still uses the same overall system expanding templates from Scribunto via expandTemplate on the Template:Taxonomy/ hierarchy.

It looks like you have been working on a Scribunto Lua module replacement for this at Special:PrefixIndex/Module:Sandbox/trappist the monk/taxonomy. Moving the taxon graph data out of the wikitext templating system us a good idea, however, I do not believe your pseudo-alphabetical grouping of the nodes is a good idea. Such groupings just causes issues for editors/maintainers of the data (e.g., I would not call Module:Sandbox/trappist the monk/taxonomy F editor friendly) and as you have seen causes memory and speed issues for traversals of the data nodes (you shorten the traversal depth some at the high cost of much larger nodes because you are grouping multiple original nodes by arbitrary pseudo-alphabetization).

In addressing portability, using Tabular Data either directly from Scribunto or via something like Module:Tabular data is good idea but Wikidata is probably a better idea and it already has some of the groundwork set with taxon name (P225), taxon rank (P105) and parent taxon (P171). E.g. Felis (Q228283) already has statements making claims with such properties. This allows one to create a Scribunto module to traverse these taxon Wikidata item nodes. The real issue is creating all the tooling to allow the complete system to be migrated and then ensuring all the data gets migrated.

—Uzume (talk) 19:02, 13 November 2021 (UTC)

I'm pretty sure I understand what the current automatic taxobox system does. In my view, the current system's fatal flaw is that it is not now, nor will it ever be, easily shareable among various wikipedias. We cannot expect other wikipedias to import thousands upon thousands of little taxonomy templates. That will just never happen. So, the lua implementation is one way that the data might be made available to those other wikipedias for a relatively low cost – importing a couple of hundred data modules is much easier than importing tens of thousands of templates.

Module:Sandbox/trappist the monk/taxonomy is not a replacement for Module:Autotaxobox. It is a tool that I wrote to explore how to fetch data from the compiled data modules and, using those data, to walk the taxonomy hierarchy from a named taxon to the root (Life, Veterovata, or Ichnosdata).

[You] do not believe [my] pseudo-alphabetical grouping of the nodes is a good idea. Suggest a better way to assemble the data. The memory issue is resolved and speed was never really an issue. Module talk:Sandbox/trappist the monk/taxonomy renders in less than 1.5 seconds and is doing much more than typical taxoboxen require.

I have not experimented with tabular data. I have experimented with wikidata. The groundwork that you mention is, I think, not really there. The automatic taxobox system does not need nor does it want most of the properties associated with wikidata's taxonomic QIDs and, similarly, those wikidata taxonomic QIDs don't want or need the dedicated automatic taxobox system properties. I agree that wikidata is the best solution for shareability and for easy creation and maintenance of nodes and their properties. Wikidata is certainly best for internationalization; each taxonomy node wants to link to a local article. At en.wiki, for example, Template:Taxonomy/Angiosperms links to Flowering plant. That's fine for en.wiki but will not work for other wikipedias; see Angiosperms (Q25314). Alas, the wikidata option is apparently not acceptable to the editors at wikidata; see d:Wikidata:Project_chat/Archive/2021/10#is this possible?

It is very likely that my experiment is dead because there is insufficient interest locally (and especially at wikidata) in making the automatic taxobox system data available to all wikipedias.

—Trappist the monk (talk) 20:24, 13 November 2021 (UTC)

@Trappist the monk: I agree that the current system has numerous issues but its biggest flaw is its lack of portability (and thereby its maintainability across all would be users of this data) despite its (mostly) universal ability to be used (the data is by in large usable across languages but not in its current form). As for Wikidata possibilities, methinks this essay has the most in-depth view: User:Peter coxhead/Wikidata issues. It sounds like someone that knows how the system works, needs to architect and work on a means of integrating this data into Wikidata (and cleaning up the existing taxa and taxon Wikidata records to adhere to this). Your experiment is interesting but I do not think it is the correct means to move forward on this. As to moving the data out of wikitext templates, i.e., into Scribunto Lua modules, tabular JSON at Commons or Wikidata items, I believe it would be best to keep most of the individual records separate. A conceivable interim step would be to locally move all the records into Scribunto modules, however, that does not improve portability much. Condensing the number of records in the way you have might help in copying/moving the data (and in a round about way portability thereby) but makes maintenance considerably worse. Remember, if other wikis start using this data heavily that also means they will be participating in its maintenance. I do not see your experiment helping in that regard. Both compacting the records (like your experiment) and/or copying them to other wikis (as would need to be done to adopt your experiment or verbatim copying the existing wikitext template data) reduces maintainability. The Wikidata part of your experiment is much more interesting. It shows that the indeed the groundwork is there. Does it need lots of work—most definitely. As an example of something that could be done to improve things would be look at the qualifiers on such claims. For example, non-principle taxon records in Wikidata items could be annotated with has characteristic (P1552) or does not have characteristic (P6477) referring to something like optional (Q59864995), non-standard (Q105223969), unusual (Q24238907), notability (Q4993710), topic (Q3993443), or something else (in this way something that emulates |always_display= could be implemented). All such records could also be tagged as part of the autotaxonbox system, etc. (and the module implementing the system could ignore records not so tagged). There are many possibilities for such an architecture (deciding and implementing the best ones is the issue). —Uzume (talk) 23:08, 13 November 2021 (UTC)

Putting a complex system in Wikidata guarantees problems forever with undetected vandalism and misguided edits that give broken/misleading/wrong results in articles. Also, converting Wikidata factoids into wikitext for an article might sound straightforward, but it turns out there are many weird corner cases that end up giving problems. Per my "structured data might be unhelpful" comment above, a table at Commons would require massive parsing each time an edit to an article is previewed/saved—it would have to be tested to be sure but it's likely the overhead would be painful. Having data in a module makes it easily portable and it can be made to work easily for editors. Johnuniq (talk) 01:53, 14 November 2021 (UTC)

I've spent a little time thinking about tabular data for this application. The raw data, in lua-table form taken from the 87,000+ taxonomy templates, is about 13MB. The maximum size of a .tab file is 2MB so the data must be divided amongst multiple .tab files much like the current experiment divides the data amongst about 100 lua data modules. I think that the same general approach can be applied to locate and load a particular taxon's .tab data as is used to locate and load an appropriate lua data module. For this application, it is not necessary, I think, to load the whole .tab data into a new lua table as you do in Module:NUMBEROF/data. This application generally will need only one or a few rows of tabular taxon data from a .tab file so, in the simplest case one could:

local function taxon_data_get (taxon, taxonomy_data_file)	-- returns a sequence table of data for <taxon>; nil else
	local taxodata_t = mw.ext.data.get (taxonomy_data_file)	-- load the tabular data file as a sequence table of sequence tables
	for _, row_t in ipairs (taxodata_t.data) do				-- <row_t> is a sequence table of automatic taxobox data
		if taxon_name == row_t[1] then						-- is this the row we're looking for? <row_t[1]> is a taxon name
			return row_t									-- yep, return the sequence table
		end
	end
end

Assuming that the data are sorted by taxon name, a binary search would be faster.

There are other limitations. string and localized data types are limited to 400 characters (does that include multi-byte Unicode characters?). This length limit can, and regularly is, exceeded by the values assigned to |refs= in the taxonomy templates. This limitation alone may be sufficient to exclude tabular data from consideration.

JSON data structures are notoriously delicate which is why, I suspect, that the TemplateData editor exists; too easy to break. That means that there must be some sort of similar editing tool for tabular taxonomy data. As I understand it, nothing of the sort exists except for third-party csv-to-JSON tools that require editors to 'edit' the entire .tab file to fix a typo. The only things that should be editable by the general editing population are the data in the data rows. Editors should be able to add and remove rows or change the content of rows but should never have access to the structure outside of "data": [].

Yeah, wikidata is subject to vandals but so are the 87,000+ taxonomy templates; I suspect that those templates are mostly unwatched so undetected vandalism and misguided edits that give broken/misleading/wrong results in articles is just as much a problem here. Granted, at wikidata, the vandalism breaks multiple wikis ... Though I probably did not say it well, one of the reasons that I argued for an isolate set of QIDs and associated properties for the automatic taxobox system was simplicity which can go a long way to avoiding those weird corner cases that end up giving problems.

Alas, the lua data module solution results in unsynchronized data wiki-to-wiki and while marginally portable, does not support true internationalization as tabular data and wikidata do.

—Trappist the monk (talk) 20:58, 14 November 2021 (UTC)

I think the experiment has been useful. Storing the data in submodules has advantages (especially portability), but how to make them easily editable by users unfamiliar with Lua is a major stumbling block. The flexibility of the template-based automated on English Wikipedia is hard to replicate, but most Wikipedias don't have similar numbers of editors, so ways of making the system available to other Wikipedias are worth exploring. One can only find the solution by testing various ideas.

Several Wikipedias do use Wikidata sourced taxoboxes. I know they are used on the Hebrew and Catalan Wikipedias. There was a prototype version on English Wikipedia at Module:Taxobox, which got deleted only this month. It no longer worked, possibly due to Wikibase changes, but I believe the version on the Hebrew Wikipedia is a development of this. There is an archived discussion at Taxobox 2013.

A problem with Wikidata is the taxa often have several parent taxa and these are rarely sourced. One can imagine a system where the parent is referenced by a particular system (e.g. MSW3, AGPIV) and the appropriate parent could be followed by checking the reference. Adding some sort of qualifier to guide the automated taxobox system "should" also be possible. Often the parents include an immediate parent and a higher level one and following the path of the immediate one reaches the higher level parent. It might be possible to follow multiple paths and use the information to check for the best choice of parent. With the will, there ought to be a way. — Jts1882 | talk 09:51, 15 November 2021 (UTC)

Continuing with that thought, maybe the next part of the overall solution would be to have a bot generate a list of issues based on problems identified at User:Peter coxhead/Wikidata issues (if even possible), such as places where "instance of taxon" should be a synonym. Then work can be done on fixing this issues. When that work is done, Wikidata can then be reexamined to see if it can work. Gonnym (talk) 10:53, 15 November 2021 (UTC)

Template:Chembox

Hello! I've recently started a discussion in here about a, in my opinion, needed change in regard to the way {{chembox}} is designed. It's one of the few infoboxes de facto that doesn't follow the naming convention and it's made up by tens of subtemplates to make way for a needed modular function. It was suggested to me that things could be much easier if it was to be rewritten in Lua modules but my knowledge on Lua is limited to small localized string manipulations so I'm in no way near the needed level to be up for the task myself. I thought about asking for help here and maybe some people could be interested to join the discussion there themselves to see the reasons for the change and the details that are being discussed. - Klein Muçi (talk) 12:35, 19 November 2021 (UTC)

I see there is a discussion at Template talk:Chembox#Lua rewrite. Before contemplating the technical details, it would be desirable to have a clear indication that there is a problem. I see links (example) to old comments suggesting that previous attempts were rushed and based on the "not my problem—someone else will fix any defects" principle. That needs to be resolved. Johnuniq (talk) 22:51, 19 November 2021 (UTC)

@Johnuniq, well that was sort of the point of the notification here: To have more people, especially from a technical background, join the discussion there to state their thoughts on the matter. It wasn't about getting to work immediately on the "problem". :P - Klein Muçi (talk) 22:57, 19 November 2021 (UTC)

Are there any links to articles showing incorrect or badly formatted information? Is there a link to a discussion where someone has shown a demonstrated need to add or remove something but was unable to due to limitations of the current system? If these links are not available, it's likely there is no actual problem. If that is the case, my opinion is that outsiders should not seek to impose a solution on a functioning wikiproject as there is a need for collaboration in order to maintain editors who maintain articles. Johnuniq (talk) 23:14, 19 November 2021 (UTC)

@Johnuniq, uhm, I'm sorry but who are you referring to as "the outsiders" in this case? - Klein Muçi (talk) 23:43, 19 November 2021 (UTC)

Me, and possibly other regulars here. Johnuniq (talk) 00:14, 20 November 2021 (UTC)

@Johnuniq, okay, that makes more sense. I was preoccupied that might have been directed to me because I'd find the term insulting and in contrast with the openness atmosphere that Wikipedia values if I was being called an "outsider", and hence the owner of an opinion of lesser value, because I wasn't that much active on the said project before.

Returning on the point, even though I was hoping for all discussions in regard to the topic to be held in one place, I'm again re-stating my reasoning that the main point that brought me to open the discussion were the difficulties that that kind of subtemplate design brings to the internationalization of the said template. Having to hunt down and import tens of templates is a work that takes days to accomplish, let alone deal with the strings localization. This is the main reason because, if you check my contributions on EnWiki, you'll see that all my work here usually revolves on small technical matters in regard to internationalization aspects, usually in regard to small wikis (more often than not to SqWiki which is my homewiki but to other languages as well, lately LaWiki). Other than that, again, as I've already stated there, I thought that having 1 module instead of 100 templates would be considered better apriori because it would make maintenance and update easier, the reason why, in general, multiple subtemplate design has fallen out of favor lately when compared with Lua modular design. One of course can say that i18n matters are of no concern to one project and when confronted with that, I'd have nothing to add anymore but my general belief is that if we can help others while not making things worse for ourselves we should generally do that. Klein Muçi (talk) 00:39, 20 November 2021 (UTC)

Yes, internationalization is worthwhile. Johnuniq (talk) 00:44, 20 November 2021 (UTC)

Module:Redirect template

Hello. Quick question. Where is Module:Redirect template#L-63 getting its data from? I am trying to get a list of what "R to"/"R from" templates can be used in what namespaces (e.g. {{R to acronym}} can only be used in mainspace), and I think the code I linked above is getting that data from somewhere. This is the code that decides whether or not to display {{Incorrect redirect template}}, so it must know somehow. Thanks. –Novem Linguae (talk) 03:08, 20 November 2021 (UTC)

I had a quick look at Module:Redirect template and it does not appear to have any special knowledge. Editing {{R to acronym}} shows it has |main category=Redirects to acronyms. That uses line 10 of the module which causes or v[1] == namespace to be true in line 58. That gives the result. In other words, the calling template identifies what namespace is suitable. Johnuniq (talk) 04:15, 20 November 2021 (UTC)

Tools for Lua

I started my first Lua module tonight. I was wondering if you guys had recommendations for 1) a step debugger, and 2) a RegEx site that checks Lua-flavored RegEx, similar to regex101.com. I googled for both. I didn't find a RegEx site, and I tried two step debuggers in VS Code but ran into various problems. Also happy to hear any pro tips in general. Thanks. –Novem Linguae (talk) 12:54, 26 November 2021 (UTC)

ZeroBrane Studio is good for trying Lua on a local computer. Problems: First, the Lua used here is version 5.1 with some modifications that provide a couple of features from later versions. That's not a big deal but it means that some newer features do not work here. Second, pretty soon you want to use mw functions and they won't work on your local computer unless you do a bunch of heroic things that take far too long. If you ever debugged primitive Basic programs using print you will be right at home using mw.log (see mw:Extension:Scribunto/Lua reference manual). I'm pretty sure there is nothing that checks Lua's unique regex. Johnuniq (talk) 00:56, 27 November 2021 (UTC)

There is https://gitspartv.github.io/lua-patterns/ which explains the contents of a Lua pattern, but unfortunately it can't check them against test strings. — Mr. Stradivarius ^{♪ talk ♪} 01:52, 27 November 2021 (UTC)

This looks promising for Lua RegEx. Thanks! –Novem Linguae (talk) 01:55, 27 November 2021 (UTC)

Excellent. I got that working right away, thank you. If anybody else wants to step debug in ZeroBrane Studio, I just pasted my Wikipedia lua code into a new file, commented out the return p at the bottom, then added

frame = {args = {'input 1', 'input 2', 'etc'}}
print(p.functionToCall(frame))

–Novem Linguae (talk) 01:52, 27 November 2021 (UTC)

If you use IntelliJ, you could use its lua plugin to avoid the overhead of a whole another IDE on your system. I haven't used this much, though. – SD0001 (talk) 03:12, 27 November 2021 (UTC)

Luanalysis is another good option for IntelliJ. Gonnym (talk) 11:18, 27 November 2021 (UTC)

Just now I documented all these answers at Help:Lua debugging#Off-wiki tools. –Novem Linguae (talk) 10:19, 27 November 2021 (UTC)

You might also find

useful. ~ Tom.Reding (talk ⋅dgaf) 13:07, 27 November 2021 (UTC)

I used lua 5.1 on local machines without doing heroic things to emulate mw.xxx libs. Typically you need relatively small number of the plethora of mw calls in one module, and you do not have to emulate full functionality. I had a piece of code that looked roughly like so:

if not mw then mw = { ['text'] = { -- here come the 2 or 3 i actually use, e.g trim }, -- etc. etc. } end

This scaffolding lets you test your code on wp with no change.

I am not familiar with a debuuging framework that allows stepping/breakpoints, and for lua modules i did not miss it that much. Print statements (on wp use mw.log/logobject, and "emulate" as described above) were usually enough.

It is useful to be familiar with "preview a page using this module" which allows you to test your code on wp without saving every change, and when the develooment encompasses several modules and templare, with special:TemplateSandbox. Peace - קיפודנחש (aka kipod) (talk) 04:12, 28 November 2021 (UTC)

Notepad++

FWIW, the new release of Notepad++ (v8.1.9.3) has added "Lua function list capacity". Andy Mabbett (Pigsonthewing); Talk to Andy; Andy's edits 15:07, 14 December 2021 (UTC)

Template:Chembox has an RFC

Template:Chembox has an RFC for possible consensus. A discussion is taking place. If you would like to participate in the discussion, you are invited to add your comments on the discussion page. Thank you. DePiep (talk) 09:49, 16 December 2021 (UTC)

How to Access Foreign Modules

How do I access de:Modul:Hiero/Cartouche in here to make use of it in an English template without a fork? --Vollbracht (talk) 16:54, 1 January 2022 (UTC)

Alas, you don't. Somewhere there is discussion about someday creating a commons-like repository of templates and modules. I don't recall where I have seen discussion about that but I seem to remember that there are significant hurdles to be overcome.

—Trappist the monk (talk) 16:59, 1 January 2022 (UTC)

phab:T121470, phab:T41610. – SD0001 (talk) 09:30, 2 January 2022 (UTC)

Added to script error category, but not displaying "script error"?

I've imported some missing dependencies of a lua template from commons, but pages using the template are added to the lua script errors category, however the "script error" text isn't displayed in those pages so I'm not sure how to debug without the error message displaying... I checked Help:Lua_debugging but that doesn't seem to mention this happens- Ideas? Page with error is Category:Ichneumonidae and template is Template:Wikidata_Infobox (but the error is likely in the recently imported dependencies i.e. Module:Taxontree! -Mvolz (talk) 13:25, 27 December 2022 (UTC)

I might not have time to investigate so I'll dump the following while I can. Looking at the HTML source and searching for "Scribunto" allows the following to be extracted:

Lua error in Module:Wikidata4Bio at line 1997: attempt to concatenate a nil value.

mw.lua:333: in function "expandTemplate"

Module:Wikidata_Infobox:596: in function "x"

Module:Wikidata_Infobox:1518: in function "getBodyContent"

Module:Wikidata_Infobox:1533: in function "body"

Module:Wikidata4Bio needs refactoring. It currently has 66 global variables which is 66 too many. At any rate, the current problem is that something in line 1997 is nil. Johnuniq (talk) 22:30, 27 December 2022 (UTC)

The problem appears to be function getAdditionalCategoryAboutWikidata() at line 1935. That function might return an empty string or nil. It is returning nil in this case. I can't work out (in the minute I have) what the purpose of the function is. In fact, it looks as if it is not complete. It should always return a string. Johnuniq (talk) 07:01, 28 December 2022 (UTC)

I have forced getAdditionalCategoryAboutWikidata() to return empty string at Module:Wikidata4Bio line 1960. That got rid of the script error but added Category:Biology pages with wikidata item specified in VN to Category:Ichneumonidae (apparently from line 1619 or 1621).

getAdditionalCategoryAboutWikidata() is clearly incomplete; all of the if tests return only empty strings; the function tests for the presence of wikibase but then does nothing with wikibase. Perhaps this module should be returned to commons and left there until it has been completed because it certainly isn't ready for use here.

The original author hasn't been around commons since September 2022 (or at en.wiki since 2010) so it is unlikely that help will be available there.

—Trappist the monk (talk) 14:34, 28 December 2022 (UTC)

See Wikipedia:Templates for discussion/Log/2022 December 27#Wikidata Infobox and dependencies. Johnuniq (talk) 22:53, 28 December 2022 (UTC)

List of Acts of the 3rd Session of the 44th Parliament of the United Kingdom

any idea why there are now script errors in List of Acts of the 3rd Session of the 44th Parliament of the United Kingdom and a bunch more similar cases in Category:Pages with script errors? I am not spotting any recent changes? Frietjes (talk) 01:05, 10 February 2023 (UTC)

This is new to me. I made the po variable local (line 147) and that quashed one error message when I previewed List of Acts of the 3rd Session of the 44th Parliament of the United Kingdom. That is, I think a good thing, but likely will break a bunch code that relied on the 'traditional' global-ness of lua variables...

—Trappist the monk (talk) 01:25, 10 February 2023 (UTC)

I'm afraid there are twelve globals (args frame o os po poIsLongTitle poIsShortTitle poLongTitle poNote1 poNote2 poYearOffset s) and it's unclear how the p.whatever functions are supposed to work (are they actually local to the module or are they intended to be called from elsewhere?). I started looking at cleaning but it needs some deep thought. The actual problem is due to recent edits at Module:Roman by Uzume which introduced require[[strict]] and that causes Module:Legislationuk to foul up when it writes to one of its globals. Thanks for the fix Uzume—I'm not blaming you, I'm just saying what led to the issue reported here. By the way, please use require('strict') which is simple and understandable whereas require[[strict]] uses two Lua tricks that make the code unclear and inconsistent. Johnuniq (talk) 04:22, 10 February 2023 (UTC)

I cannot speak to the assertion "a bunch more similar cases in Category:Pages with script errors" by Frietjes (haven't there always been many?; currently there are only three in article space and they are all for resource reasons), however, I helped rectify Module:Legislationuk (ug, I think I know some third-graders that could write better code; I do not mean to slight Theknightwho as he did something I didn't but I wish the code was of considerably better quality). That said, I agree with Johnuniq "it's unclear how...[it's] supposed to work". Neither the module nor Template:Legislationuk provide any documentation and there seem to be obvious bugs (aside from poor/mistaken use of global variables). —Uzume (talk) 06:19, 10 February 2023 (UTC)

until this recent fix there were about 30 articles in the category, but now we are back down to only four. thank you Trappist the monk for fixing it. Frietjes (talk) 15:52, 10 February 2023 (UTC)

Ummm... credit for the fix goes to Editor Uzume...

—Trappist the monk (talk) 16:08, 10 February 2023 (UTC)

Yesterday I fixed an error in Module:Year in various calendars which was introduced in 2015, so something has caused these errors to start being recognised as errors — Martin (MSGJ · talk) 17:16, 10 February 2023 (UTC)

require('strict') like the former Module:No globals (see also Module:No globals (Q16748603)) "poisons" the global variable tables with metamethods to prevent unintentionally mistaken/poor global variable usage (i.e., it becomes an error to write to a global or read from an undefined global). Module:Year in various calendars requires Module:Roman which in turn now requires strict (as of 1138265559), thus (at least until Martin fixed it) the global metamethods flagged the problematic usages of addtext in that module raising errors whereas they were ignored previously. The fact that adding strict to Module:Roman affects consumers of Module:Roman underscores the problems of global variables to begin with. Just imagine debugging an issue where Module:Roman inadvertently also used the global variable addtext (it would be extremely difficult find and fix such an issue). —Uzume (talk) 18:34, 10 February 2023 (UTC)

I'm happy to do a complete rewrite of this to make it more intuitive to use (and maintain). I'm much, much more experienced with Lua now than when I first wrote this (which was my very first coding project in any language!). I'm also well-aware that it's pretty crap, because I had absolutely no idea what I was doing. The whole thing needs to be rewritten from the ground up - preferably using a constructor. Theknightwho (talk) 08:42, 11 February 2023 (UTC)

@Uzume I've started a complete rewrite at Module:Legislationuk/new. I'm afraid that I'm not very familiar with Wikipedia's Lua conventions (as I spend most of my time editing Wiktionary these days), so please do let me know if anything's amiss. Theknightwho (talk) 14:27, 11 February 2023 (UTC)

I am pleased to hear that though I wonder why you didn't just name it Module:Legislationuk/sandbox. I like that you have decided to use mw.html vs. attempting to generate HTML tags by hand with string concatenation. I am concerned why you are trying to parse wikitext table markup though (your comment refers to this as "pseudo-template"). If you want to create your own markup (which is not trivial and thus probably not a good idea) I would refrain from using symbols that are already used by wikitext like braces, brackets and pipes. I do not understand why you are just not using parameters and arguments directly and instead seem to be trying to create your own language. I do not really understand what you are trying to accomplish (maybe you should try documenting things before writing them). —Uzume (talk) 16:39, 11 February 2023 (UTC)

@Uzume I wouldn't have made that choice if I were starting from scratch, but given that this affects over 100 pages I felt it was easier to just roll with the format that it's already in. I'm unsure how {{!}} would behave, but I find it very unlikely that it would ever need to be used in this context.

I can only vaguely remember the reason for doing it this way in the first place, but I think it had something to do with hitting the Lua memory limit. This is something we have major issues with on Wiktionary (generally on pages with several thousand invocations), but I suspect in this case it was just down to highly inefficient code - and therefore easily avoidable. Do remember that I did this some time ago, when I knew very little about coding! Theknightwho (talk) 17:30, 11 February 2023 (UTC)

Again, your best bet would be to describe what you are trying to accomplish first and not just letting it grow through feature creep. I frankly believe you might be better just developing this thing as a template without Scribunto Lua (but again I am not sure exactly what you are trying to accomplish yet as there is zero documentation save the somewhat problematic code itself). I strongly urge you to first document how {{legislationuk}} is supposed to work and what it provides to any would be consumers of the template. Then people can work on the solution and others (perhaps with different/better experiences that just you) can also join in and lend a hand (even if they are only with ideas on the best approach, etc.). As an exmaple, you quote current usage but I see (via Special:WhatLinksHere/Template:Legislationuk) that most of the usage is confusing at best. I see things like {{legislationuk|act a bunch of wikitext table markup...}}. That table markup contains pipebars that separate template arguments (so causes problems for processing). And why the leading "act" text that appears to do nothing (I could maybe see |act= with an equals sign to define a parameter name)? —Uzume (talk) 18:40, 11 February 2023 (UTC)

@Uzume The overall goal is to create a systematised way to list acts (or other legislation), in a way that is easy to use and intuitive to understand. For comparison, just look at this list, in the old style.

One useful thing that the template does is automatically produce citations using the data saved at Module:Legislationuk/data. Pre-1963 legislation uses an entirely different system that is based on which regnal years of the monarch that a given parliamentary session spanned. This can get very confusing. That lookup table is designed so that you can input any date (going back to 16-whenever), and it'll tell you what the session citation is. It will also produce automatic URLs to legislation.gov.uk and the Parliamentary Archives (both of which are determined by the citation). What's more, you can also generate the list article introductions automatically in a templatised form, with the relevant bits filled in (e.g. numer of acts passed, type of acts, dates).

However, I agree that the current format is problematic, but I haven't really looked at this in any detail in about 18 months. At the very least, it's a necessary step to translate what we already have into a form that's actually possible to understand. I have no intention to start using it on any further pages until we've worked out what the final format should be, though.

By the way: the leading "act" is indeed redundant. I put it there in case this system gets used for other types of legislation like statutory instruments, as they would inherently need a modified layout. Essentially, a small form of future-proofing. Theknightwho (talk) 19:05, 11 February 2023 (UTC)

@Theknightwho: Now it seems like we are getting somewhere, however, we are also deviating away from Lua and the error reported at List of Acts of the 3rd Session of the 44th Parliament of the United Kingdom which this original discussion was about. So I believe it would be better to move this discussion to a more appropriate forum: Template talk:Legislationuk#Goal —Uzume (talk) 05:26, 12 February 2023 (UTC)

require strict

I mentioned above my humble opinion that require[[strict]] is unclear and inconsistent and should not be used. Instead, use require('strict') which is immediately recognizable and is consistent with convention and the documentation.

@Uzume: You have introduced a few of these lately, most recently diff at Module:Plain text. Are you going to continue using require[[strict]]? I'm raising this here for general opinions on the style. Johnuniq (talk) 01:36, 11 February 2023 (UTC)

Use require ('strict'). I don't know how to decode require[[strict]]. At mw:Extension:Scribunto/Lua reference manual all but two of the require() examples and mentions use the parenthetical form; the two exceptions omit the parentheses: local util = require 'libraryUtil'. There is (apparently) a standard convention for requiring another module; we should use that convention (the two exceptions in the documentation should be tweaked to comply).

—Trappist the monk (talk) 01:58, 11 February 2023 (UTC)

[[strict]] is a string, the same as 'strict' although there are some differences about what the string can contain (irrelevant for simple text like strict). Lua does not require parentheses to call a function if the argument is a string or a table. Those two tricks mean require[[strict]] is the same as require('strict'). Programming works best when everything is consistent so the parens should never be omitted. Johnuniq (talk) 02:49, 11 February 2023 (UTC)

Of course, long brackets, which I routinely use in commenting ... Doh!

And yeah, parens should never be omitted

—Trappist the monk (talk) 03:54, 11 February 2023 (UTC)

Omitting parens is documented even at mw:Extension:Scribunto/Lua reference manual#Function calls where it shows examples: func{ arg1 = exp, arg2 = exp } and func"string". It is mostly applicable when passing a single literal argument.

And there are plenty of examples even in our documentation, e.g., frame:callParserFunction (with frame:callParserFunction{ name = '#tag', args = { 'nowiki', 'some text' } }), frame:expandTemplate (with frame:expandTemplate{ title = title, args = table }), frame:extensionTag (with frame:extensionTag{ name = string, content = string, args = table_or_string }), frame:newChild (with frame:newChild{ title = title, args = table }), frame:preprocess (with frame:preprocess{ text = string }).

You are welcome to replace my code but don't forget: most of these too (many of which are even more esoteric than my simple usage). And that was just with the require function.

Next you are going to tell me I shouldn't use the : to implicitly declare and pass things as self or metamethods in constructions like:

setmetatable({1,2,3},{__index=table}):concat[[_]]:rep(3) -- "1_2_31_2_31_2_3"

Why don't I just stop using . notation when dereferencing tables:

_G['mw']['getCurrentFrame']()['args'] -- mw.getCurrentFrame().args

I am sorry if you do not know the language well and perhaps do not like my style but the point is it is there are reasons for the language features and I personally feel using the long string format to pass a single string literal to require was in fact clearer, especially since we are only using it for its side-effects (on the global tables) and not using the return value.

—Uzume (talk) 16:10, 11 February 2023 (UTC)

Uzume, your replies having data type Next you are going to tell me I shouldn't ... and I am sorry if you do not know the language well do not comply with the OPs. This requires post parsing, which makes it harder to understand your answers obscured by this. Nobody disputed or questioned correctness, you were explicitly and IMO kindly asked about choice and deviation of style. DePiep (talk) 05:42, 12 February 2023 (UTC)

@DePiep: Perhaps my reply was confusing. I apologize. However, I believe I did specify why I felt that style was an improvement as well as showed that both syntax mechanisms I used are already quite prevalent within the body of Scribunto modules present on Wikipedia and Wikimedia at large. —Uzume (talk) 21:39, 12 February 2023 (UTC)

OK, thanks for this reply. A next time I'll be glad to read more from you. DePiep (talk) 05:28, 13 February 2023 (UTC)