Wikipedia:Bots/Requests for approval/DrilBot 5

The following discussion is an archived debate. Please do not modify it. To request review of this BRFA, please start a new section at WT:BRFA. The result of the discussion was

Withdrawn by operator.

DrilBot 5

Operator: Drilnoth

Automatic or Manually assisted: Manually assisted (at first), and automatic once I'm more fully sure of pywikipediabot's abilities.

Programming language(s): Python, via Pywikipediabot.

Source code available: Simple regex can be made available, but mostly just standard pywikipedia.

Function overview: Replace templates and template parameters on request.

Links to relevant discussions (where appropriate): I'll check for consensus for an individual task before running it, unless it seems very uncontroversial (as with the example I put forth below).

Edit period(s): When there's something to be done and I have the time to work on it.

Estimated number of pages affected: Varies by task. Example below would be about 375 pages.

Exclusion compliant (Y/N): Yes (I think; pywikipediabot is exclusion by default, right?)

Already has a bot flag (Y/N): Y

Function details: Details of use have been modified; see discussion below Replaces templates and template parameters on request or to clear out uncontroversial backlogs; primarily in the area of WikiProject banners, but also other things (e.g., if all uses of a template need to be renamed because another template is being moved to the previous one's location). Individual tasks which don't seem really uncontroversial (for example, fixing up Category:Dungeons & Dragons articles in need of work force banner consolidation) will be discussed with appropriate WikiProject and/or at the village pump. If a more specific task description is needed for approval, I'd like the workforce banner consolidation approved as the main task, and other WikiProject banner work after that. This is purposefully a simple task; I haven't used pywikipediabot before and don't want to do anything too complicated. Sample edit in bot's userspace: [1]. –Drilnoth (T • C • L) 17:41, 1 December 2009 (UTC)[reply]

Discussion

I'm not willing to say yes beforehand to doing tasks where it requires the bot operator to decide community consensus. There was a lengthy discussion about community consensus because of tasks that appeared to one BAG member to have community consensus when they did not. If the template or a parameter has been deprecated by the community, then having a bot ready to change the templates is fine, but not blanket preapproval, imo. --IP69.226.103.13 (talk) 20:55, 1 December 2009 (UTC)[reply]

So, you feel that I should come here for each separate task? That seems like it would be kind of clunky for everyone. Would it maybe work if I asked for outside admin closure of any such discussions? If that wouldn't work either, I'd be happy enough to have a "deprecated template cleanup bot" as this specific BRFA's task, to remove deprecated templates in accordance with consensus. –Drilnoth (T • C • L) 21:57, 1 December 2009 (UTC)[reply]

I think your last point might work best. It would be a useful bot to have, templates and parameters change all of the time. Every discussion about change in a template that might be changed could be a place where the community would find use for this bot. BAG members may have insight into this discussion that is useful. I'm not that familiar with template policies. I do know there are many, and they change often enough that users are always inserting old versions. It seems to me this could be a very useful and appropriate bot. --IP69.226.103.13 (talk) 22:22, 1 December 2009 (UTC)[reply]

Sounds good; then make this proposal primarily for removing and/or updating deprecated templates and template parameters. –Drilnoth (T • C • L) 01:24, 2 December 2009 (UTC)[reply]

It seems you're planning to use regular expressions for the template replacements. How do you intend to handle nested templates, or templates where something nontrivial needs to be done with the named parameters? Anomie ⚔ 23:22, 1 December 2009 (UTC)[reply]

Good question. Just for reference, here's the py regex that I used for the sample:

        (r'\{\{ProjectGreyhawk[^\}]*\}\}\n?', ''),
        (r'(\{\{D\&D[^\}]*)(\}\}\n?)', r'\1|Greyhawk=yes\2'),

With your points, I realize that this might not be real good. In this specific case, the ProjectGreyhawk template doesn't have any parameters that would affect it anyway, so there was no need to account for that in the removal. The modification of {{D&D}} is also fairly simple, because it is just adding a parameter. However, to account for nested templates as you mention, I might use this (not tested, which I would before use and debug if needed, but it should work):

        (r'\{\{ProjectGreyhawk[^\}]*\}\}\n?', ''),
        (r'(\{\{D\&D[^\}\{]*)((?:\{\{[^\}\{]*\}\}[^\}\{]*)*)?(\}\}\n?)', r'\1\2|Greyhawk=yes\3'),

Could you maybe give me an example of what you mean by nontrivial for template parameters? –Drilnoth (T • C • L) 01:24, 2 December 2009 (UTC)[reply]

I note your replacement could end up with "Greyhawk=yes" in D&D's parameter list twice, if it were already there along with the ProjectGreyhawk banner. Also, your nesting code would only handle {{foo|{{bar}}}} but not {{foo|{{bar|{{baz}}}}}}, should that ever come up; although when dealing with project banners, you normally won't have to deal with any nesting at all. Also note that your code would also add "Greyhawk=yes" to various other templates if they are present on the page being processed; it will miss it if someone uses {{WikiProject Greyhawk}}, {{WikiProject D&D}}, {{Dungeons & Dragons}}, or other redirects to either banner; and it will not operate correctly if a page contains {{ProjectGreyhawk}} but not {{D&D}}. And depending on how you fix that last issue, be sure the {{D&D}} ends up in the appropriate place (e.g. inside a banner shell, if any) and not just tacked on the very top of the talk page.

While this may be drifting a bit off topic, some examples of non-trivial template parameter handling from stuff AnomieBOT has done in the past:

Replace {{Infobox Aircraft}} with {{Infobox Aircraft Begin}} followed by {{Infobox Aircraft Type}}, copying some of the named parameters to the former and some to the latter.
Replace {{SegaProject}} with {{WikiProject Video games|tf=Sega}} (or tf2=Sega if tf=X is already present, or tf3=Sega if both tf=X and tf2=X are already present). Also, if the {{SegaProject}} included "tf=Sonic", add tf=Sonic (or tf2 or tf3) to the WPVG banner as well.
Replace {{WP Shinto}} with {{WikiProject Japan|shinto=yes}}, and the same for {{Jmyth}} with myth=yes, {{Gaijin tarento}} with gaijin=yes, and {{Project Owarai}} with owarai=yes. Also, change "tf=X" or "tf2=X" or "tf3=X" in {{WikiProject Japan}} to "X=yes" for 15 possible values of X, 7 of which need X slightly changed in the process. And do it all in 1 edit per page, to avoid excessive watchlist traffic.
When doing wikiproject tagging, also merge all banners in a {{WPB}}/{{WPBS}} into parameter 1, set blp=yes or activepol=yes on the shell if the {{WPBiography}} has the corresponding parameters set, remove any obsolete "nested=yes" parameter from banners in the shell, and swap between WPB and WPBS based on the guidance at Template:WPBS#Usage and the number of banners inside the shell. And remove any redundant {{blp}} if the shell specifies blp=yes.
Remove "website", "imdb_id", and/or "amg_id" parameters from {{Infobox Film}}, adding {{official}}, {{imdb title}}, and {{amg movie}} (respecively) to the article's External links section if the corresponding link isn't already there (via template or via a normal external link, and keeping in mind that "id=001234" and "id=1234" are equivalent). Be sure to copy the film title from the infobox so the external link doesn't end up saying "Whatever (film)", and don't add useless external links if the field is present but empty or present but contains only a .
Replace {{SFBAProject}} with {{WikiProject California|sfba=yes}}, also copying any "importance" from the former into "sfba-importance" in the latter; same for {{WikiProject Los Angeles}} and "la=yes" and la-importance; automatically do "class=stub" if the page contains a stub template and no class is set; and set the importance based on various categories (e.g. low for Category:Unincorporated communities in California, mid for Category:California ranchos, etc) or the population listed in {{Infobox settlement}} (there are several population fields to check for). And do it all in one edit per page.

Ok, maybe these are unfair to offer as examples ;) There are plenty of simple tasks to go around, I guess my point is just to see if you won't over-simplify a complex task. Anomie ⚔ 03:02, 2 December 2009 (UTC)[reply]

(undent), okay, some sample code:

Once again, I haven't actually tested this code, so no guarantees that it is correct. Before testing any new regex, I check edits manually to make sure that there aren't any bugs.

Taking into account your comments on the basic banner-removal-and-parameter-addition, here's a better proposal. I don't think that uber-nested templates need to be worried about too much in project banners, but one could manually add in more repetition of the subtemplate-capturing code if it seems needed. As it is, it just wouldn't match templates which are uber-nested; it shouldn't screw up the page, just not edit it.

(r'\{\{(?:(?:Wiki)?Project\s*)Greyhawk[^\}]*\}\}\n?', ''), # Remove ProjectGreyhawk template
(r'\|\s*Greyhawk\s*=\s*yes\s*([\|\}])', r'\1'), # Remove |Greyhawk=yes parameter if it appears in a template

# Following split onto multiple lines for readability; it wouldn't be in the final code.
(r'(?:\{\{\s*(?:(?:(?:Wiki)?Project\s*|WP)?D\&D|WPDND|Dungeons\s*\&\s*Dragons)\s*) # Detect template opening and banner name, but do NOT capture them; will be replaced regardless of content
  ((?=\}\|)[^\}\{]* # Ensure that the template name doesn't continue further, and detect everything up to the beginning or end of a template (to be captured in \1 at the end of the next step).
  (?:\{\{[^\}\{]*\}\}[^\}\{]*)*)? # Accounts for any templates which are nested one-level deep and captures them and all content continuing to the end of the banner itself in \1.
  (\}\}\n?)', # Captures banner ending and newline, if any, in \1.
  r'\{\{D&D\1|Greyhawk=yes\2'), # Actual replacement adds parameter and converts template name to the non-redirect form

# Last regex in single-line form
(r'(?:\{\{\s*(?:(?:(?:Wiki)?Project\s*|WP)?D\&D|WPDND|Dungeons\s*\&\s*Dragons)\s*)((?=\}\|)[^\}\{]*(?:\{\{[^\}\{]*\}\}[^\}\{]*)*)?(\}\}\n?)', r'\{\{D&D\1|Greyhawk=yes\2'),

Does not yet account for the possible lack of a D&D banner; I'll need to do a little more Python and regex learning first ^{(adds too mental to-do list)}. I plan to stay away from the more complicated things like you mention until I'm more confident of my regex abilities; as you said, oversimplifying the regex would be a Bad Thing™. –Drilnoth (T • C • L) 17:09, 2 December 2009 (UTC)[reply]

You have unbalanced parens in your regex (an extra ")" just after the \1 group, it looks like), and I'm not sure if removing every "|GreyHawk=yes" parameter is really the best thing to do (in this case it probably doesn't matter, but for some parameter more likely to be used by other projects it would) (also, you forgot a "\s*" after the "yes"), and it might introduce extra page churn in that it would uselessly change an existing {{D&D|Greyhawk=yes|foo=yes}} to {{D&D|foo=yes|Greyhawk=yes}}, but otherwise it looks ok. Any particular reason for the adjacent but separate \1 and \2 groups instead of putting them both in \1?

I'm interested to see the regex for adding a banner, for AnomieBOT I ended up writing a parser that understands just enough wikitext to find templates nested to arbitrary depth and to process the template's parameter list. Anomie ⚔ 21:13, 2 December 2009 (UTC)[reply]

I just modified the code above to account for some of your comments, rather than repeating the whole block again. In this particular case, |Greyhawk=yes is used on very few pages even when this template is used (hence, the reason for a bot :) ), so I wouldn't be too worried about the parameter removal (which could probably be refined to affect only the individual template) or churning. I don't have much time left today, so I'll try to cook up regexes for finding the proper place to put a banner and how to limit the parameter removal to a single template's. –Drilnoth (T • C • L) 22:16, 2 December 2009 (UTC)[reply]

It seems that I won't have time for another week or two to work on this. It can be considered

Withdrawn by operator., and I might reopen it then once I have more time. –Drilnoth (T • C • L) 03:30, 5 December 2009 (UTC)[reply]

Ok, just undo the next edit if/when you come back to it. Anomie ⚔ 04:49, 5 December 2009 (UTC)[reply]

The above discussion is preserved as an archive of the debate. Please do not modify it. To request review of this BRFA, please start a new section at WT:BRFA.