Wikipedia:Bots/Requests for approval/WildBot 3

The following discussion is an archived debate. Please do not modify it. To request review of this BRFA, please start a new section at WT:BRFA. The result of the discussion was

Approved.

WildBot 3

Operator: Josh Parris

Automatic or Manually assisted: Automatic

Programming language(s): Python, pywikipedia

Source code available: https://svn.toolserver.org/svnroot/josh/ (revision 6)

Function overview: Add checking of #section anchors for existence to existing bot

Links to relevant discussions (where appropriate): Guideline: Wikipedia:Linking#Checking links as they are created

Edit period(s): Continuous

Estimated number of pages affected: ~~I'd guess less than 5% of new pages have #section links, and perhaps 20% of those would be wrong. At 1000/new pages a day, this would be about 10 edits.~~ Hard figures show: 4% of new pages have #section links, and 32.5% of these are wrong; At 1000/new pages a day, this would be about 13 edits/day.

Exclusion compliant (Y/N): Y, standard in pywikipedia

Already has a bot flag (Y/N): Y

Function details: At the same time as checking new page's wiki markup for links to dab pages, the bot will also check for links containing a #section anchor to ensure the anchor appears on the target page. Normally this is a section heading, but there are techniques available (templates like {{Anchor}} and raw HTML tags) which create an anchor without a ==section==; to detect these cases, the HTML of the target page will be downloaded and searched for these anchors.

WildBot found one or more links in this article with broken #section; for more information on #section links see Wikipedia:Linking#Piped links to sections of articles.
The broken #section links found were: Broadway#Golden years, New York#Histery

Discussion

This is certainly a good idea. @harej 16:22, 17 January 2010 (UTC)[reply]

Isnt there an inline template for this, similar to {{deadlink}} or something? If there is it would certianly be more helpfull. Would you mind telling me why you think this should only be limited to new pages? You could do a dump scan for the whole project. Tim1357 (talk) 16:57, 17 January 2010 (UTC)[reply]

Bandwidth; I just don't have it. Running the bot as is consumes a solid 20% of a bandwidth I have available. Unless I get a Toolserver account, recent changes or a database scan is off the cards. Additionally, I've got plans to make the bot smarter and more helpful, so I don't want to bomb every broken page link in the 'pedia with a mere advisory. Josh Parris 22:20, 17 January 2010 (UTC)[reply]

You could get a Toolserver account if you'd like; it would probably help with the running of your bot and it's not very difficult to get one if you can demonstrate need. @harej 00:34, 18 January 2010 (UTC)[reply]

DaB said he'll look at my application from 29th Dec on Sunday. Today's Sunday in Germany I believe. Or has it just finished? Anyway, WildBot's approval may help things along there. Josh Parris 00:53, 18 January 2010 (UTC)[reply]

I've had a look, there's nothing for inline work. It may be inappropriate to inline too, because the link still kind-of works, it just goes to the target page rather than a part thereof. Josh Parris 00:58, 18 January 2010 (UTC)[reply]

On another note, I would appreciate it if the bot only works only in the main-namespace. Tim1357 (talk) 16:57, 17 January 2010 (UTC)[reply]

I was thinking along these lines but couldn't think of a reason not to check the other namespaces the bot currently patrols. What difficulties do you foresee outside of mainspace? Josh Parris 22:20, 17 January 2010 (UTC)[reply]

Damn, one other thing. It is generaly frowned upon for bots to download the html markup. If I may suggest a more server-friendly version: use http://wiki.riteme.site/w/index.php?title=<title>&action=raw&templates=expand. That solves the problem of the {{Anchor}} template. Tim1357 (talk) 17:06, 17 January 2010 (UTC)[reply]

That's pretty much what I've done; I called the API version (which I'm not sure, having seen your suggestion, is the best idea). Josh Parris 22:20, 17 January 2010 (UTC)[reply]

So in order of questions:

Thats ok, if you only want to do new pages, thats fine. You could look into the toolserver idea if you want.
My reasoning is that there realy is no need for notifications outside of the mainspace. Plus there is no "talk" pages for talk pages, if you know what I mean.
Ok, if you really need to download the html, thats fine. I just thought the templates=expand bit would be helpful; I myself just found out about it. Tim1357 (talk) 00:55, 18 January 2010 (UTC)[reply]

Yes, I prefer your method over my API call. WildBot task 1 doesn't do talk pages, so no probs there. Toolserver account is in process. Josh Parris 01:07, 18 January 2010 (UTC)[reply]

Nice, this gets the thumbs up from me as long as this acts in the manor that the Disambiguation Wildbot does. Tim1357 (talk) 01:10, 18 January 2010 (UTC)[reply]

Approved for trial (50 edits). Please provide a link to the relevant contributions and/or diffs when the trial is complete. @harej 03:30, 23 January 2010 (UTC)[reply]

Coding... adding this functionality has demanded a substantial internal redesign for WildBot, as it's no longer making one edit to a talk page (at least, not internally). The hard figures above were produced by a very rough draft. Josh Parris 22:20, 24 January 2010 (UTC)[reply]

Doing... The trial has commenced, with some preliminary results are available in this a seeded group of #section checking with nine hits. The rest of the results are going to be spread out though the normal run of WildBot. There's code to limit it to 50 #section edits per run. Josh Parris 01:01, 26 January 2010 (UTC)[reply]

Thus far 25 edits have been made, and I've discovered a number of things. Turns out that pywikipedia has code to detect valid section references - but it doesn't work correctly when there's markup; the common case being an article link in a section header. People put all kinds of crazy stuff into section headers. I won't bore you with the stories. I seem to have bitten off quite a large, chewy part of the world. The internal re-coding has been shaken-out, so I'll soon be tidying up the code and running that in production. Josh Parris 13:57, 27 January 2010 (UTC)[reply]

33 edits Josh Parris 04:33, 28 January 2010 (UTC)[reply]

Trial complete. I'll be posting links to the edits in a few hours Josh Parris 02:45, 29 January 2010 (UTC) [reply]

Trial Edits

If I might add, this has been terribly buggy. I'm going to be keeping a very close eye on it in its early life, the multitude of problem that turned up during the trial haven't endeared the code to me. Josh Parris 11:48, 29 January 2010 (UTC)[reply]

Approved. Seems good to me. Tim1357 (talk) 00:54, 30 January 2010 (UTC)[reply]

The above discussion is preserved as an archive of the debate. Please do not modify it. To request review of this BRFA, please start a new section at WT:BRFA.