Talk:Index of underwater diving

List size

The list is experiencing problems with rendering. It sometimes runs out of processing time due to the large number of annotated link templates. The apparently obvious solution of using local annotations has two problems:

1. Local annotations will considerably increase code size. Order of double code size. This is a minor problem.
2. Local annotations will require local citation for verifiability. (annotation by link to short description avoids this issue as each short description is required to be verifiable at its source page) This is a major problem in editor effort and maintained over the long term. It will probably also not reduce render time by much as each annotated link template will be replaced by a citation template.

Whatever solution is attempted, will probably be temporary, as the list will grow. It will be necessary to fudge up a table of contents to allow cross page linking if the page is split. Such a ToC should be compatible with the current style of sticky ToC on the left margin. The system used at Glossary of underwater diving terminology may work, so I will be experimenting. · · · Peter Southwood ^(talk): 07:46, 16 June 2024 (UTC)

The Transhumanist, Do you have any suggestions? · · · Peter Southwood ^(talk): 07:46, 16 June 2024 (UTC)

@Pbsouthwood:

Some thoughts, rather than suggestions, are...

Indexes and outlines (which are essentially classified indexes) have been treated as navigation aids since 2001, shortly after Wikipedia came online. Providing citations to verify that each item belongs in the list, that is, that it is actually a topic of the subject named in the title, has not been applied to indexes or outlines, generally speaking, because you can simply determine if a topic belongs via click-through. I believe the same principle applies to short descriptions, and to citations that are included on the target pages, as they can be checked by clicking through.

Note, that short descriptions do not include citations, and it follows that substituted short descriptions using the annotation template shouldn't need to include citations, either.

The page Help:Template limits includes diagnostic help tips for evaluating page performance issues related to template load, and it provides tips for reducing that load, such as splitting the page.

Historically, outline treatment on extensive subjects has followed two paths, one being the splitting of outlines when they become unweildly, into outlines of subtopics; and the second being developing multiple outlines on the subtopics. Geography, for example is a very broad subject, and its outline treatment includes hundreds of outlines (all countries fall under regional geography, for example).

Lists can also be split off from outlines.

As for indexes, they are ideal for splitting. See Index of Japan-related articles, though you don't have to have an index page for each letter of the alphabet, you can use ranges ("A-M" and "N-Z", for example).

I hope these comments help. Let me know if you need further assistance.

Sincerely,    — The Transhumanist   10:45, 16 June 2024 (UTC)

The Transhumanist, thanks for your contribution. I agree with part of it but not with all. I will deal with each item separately for the benefit of others who may read this:

• Note, that short descriptions do not include citations, and it follows that substituted short descriptions using the annotation template shouldn't need to include citations, either. Short descriptions are supported (or should be) by the content of the lead of their article. The lead is required to be supported by citations somewhere in the body text. Therefore the short description should be (as a requirement) supported by cited content somewhere in the associated article. If it is not, it should be changed. I think this is also explained in Wikipedia:Short descriptions
• An annotation to a listed link is a claim about the linked topic, which must be subject to verifiability, like any other claim of fact on Wikipedia. One of the principles of verifiability on Wikipedia is that any claim that has been, or is deemed likely to be challenged, must be cited in the same page, preferably in the same paragraph, though this is not always reasonable, for example the case mentioned of content in the lead, which can be cited in another section of the same article.
• An annotated link transcludes content (the short description) from a page on which it is required to be verifiably cited, to a page in which it is only displayed, without direct verifiability, but the path to verifiability of that specific content is unbroken. The annotated link, to the short description, to the lead section, to the content from which the lead section has been summarised. If a short description is substituted this chain is broken, as the local annotation or the original short description can be changed without this being apparent from the other page, in which case the annotation, a claim about the linked topic, is made without an intact chain of verifiability. Is this acceptable? Opinions probably differ, and I cannot say, but I can reasonably claim that there is no consensus on the matter by the general community of editors that I am aware of. We could test this with an RfC, and maybe we should. I am confident that the logic supporting annotated links via the template is solid, but the use of unlinked local annotations is a grey area if it has not been decided already, and I am not aware of such a discussion and consensus. Maybe you know of one? Until then I plan to avoid the issue and get on with it by keeping annotated links, which automatically upgrade, and require no additional in-list maintenance, which is relatively efficient.
• As I have mentioned, I am looking into splitting the list, and A to M and N to Z are both the obvious and actually the most useful split in this case. The split is inevitable, unless the software is upgraded to handle larger template inclusion limits, which I do not see happening in the near future, though it too is probable also eventually inevitable. I have split large lists in a similar way before, but that was before Vector 2022, and I don't know how best to integrate the ToC. I can make it sort of work, but not seamlessly. Anyway, thanks for your comments, they indicate to me that a split is the way to go for the foreseeable future, and later probably more similar splits. Crosslinking of ToCs will just be what I can manage at the time, and can be improved if anyone can work out a better way. Cheers, · · · Peter Southwood ^(talk): 13:06, 16 June 2024 (UTC)

• Pbsouthwood An annotation to a listed link is a claim about the linked topic, which must be subject to verifiability, like any other claim of fact on Wikipedia. One of the principles of verifiability on Wikipedia is that any claim that has been, or is deemed likely to be challenged, must be cited in the same page, preferably in the same paragraph, though this is not always reasonable, for example the case mentioned of content in the lead, which can be cited in another section of the same article.
  
  Yes, that is the written rule, but in practice, that has generally not been applied to navigation pages such as outlines and indexes. The argument almost always gets shot down in AfDs of those page types, in favor of click-through. Annotations are generally pulled from the leads of the pages linked to, making it a simple matter of clicking on the link to verify the annotation. In cases where facts are included that aren't in the lead of the topic linked to, I would agree that citations should be provided.
  
  The consensus for citations in annotations is reflected in the written rule, and that consensus has not, as far as I know, explicitly considered navigation pages. Thus, the de facto treatment of those has emerged in AfDs.
  
  The main danger to navigation pages such as outlines (and by extension, indexes with annotations) has been the content forking argument (applied to the whole page rather than to individual items in relation to the corresponding root article), which became more relevant to the nominators when annotations were present in the navigation page, making it appear more like an article. Outline of cricket was a victim to the forking misconception. Based on the arguments posed in its AfD, all navigation pages should have been deleted. I've endeavored to deal with that problem by updating WP:CFORK with clarifications, which met zero resistance by the vigilant watchers guarding that page. Consensus applies via WP:EDITCONSENSUS, and is not likely to be allowed to be changed lightly, as the watchers won't accept changes that don't reflect standard practice. Now, the guideline better clarifies unacceptable content forks as pages of the same type about the same thing, preventing pages of different types (Outlines, Glossaries, Indexes, Timelines, etc.) from being deleted per the "forking the root article" argument, as they are now included in the "acceptable content forks" section of the guideline.
  
  Getting back to building indexes... Rather than transcluding the short description, which can be too vague (e.g., "Oxygen is an element", which would be useless in an index of elements) and require rewriting which affects scalability, transcluding the first sentence or two of the linked topic's lead paragraph may be a useful alternative by providing richer context.
  
  I hope these comments help. Sincerely,    — The Transhumanist   12:45, 18 June 2024 (UTC)

The Transhumanist. Transcluding a few sentences from the lead of an article would work well for some articles, but not for all. I have no idea what the proportions would be, but as with short descriptions, some lead sentences are appalling. Fixing them would be good, just like fixing a short description is an improvement of an article. This could be an excellent optional alternative. The biggest downside for me is that I do not know of a simple way to transclude those lead sentences as easily as using {{Annotated link}}. Do you know of a similar method that does not require manual preparation of the selected sentence(s) for transclusion in each article? Cheers, · · · Peter Southwood ^(talk): 17:05, 18 June 2024 (UTC)

More thoughts: {{Transclude lead excerpt}} is close, but uses whole paragraphs, which in many cases will be more than necessary, and often more than desirable. A 500 word annotation could be a bit TL;DR, and annotations in indexes could get out of hand. Putting in <Includeonly> tags leaves the problem that other transclusions of the same lead section may need to use different content. I don't know how that could be managed. A specified portion of the lead would have to be allocated, and marked so that later editors are aware that in may be used for this purpose. Short descriptions are inherently separate and it is fairly widely known that they have specific uses, so less of a problem with unexpected changes there. A dedicated annotation statement, following the concept of the short description would be possible, but might meet resistance from the editing community, and may require action from WMF, which is likely to be simply ignored following tradition if it does not align with their current priorities. It may also be more overhead, which could push the usable list size down even more, just where a larger size may be most needed. Meanwhile, short descritions may not be ideal, but they are easy to use and already exist for the majority of articles, a number which will increase faster the more they are used as annotations, and the quality of which should also improve with more use and more visibility· · · Peter Southwood ^(talk): 07:17, 21 June 2024 (UTC)

I'll look into it.    — The Transhumanist   07:24, 21 June 2024 (UTC)

@Pbsouthwood: Unfortunately, {{Transclude lead excerpt}} can specify paragraphs, but not sentences. Parsing sentences is problematic, due to punctuation ambiguity, for example, an abbreviation (with period) followed by a proper noun (with capital letter) could be parsed as a sentence break. It would take an AI solution to parse such sentences correctly across an entire system, and by the time that does become available, the issue will likely be moot, because the same level of technology could be used to compile entire encyclopedias. (My guess is that we're looking at between 2 and 10 years for this to become reality, but probably somewhere in between.)

As you pointed out, some paragraphs are huge, and would bloat lists they were included in as annotations. Maintaining paragraphs to be acceptable for annotation transclusion would be cumbersome. So, "Transclude lead excerpt" is best reserved for use in lead sections of pages in support of a root subject, where size is not an issue as it is with annotations.

"<Includeonly>" tags are a potential nightmare, are another element requiring monitoring and maintenance, and can create conflicts in content use, such as interfering with the main use case for {{Transclude lead excerpt}}, and may likely be removed by other editors. It's not manageable (because it cannot be standardized due to content use conflict), nor scalable, nor sustainable. It would be easier to write annotations from scratch, which is already too cumbersome.

Having a dedicated annotation could be implemented at the project level, but it would be another system-wide component like short descriptions and categories for the community to maintain, and because it would be highly redundant with short descriptions, would likely face major resistance and/or neglect.

In conclusion, the best option for transcluded annotations, for now, is {{Annotated link}} and its leveraging of short descriptions.

Another option, is to not use annotations at all, as links are supported by pop-ups. The user need only hover the mouse pointer over a link to get a glimpse of what it is. True, that isn't as fast to read and browse as annotations, but it is there by default, which is good, because, it provides a solution until something better is implemented.

Automated annotation insertion, and automated on-the-fly navigation page generation, is likely just around the corner, courtesy of AI.

Just some thoughts. Cheers,    — The Transhumanist   23:42, 24 June 2024 (UTC)

Interesting developments indeed. I think I will stick with {{annotated link}} for WikiProject Underwater diving until something better comes up as it works adequately for my purposes, which is mainly keeping track of project content, improving short descriptions, and for planning new articles and redirects. To a large extent, the index is a by-product, but I tend to discover things I was not looking for while working on it. Cheers, · · · Peter Southwood ^(talk): 09:18, 26 June 2024 (UTC)

P.S.: General-purpose generative AIs, such as perplexity.ai can already produce web directory pages, but for producing an entire web directory without human assistance, they're not quite there yet. But AI assistant agents, such as Auto-GPT, might be...

P.P.S.: As an experiment, I told perplexity.ai (which employs Anthropic's Claude) to "Create a web directory page on underwater diving, with 50 wikipedia links only, and only wikipedia links. Then, add an annotation to each entry.", and it did. For comparison purposes, I had it do another one on blueberries, in case the first one was springboarding off of your work. The results are impressive, giving the impression that much much more capability is on the horizon. It is easy to imagine an application that does this for a list of outlines or indexes, and refreshes them at regular intervals via bot. Auto-GPT has been undergoing rapid development, and it might be able to handle this complexity of assignment already. It's time to take a look at that program again. —TT

I ran the same query on perplexity. I don't know if it always gets the same results for the same query, and I would guess not, but a couple of interesting results:
It found an article I didn't know about, a few that don't exist as articles, but are subtopics that that would be legitimate redirects to sections, and the annotations I checked all appear appropriate. Cheers, · · · Peter Southwood ^(talk): 10:07, 26 June 2024 (UTC)

Time to split again?

Render time is still rather long and getting longer: N to R size is 19135, S to Z is 30706, S alone is 13416, so T to Z would be 17290. A 3-way split looks best balanced. Unless there is better alternative by the time I get round to it I will probably just go with one of these. Similar split is likely with A to M. Cheers, · · · Peter Southwood ^(talk): 07:46, 10 August 2024 (UTC)

Possible split of A to M: A to C is 23661, D and E is 20380, F to K is 17908, L to M is 9395. Maybe move N to this for L to N of 12713, leaving O to R at 15817. · · · Peter Southwood ^(talk): 08:14, 10 August 2024 (UTC)