Talk:Linear least squares

Normal equations

The page normal equations redirects here, but this page never quite defines the normal equations in full generality, just for a 2 variable example. This is completely standard stuff and really should be added! 2601:281:8200:36:D6A0:25B0:7C03:7A6E (talk) 05:31, 4 December 2020 (UTC)

I noticed that too. It seems to be lacking a basic introduction to the concept of finding the least squares solution to Ax = b. The-erinaceous-one (talk) 01:41, 19 December 2023 (UTC)

Problem with mathematicians editing Wikipedia

Imagine you're a high school student or undergrad engineer who came to this article to learn how do a least squares calculation so that you can understand it. You want to see some math that simply explains it instead of reading a lot of text. Instead, you're presented only with a linear algebra formulation and statistics involving terms like homoscedasticity in the first paragraph and it doesn't get easier from there. Everyone who does not know what least squares is will find this article useless. This problem is rampant in Wikipedia. It would be a lot better if Mathematicians refrained from editing articles until they make an effort to express things in high school algebra, and not make things so perfectly precise and complete that no one understands it except graduate-level experts editing it, trying to one-up each other. Their proof of superiority to prevent future edits is to make sure no one wanting to make it intelligible is unable to prove the alien language is in error. Lack of edits is not proof it's finally perfectly correct, but proof no one understand it. Ywaz (talk) 11:58, 21 January 2022 (UTC)

I agree that many math-related Wikipedia articles are difficult to understand. Part of the problem is that Wikipedia:Wikipedia is not a textbook. And part of the problem is that writing technical material for audiences at varying levels is difficult and time-consuming. But your complaint stands.

What do you think of the Example section of this article? Is it accessible? What if we moved it earlier in the article? Mgnbar (talk) 12:54, 21 January 2022 (UTC)

I agree with this. Actually, I would propose that ALL linear algebra notation be removed from math articles on non-advanced topics that would be of interest for general users, simply because LA notation is not taught as a course in universities unless you are in STEM fields, that too until you are in 2nd/3rd year, and quite often is not used extensively in practising fields of engineering/CS (aside from R&D/academia). This makes articles in LA notation inaccessible to any student in high school, or from the humanities who would likely have only studied elementary matrix operations, if at all. I understand wikipedia is not a textbook, but it is an encyclopedia, and I remember pouring over hardcover Encyclopedia Britannica back when I was a teenager in school as a good source of information which I could easily follow. 2405:6E00:2F8A:7000:343C:362D:8785:4E87 (talk) 02:52, 21 March 2023 (UTC)

To readers who know linear algebra, linear least squares is vastly simpler in linear algebra notation than in other notation. That's why the Wikipedia:Reliable sources often present linear least squares in linear algebra notation. To remove linear algebra entirely would mean that we do not accurately reflect reliable sources and do not serve those readers well. It's a non-starter.

But this does not mean that the entire article should require linear algebra. Math articles work best when they begin with an elementary treatment, just as you say, and then move on to the harder stuff. So...

What do you think of the Example section of this article? Is it accessible? What if we moved it earlier in the article? Mgnbar (talk) 03:02, 21 March 2023 (UTC)

"To readers who know linear algebra..." - the assumption is that they do, but again, this is a topic of interest for a vast number of people who wouldn't be familiar with LA-notation. I also disagree with your statement that removing LA notation would not accurately reflect reliable sources, because there are plenty of sources including high-school textbooks and statistics books written for the humanities which do not assume a background in linear algebra, but still cover least squares (and many other topics which I indicated should not be in LA-notation). Many, many books cover the topics with just index-notation (which is taught in high-school and non-STEM university mathematics), even(!) tabular calculations. I am not suggesting that tables be employed, just that articles which may be of use and interest to an audience (e.g. likely those with high-school mathematics/non-STEM math university education), be accessible to them based on their background. However, knowing from other discussions on math pages how many orders of magnitude more an article's pedantic accuracy is more important than readability and accessibility for the mathematics contributors and editors, I highly doubt any mathematics-related wikipedia article will be left without LA or set-builder notation - the dearth of them should actually indicate there is a problem because every article ends up reading like a university-level mathematics textbook, despite the claim that wikipedia is not a textbook!

The example is accessible and should be at the beginning of the page. 2405:6E00:2F8A:7000:343C:362D:8785:4E87 (talk) 10:58, 22 March 2023 (UTC)

Thanks for your input (although I think that you've misunderstood some of my statements). I will work on moving the example higher in the text. I will try to craft a presentation that doesn't use linear algebra, before the presentation that does. Mgnbar (talk) 18:38, 22 March 2023 (UTC)

IMO the article will become even less readable if the example is moved further up -- it currently sits well as a self-contained appendix.

Also, it's perhaps worth flagging that most readers looking for a first article on least squares should probably be heading to Least squares and then Ordinary least squares rather than this article, which is more focussed on generalisations of OLS.

It's possible that some of these generalisations could be motivated more pictorially, but linear algebra rapidly becomes essential -- numerical solvers will at the very least require loading up matrices; while understanding the different solution algorithms and their different pros and cons I would think is necessarily going to involve some core linear algebra however you cut it. Jheald (talk) 21:42, 22 March 2023 (UTC)

Starting to clean up the article

I just added an unreferenced section template under the section Linear least squares#Discussion, and as I said in my edit message, I intend to add some refs myself this week.

But reading even the referenced sections, I see there is a lot of repetition and content overlap - sometimes within the same section. It may not be immediately apparent (especially to readers who aren't as familiar with the concept), because although this same concept is studied in Statistics and Linear Algebra (starting from high school level classes) - the notations and terminologies are very different. For instance, the first and third paragraphs in the Discussion section are saying the exact same thing.

I'm not the most bold editor but I'm going to try my best to clean up the redundant info here and add some verifiable sources, and would appreciate if fellow editors helped wikify the article and suggest any improvements or citations as they see fit.

Also, I hope frequent editors of this article aren't concerned if the article size reduces a bit. I spent most of last year TA-ing two undergraduate courses (Statistics for ML and Linear Algebra + Multi-variable calc) so I'll explain the bigger edits in detail. AradhanaChatterjee (talk) 19:30, 15 September 2024 (UTC)

Yes, I agree that this article could benefit from some cleaning up. I think that it has been accruing cruft from editors of various viewpoints for a bit too long. If the length could decrease while still covering all of the important content, then that would be a good thing. :) Mgnbar (talk) 20:02, 15 September 2024 (UTC)