Talk:Metamorphic testing

04:29, 23 September 2018

Hi 82.2.1.89,

I have reinstated the excellent list of applications and related references of metamorphic testing. I find it a very useful guide for researchers and practitioners who want to embark on this important technique.

I do not want to start a second edit war in this Wikipedia entry. To avoid any bias to one particular research group, I have carefully cited the papers by different researchers in Australia, Asia, Europe, and North America.

I have to admit that my submitted list did not fully satisfy my original target of "different researchers". The name "T.Y. Chen" appeared twice. However, please do not treat this as a mathematical condition. Please feel free to recommend better references. On the other hand, I would appreciate it if you could refrain from deleting the contents. It would defeat the purpose of disseminating useful information.

Laiwoonsiu (talk) 04:29, 23 September 2018 (UTC)[reply]

09:47, 23 September 2018

I have revised my list of applications so that NONE of the references are from the Australian group of MT researchers. I know that as a result, my list is heavily biased because, according to the TSE Survey in 2016, the Australians have published 30% of the MT papers. But I have no time for more deletions by my learned friend. Following Wikipedia advice, I am reporting this via "Talk", so that third-party researchers may submit a more balanced version of the Wikipedia entry.

Laiwoonsiu (talk) 09:47, 23 September 2018 (UTC)[reply]

04:20, 24 September 2018

Following the TSE survey paper (2016), I have added missing application areas and the related Wikipedia links and references. Again, I have carefully cited different authors to avoid the recurrence of the edit war.

Laiwoonsiu (talk) 04:20, 24 September 2018 (UTC)[reply]

Concern over the Undoing of Revisions Regarding "Metamorphic Robustness Testing"

Hi Wikipedians,

I am writing to express concern over the undoing of my revision regarding "metamorphic robustness testing".

I cited recent research results based on this new technique, revealing that the presence of simple hyphens in the titles of academic papers adversely affects citation counts and journal impact factors. User:Clarinetguy097 undid my revision, pointing out that I “added a reference to a recently published paper to several pages. This appears to possibly be conflict of interest editing. If you are one of the authors of the paper, please review WP:SELFCITE and ensure compliance. I have also noticed similar behavior on some of the same articles about a week ago. If you are using multiple accounts, please review WP:SOCK.”

In fact, the reference was cited by major international journals, newspapers, and blogs. Both Web of Science and Scopus denied the allegation, and the original authors responded. The amount of interactions shows its importance.

Furthermore, “Lai Woon Siu” is my real name as shown in the passport. I am just a lone crusader who wants balanced views to be shown in Wikipedia. I do not have the time and energy for edit wars.

I would appreciate any channel for appeal.

Laiwoonsiu (talk) 05:03, 21 July 2019 (UTC)[reply]

Your source is a primary source, but articles should be based on reliable secondary sources, so Clarinetguy097 was completely correct in reverting your edit. Time will tell whether these authors were correct or not. --Randykitty (talk) 15:02, 21 July 2019 (UTC)[reply]

Hi Randykitty,

Thank you so much for your clarification, which is enlightening and useful.

According to WP:NOR, “A secondary source ... contains an author’s analysis, evaluation, interpretation, or synthesis of the facts, evidence, concepts, and ideas taken from primary sources”.

Thus, the journal citation reports in Web of Science are a primary source for the Wikipedia “impact factor” page. Similarly, the paper Zhou et al. (2019) is a primary source for the Wikipedia “metamorphic testing” page.

On the other hand, the paper Zhou et al. (2019) is a secondary source for the Wikipedia “impact factor” page because it “contains an author’s analysis, evaluation, interpretation, or synthesis of the facts, evidence, concepts, and ideas taken from primary sources”.

If you agree with this interpretation, I will be happy to remove the discussions about Zhou et al. from the “metamorphic testing” page and put them in the “impact factor” page only.

I look forward to your expert advice.

-- Laiwoonsiu (talk) 09:56, 22 July 2019 (UTC)[reply]

No, I don't agree. WoS provides the raw data for the analysis of Zhou et al, which therefore is a primary source. Zhou et al does not belong here nor in the article on the IF. Sorry. --Randykitty (talk) 10:56, 22 July 2019 (UTC)[reply]

Dear Wikipedians,

(Gasping) (Sweating) Although I proclaim myself as a crusader, I do so only for the sake of software correctness and robustness. I do not have any intension to denounce Web of Science or Google in my Wikipedia edits. In fact, WoS provides a very sophisticated “analytics” system, much much more than “raw data” in the data-information-knowledge-wisdom pyramid. Similarly, Google provides an unparalleled search engine. Our job as software testers is to verify whether there are failures, rather than to denounce these systems. The former is meant to be supportive.

For this reason, a bug is a bug. It may take some time to fixed for a large and complex system. In the meantime, academics may be well advised to avoid hyphens in their paper titles. Such is the usefulness of my editing. I do not have the energy for infinite debates. I will leave the matter to the decision of Wikipedians.

-- Laiwoonsiu (talk) 04:16, 23 July 2019 (UTC)[reply]

Mr. Siu, what is the allegation that you are referring to? The way you phrased it makes it seem like there was a controversy related to the paper. Clarinetguy097 (talk) 03:45, 22 July 2019 (UTC)[reply]

Hi Clarinetguy097,

Thank you for your interesting question. A little background history is in order:

Metamorphic testing (MT) is a technique to verify “untestable” programs. In particular, Zhou et al. applied MT to popular software systems and revealed major failures. Examples include Google search in 2016, Uber driverless cars in 2019, and Web of Science in 2019. While Google and Uber gracefully accepted the bug reports, Web of Science issued a public statement denying the presence of software faults.

The stories of the Google and Uber failures were reported in the Wikipedia “metamorphic testing” page without problem. Yet, the Web of Science story was repeatedly reverted.

Such is the controversy.

-- Laiwoonsiu (talk) 14:49, 22 July 2019 (UTC)[reply]

So the paper was controversial. However, in your edits to this page, you only talked about the conclusion of the paper, without mentioning that Scopus and WoS contested it. In my opinion you gave undue weight to the paper, because there was no mention of the opposing view. I agree with Randykitty that it's now best to wait for a clearer consensus on whether there were software faults or not, and then decide what to do once there is more information. I see now that you probably don't have a conflict of interest, but my suspicion of undue weight still stands. Regarding your above reply to Randy, it may be true that it is in the best interests of authors to not use hyphens, but they should find out this information from somewhere other than Wikipedia. Clarinetguy097 (talk) 05:24, 23 July 2019 (UTC)[reply]

Dear Clarinetguy097,

Although I cannot find time for infinite debates, I feel obliged to reply to the serious allegation about “undue weight”.

The “metamorphic testing” page is about software testing. According to Wikipedia, “Software testing involves the execution of a software component or system component to evaluate one or more properties of interest. In general, these properties indicate the extent to which the component or system under test ... responds correctly to all kinds of inputs. ... Software testing can provide objective, independent information about the quality of software and risk of its failure to users or sponsors.”

Software testing methods cover actual techniques, rather than whether the original software developers appreciate the revealed failures. Please review all the Wikipedia pages about software testing methods. You will notice that they discuss how one technique is better than another, but not how much the end users really like the failures unveiled. This is not “undue weight”.

By the way, WoS has just published the 2018 Journal Citation Reports. It is unlikely that they will admit any software faults in their system. On the other hand, it would be natural for them to fix the bugs at the earliest opportunity. Therefore, it may not be worth “wait[ing] for a clearer consensus on whether there were software faults or not, and then decide what to do once there is more information”.

Further, it is quite surprising to me that Wikipedians would encourage authors to “find out this information [about the adverse effect of hyphens] from somewhere other than Wikipedia”. I thought we were the best encyclopedia in town.

Hope my reply is helpful.

-- Laiwoonsiu (talk) 04:12, 24 July 2019 (UTC)[reply]

I don't know about the best, but we should definitely aim for quality by using high quality sources. Clarinetguy097 (talk) 00:27, 25 July 2019 (UTC)[reply]