Wikipedia:VisualEditor/Why/User Test Data

Main page
Feedback at MediaWiki
Sandbox (no account required)

About

Documentation:
Development:
- Roadmap up to 2017
- Updates (2013-2023)
- Weekly status reports (2015-2016)

Help out

Research

Main research page

Other

Why are the developers building this? (2013)
- User Test Data (2013)
Customization
Known problems
Requests for Comments (RfCs): Jul 2013 a, Jul 2013 b, Jul 2015, Sep 2015

As part of our preparations for the 2013 launch we ran a group of users from UserTesting .com through a set of tasks, first using the VisualEditor and then using wiki markup. Some of them had tried to edit in the past; many were total newcomers. The script for the tests can be seen below. This is not the only data we are relying on; we are also (and more usefully) gathering quantitative data from a test on the English-language Wikipedia, in which 50 percent of new users were given the VisualEditor by default.

Using the source editor

"This automatically looks confusing and overwhelming."
"There was tons of foreign jargon amidst the content and it looked nothing like how the real page looked."
"I don't want to have to learn a new language just to edit Wikipedia."

When presented with the source editor, users tended to have the same set of problems. Many of these centred around identifying what they were expected to change; with so much markup, they found it difficult to identify things in the markup view that matched what they'd seen when reading the rendered page. Users were also worried by the clutter of the editing interface, particularly the mass of buttons at the bottom of the "save page" window.

Users struggled to understand the wikimarkup found in a moderately-sized article; when they managed to identify bits, it was almost entirely from comparing their memory of the rendered page to individual words, and looking at the formatting around those words (for example, noting that all of the headers had equals signs, and thus determining that equals signs made headers work). With one exception, every user found the source editor intimidating and would opt not to use it.

Using the VisualEditor

"I've given up in the past because it seemed too confusing. With [the VisualEditor] it seemed like anyone can figure it out."
"[The VisualEditor] feels more like editing a word document and isn't as intimidating as the [source editor] which feels more like editing code."

Several problems were raised with the VisualEditor. Many users found adding links to be confusing, something we have noted and are evaluating, and (as known) the VisualEditor was slow to load for some testers. Several other (now fixed) bugs, such as problems with saving the page, also frustrated users. However, all but one of the testers concluded that they preferred using the VisualEditor to using the source editor, one of them noting that "[with the VisualEditor] I would be more likely to make edits. That interface was a lot easier to understand and I had more confidence that the changes I was making were the changes that I wanted to make. I also like that I had an opportunity to review the changes and note them."

Test

Script

This is a test of a new editing interface for Wikipedia pages. We will be testing whether it is easier to edit Wikipedia using the current editing interface or the new interface. In this scenario, let's say that you've decided to correct the article "Golden-Crowned Sparrow". Try to make the corrections that we give throughout, and then tell us what parts of the process felt confusing, and whether it was easier or more difficult to make the corrections with the first interface you are presented with, or the second interface.

Log in as "[account]" with the password "[password]". Then click the "Golden-Crowned Sparrow" link.

The first sentence is "The Golden-crowned Sparrow (Zonotrichia atricapilla) is a large Turkish sparrow found in the western part of the Middle East."

Now correct the link "Turkish sparrow" so that it points to "American sparrow" and press the "Review and Save" button. Complete the process of saving changes.

Scroll down to the bottom of the article. There is a section marked "Further reading".

Delete the "Further reading" header entirely. Using the toolbar, make the bold text "Theses" into a level 2 header. Do the same for the bold text "Articles". Save your changes.

Log out (the "log out" link is at the top right on your screen), and log in again as "[username]" with the password "[password]". Go back to the Golden-Crowned Sparrow article. (In case you need it, the link is https://www.mediawiki.org/wiki/Golden-Crowned_Sparrow).

Under "description" there is the sentence "Birds with similarly sized gold crown patches are far more likely to engage in agonistic behavior".

Below that is the section "Similar species". The title for that section ("Similar species") is less prominent than the title for other sections. Push the 'edit' button, and make the section title the same size as the section title for "description". Save your changes.

Questions

Which editing interface would you prefer, the first or the second?

What frustrated you most about the editing interface in the first account?

What frustrated you most about the editing interface in the second account?

Would you be more or less likely to edit Wikipedia using the editing interface in the first account, or would it not make any difference to you?