User:Bilby/Freelance paid editing study

Over a five month period in 2012, a pilot study was conducted into the impact of freelance paid editors on Wikipedia.

Background

The paid editing debate is a complex one for Wikipedia. Wikipedia does not have a policy that specifically forbids paid editing, and the Conflict of Interest (COI) guideline only goes as far as recommending that editors with a COI avoid editing the articles directly. Instead, the focus has been to look at the edits of paid and other COI editors to see if they have breached any of the content policies, and respond on those grounds.

Over many years there has been a push for a stronger response, but the discussions have generally failed to make much progress. Certainly there is agreement than many of the behaviours of editors with a COI are problematic, but it has been unclear as to how to address these issues. There are a few reasons why progress has been difficult. One is that there hasn’t been a clear agreement as to what paid editing entails, and what forms of paid editing represent an issue for Wikipedia – the current language is to couch it in terms of "paid advocacy", to distinguish the behaviours from other forms of more acceptable paid editing, but this does not seem to be fine-grained enough to make the distinctions needed for the debate. Another concern has been the lack of data looking into the impact of paid editors on Wikipedia. There is anecdotal evidence on a case-by-case basis, but it may be that more quantitative data is required to assist the community in the discussions.

It is that second issue which this pilot study was intended to address.

Scope

Rather than look at the broad issue of paid editing, the study was focused on what can be described as “freelance paid editors”. These are people hired on a short-term basis to edit one or more articles on behalf of a client, often through the online freelance marketplaces. Such editors can be distinguished from other paid editors by:

• The lack of an ongoing relationship with the article’s subject.
• Being directly employed to edit articles on behalf of a third party.
• Having a low barrier to entry into the marketplace.

Paid advocates, in the current debate, are generally taken to be Public Relations professionals. But such editors are normally trained professionals in their field, with a code of ethics and an ongoing relationship with their clients. Freelance paid editors, on the other hand, need no particular prior experience, and are typically only employed by the client to create a single article or make a one-off series of edits.

Accordingly, any findings from this study can not be taken to relate to paid editors in general, but only to the particular subset that were considered to be within scope.

Methodology

Over the course of five months, seven websites, Elance, Freelance.com, Freelancer.com, Guru.com, ODesk, People for Hire, Scriptlance, and VWorker were watched in order to identify jobs posted for Wikipedia editing. To avoid biasing the study towards particular editors, each job posting was scrutinised to see if the target Wikipedia article or articles could be identified before a contractor was hired. If this was possible, the posting was flagged and followed – if not, the posting was entered into a database, but not used in the study. Identification of the target article had to be based solely on publicly accessible data.

The original intent was to conduct the study over a six month period. However, as there was a risk that behaviours might be identified during the study which would cause immediate harm to Wikipedia, a number of conditions were defined prior to the start of the pilot study which, if triggered, would result in the immediate termination of the study in order to act to assist Wikipedia directly. One of these conditions was met just prior to the five month point, and accordingly the data collection phase was ended at that time.

Due to privacy concerns and the risk of outing Wikipedia editors, the data will not be publicly available. Being paid as a Wikipedia editor is not currently against policy, so privacy concerns were deemed to outweigh other interests.

Data

Over the course of the study, 247 advertisements were identified directly relating to Wikipedia editing across the seven sites. Of those, nine were identified as being posted outside of the study period, and were subsequently discounted. 149 of the advertisements did not have sufficient information so that the target article could be identified prior to a contractor starting work on the job, leaving 96 articles which fell within the study's scope. At the completion of the study, 50 of those jobs had been assigned to contractors, accounting for 52% of the advertisements that were within scope, and 20.2% of the advertisements found during the study. An additional eight jobs within scope were not assigned to a contractor, but were later completed by a known contractor on Wikipedia. It was surmised that these jobs were advertised in multiple locations and therefore assigned through a different freelance marketplace; advertised more than once on the one marketplace, but the second advertisement was missed; or that the contractors were hired outside of the marketplace's processes.

Job types

The majority of the jobs (XXX, or XXX%) within the study's scope were to create new articles for the client. Typically these were either for a company or an individual, although in some cases the article was about a topic area in order to create links back to the client's website, or a specific product or products belonging to the client. The next most common job (XXX, or XXX%), was to provide links back to the client's website. Other jobs were to expand articles (XXX) or to remove tags (XXX).

X jobs were to vote at a deletion discussion. (note to self - see record 46)

Although outside of scope, one series of job advertisements stood out as requiring the contractor to post comments on behalf of the client, rather than directly making edits to the article, and one was a search engine optimisation (SEO) task to raise the profile of an existing Wikipedia article.

Users

Editor status

None of the identified editors were Administrators on the English Wikipedia, although one who fell just outside of the scope was an administrator on another language's Wikipedia, and one was a Checkuser on a different Wikipedia.

COI

In spite of the current conflict of interest guideline, which recommends that editors disclose their relationship with the subject, none of the editors studied volunteered their conflict of interest.

Few of the editors who fell within the confines of the study were identified as having a potential conflict of interest while editing. On the whole, the editors neither volunteered their conflict of interest nor had to deny possessing one. There were, however, some exceptions.

During the study, XXX editors were identified as paid editors, accounting for XXX% of the editors that fell within the study's scope. Most didn't respond to the allegations,

One editor specifically denied having a conflict of interest, in spite of being paid to edit the article, and did not disclose the relationship to the client. This editor stood out as having also edited the articles of the client's competitors, tagging them as having been written by people with a COI, and accusing one of the editors concerned of having a conflict of interest.

Collaborative paid editing

Originally it was assumed that the this would be a competitive market, and thus there was not expected to be much in the contractors working together. On the whole this held true, with one editor deliberately identifying a competitor as a paid editor during the study period. However, there was a degree of crossover. In some cases (X), more than one editor was hired to complete the task, or (in X cases) a second or third editor was hired to complete a job after the original contractor was unable to complete it. There were also a number of instances where a known contractor assisted another in completing an article, or where an editor who does paid work turned up to argue an against the deletion of another paid editor's article.

See also

www.wikiexperts.us