Term Paper: Crowdsourcing

STRAYER UNIVERSITY

CIS-524 Term Paper

Crowdsourcing

Name

Date

Abstract In the first half of this paper, I will examine the invention and growth of crowdsourcing in the field of interface design and how the availability of crowdsourcing services is now making human computation easily available to the research community. There is currently significant interest in the exploring the use of crowdsourcing services to support existing research activities in information and data processing technologies; and how it might be used to open up new research directions, which might be technically innovative or previously have been impractical using other means.. Second, I will describe the impact that crowdsourcing has had on the field of interface design. Third, I will discuss at least 3 benefits of incorporating crowdsourcing in a design project. Finally, I will discuss at least 3 challenges of incorporating crowdsourcing in a design project.

The invention of and growth of crowdsourcing in the field of interface design has grown exponential. (Alsever, 2007) states that the basic idea of crowdsourcing is to “tap into the collective intelligence of the public at large to complete business-related tasks that a company would normally either perform itself or outsource to a third-party provider”. This means that companies would utilize the benefits of outside sources to help accomplish a certain goal or criteria that the company can capitalize on. Unlike outsourcing, crowdsourcing utilizes the resources that can be found in the United States. Small businesses with tighter budgets utilize crowdsourcing to obtain ideas for their user interface at a fraction of the cost. For example, a start-up company called ‘Health Junky’ wants a logo for their company but they do not want to pay a professional $1,000 for their ideas. Instead, they utilize the crowdsource to obtain a plethora of ideas on their new logo. They pay the winner of the logo design $100 and save 90% by using a crowdsource. Therefore, the invention and growth of crowdsourcing in the field of interface design can be linked to the age of the internet.

Crowdsourcing is the act of taking a job traditionally performed by a designated agent and outsourcing it to an undefined, generally large group of people in the form of an open call. People perform crowdsourced work for various reasons: payment, altruism, enjoyment, reputation, socialization, etc. Crowdsourcing is becoming increasingly popular and has been studied as a usability engineering method with crowdsourced usability testing, one can tap into a wide diversity of users to test an online website or application. While crowdsourcing boasts various strengths vs. prior practices, various problems currently limit the potential of existing platforms like mTurk. Worker anonymity, coupled with lack of sufficient accountability and task-based payment entices some workers to complete many tasks poorly, or even utilize bots (contrary to terms of service). For example, “spammers” or cheaters may try to maximize their individual profits without care as to the quality of work they perform. They might answer questions randomly, jeopardizing the validity of study results based upon their answers. While participants who do not fully engage in the traditional usability test in lab settings also exist, they have not been nearly prevalent as in crowdsourcing today. I will recommend contrary to traditional usability design to make crowd sourced tasks more effortful to complete such that it is no easier to cheat than to do complete the task correctly. Another challenge they identified is potentially low ecological validity: the experimenter has little control of the setting in which the mTurk user carries out a task

The impact that crowdsourcing has had on the field of interface design has had a positive and negative effect. Although small businesses and startups may not have the luxury of a large budget to spend on logos and websites, they have the resources of the crowd source to retrieve a plethora of ideas. According to (Gonzales, 2010), “It is, in essence, an open (or semi-open) invitation to a “crowd” or community—usually a wide and public one, such as online communities of graphic artists—to solve a problem or create a product, with or without monetary compensation. Most often though, prize money is dangled like a carrot”. In a positive outlook, this gives new players in the game the chance to win contests and get their name out there. In a negative outlook, it gives players that have been in the game unwanted competition. (Gonzales, 2010) states that crowdsourcing has closed the gap between professional artists and the amateurs, as well as the seasoned designers and the hobbyist. This comes to show that work can be done from the higher level range to the players that do this type of stuff for little or no commission. Although this may sound like that work may go to the cheapest biller, the company may decide to go with the company that has a business model in place to incorporate a feeling of self-preservation. The intimacy between the graphic designer and client often comes at a price that the company would typically pay for. Also, Crowdsourcing methods have attracted considerable attention because of assumed efficiencies of collecting data and solving tasks via programmatic access to human talent. In the realm of language technologies, crowdsourcing has been used for speech transcription, system evaluation, read speech acquisition, search relevance, translation, and most recently, paraphrase generation. We shall introduce and address the problem of crowdsourcing language that corresponds to a given semantic form. Some of the methods we use bear similarities to previous work in paraphrase generation. While paraphrase generation seeks mappings between surface-level realizations of language without knowledge of the underlying semantics, we focus on capturing the mapping from semantic to lexical forms. To the best of our knowledge, this is the first report on the use of crowdsourcing to address this problem. Therefore, the impact that crowdsourcing has had on the field of interface design has been both positive and negative.

Three benefits of incorporating crowdsourcing in a design project are productivity, creativity, and minimizing labor and research expenses. When it comes to crowdsourcing the, the internet is a source that taps into the realm of the World Wide Web. According to (Alsever, 2007), “Using the internet to solicit feedback from an active and passionate community of customers can reduce the amount of time spent collecting data through formal focus groups or trends research, while also seeding enthusiasm for upcoming products”. This helps increases productivity because hiring a team to collect data and number of subjects takes a great deal of time and money. (Alsever, 2007) also states that by “involving a cadre of customers in key marketing, branding, and product development processes, managers can reduce both staffing costs and the risks associated with uncertain marketplace demand”. This increases the creativity and minimizes the labor and research expenses that would otherwise be accrued through professional services. By utilizing a broad base crowd, companies can obtain creative ideas that are usually offered at a much lower rate than a professional would charge. Therefore, the three benefits of incorporating crowdsourcing in a design project are productivity, creativity, and minimizing labor and research expenses. It also can entails empowering a disparate group of people with the tools to contribute to a larger effort. Incentives to contribute should be tailored to attract the most effective collaborators, and the crowd’s motivation must align with the long-term objective of the crowdsourcing initiative to ensure consistent, good quality participation. In Ankit Sharma’s model of critical crowdsourcing success factors, summarized below, motive alignment of the crowd is the central idea, whereas the vision and strategy of the crowdsourcing initiative, linkages and trust, external environment, infrastructure, and human capital are peripheral factors.

Infrastructure: A necessary prerequisite for crowdsourcing is the availability, acceptance, and use of crowdsourcing technologies by its users. The ease of accessibility, reliability, and quality of communication technologies and infrastructure is therefore imperative for crowd participation. The global spread of mobile phones has achieved the basic condition for the use of crowdsourcing in many developing country contexts.

Vision: The crowdsourcing initiative must present a vision with a well-defined set of ideals, goals, and objectives that is sensitive to the dynamics of its environment so that the crowd can perceive the initiative as valuable and well-intentioned. While government participation can add an additional trust factor to the initiative, this is not always the case in a fragile or predatory state context.

Human Capital: Another key determinant of success in crowdsourcing is human capital, both at the level of the initiators and the crowd joining the initiative. This includes language skills, managerial skills, national orientation, traditions, and level of education. Basic mobile phone skills are an entry qualification for the crowd. In an ideal scenario, the crowd can engage the crowdsourcing initiative without prior training and with minimal interventions.

Financial Capital: The inherent nature of crowdsourcing initiatives does not make them very capital intensive, especially if based on existing telecommunications infrastructure such as mobile phones and networks. Additional investments to improve infrastructure can enhance crowd participation substantially. Also, in low-income countries, performance-based donor-funding of local community development could be used to create a positive incentive for governments to allow for greater citizen scrutiny and participation, e.g., through crowdsourced monitoring and reporting platforms.

Three challenges of incorporating crowdsourcing in a design project are costs involved screening the ideas, how to organize this distributed innovation process, and determining the success of the idea. Generally, when determining the cost and allocation of funds towards an idea, the customer must determine whether to use an open mode or a closed mode method of screening the ideas. This means that the costs to screen the ideas can come in a variety of ranges from cheap to expensive. To organize this distributed innovation process, we must take a look at the number of applicants that can provide the necessary credentials in order to participate in the study. According to Aitamurto, Leiponen, & Tee (2011), “communities are especially useful when an innovation problem is based on cumulative knowledge, i.e. when it continually builds on past advances”. The third challenge of incorporating crowdsourcing in a design project is determining the success of the idea. Aitamurto, Leiponen, & Tee (2011) stated that “A large number of participants involved is also a key feature of crowdsourcing. However, as highlighted in the following section, large numbers of ideas are not necessarily beneficial to firms, and might in some cases constitute a major burden”. This is due to the fact the number of participants range from the novice level to the professional grade. Therefore, the three (3) challenges of incorporating crowdsourcing in a design project are costs involved screening the ideas, how to organize this distributed innovation process, and determining the success of the idea.

As we move forward to the second part of this paper, let us take a look at what we have learned about crowdsourcing. So far, I have examined the invention and growth of crowdsourcing in the field of interface design. Second, I have described the impact that crowdsourcing has had on the field of interface design. Third, I have discussed at least 3 benefits of incorporating crowdsourcing in a design project. Finally, I have discussed at least 3 challenges of incorporating crowdsourcing in a design project and will now be going into the second part of this paper. I also recommend crowdsourcing system must address the following four issues:

How to recruit and retain workers?

What contributions can workers make?

Collaborations between workers in crowdsourcing environments can be explicit or implicit.

1. How to combine worker contributions to solve the target problems?
2. How to evaluate workers and their contributions?

Some examples: Wikipedia, Linux: the crowdsoucing system enlists a crowd of workers to explicitly collaborate to build a long lasting artefact of use to a larger community. ESP game workers implicitly collaborate to label images as a side effect while playing a game.

Amazon Mechanical Turk workers collaborate implicitly, e.g. enlist workers to find a missing boat in thousands of satellite images.

But not all human-centric systems address these challenges, and such systems do not fall within the scope of crowdsourcing: e.g. crowd management at a sports event does not look to recruit more members of the crowd, if anything in this case it would be preferable for members of the crowd to leave!

In the second half of this paper, I will propose a solution for generating interest in my design project from an online community. Second, I will suggest a solution for evaluating the skill set and quality of the code submitted by potentially unknown users. Third, I will describe how crowdsourcing may affect the budget and timeline of a design project. Finally, I will assess the crowdsourcing in regard to the legal, societal, and ethical issues it raises and thoroughly suggest methods to alleviate these concerns.

In order to generate interest in my design project from an online community, we must reach out to the masses in order to kick start our project. According to (Thomases, 2012), “Online user forums, particularly for IT and tech support issues, have long served as the go-to place for niche answers; today we also have broad-based question-and-answer platforms like Quora, Yahoo Answers and LinkedIn Answers, to name a few”. These websites help generate interest by providing an area where questions and innovations can be answered. Yahoo Answers, for example, provides users the ability to ask questions that can be answered by the masses. Users can vote on the answer that best suites the question if there are multiple answers to the question. For web design and development, we can visit websites that have the ability to cater to the needs of the owner and provide different design layouts that are easy to navigate through. According to (Thomases, 2012), “Sites like 99Designs or Crowdspring create an open marketplace where designers can submit designs on a name-your-own-price model; you choose the one you like. The website Threadless has even built its entire business model on crowdsourced T-shirt designs”. These websites provide an open marketplace where designers can obtain ideas on how to create their user interface. Therefore, we must look towards the online community and forums in order to generate interest in a design project.

Although we can crowdsource a number of our tasks to the open public, we must also come up a solution for evaluating the skill set and quality of the code submitted by potentially unknown users. Like many individuals, obtaining credible references for jobs that have been accomplished in the past is the key to success. We must look at crowdsourcing as contract jobs where individuals can post their resumes and client feedback on websites like Angie’s List. According to (Yamshom2011), “Angie’s List is a service listing and review site that offers user-based rankings and reviews of service professionals in local areas”. By going to a site that is similar to Angie’s List, companies that are looking to crowdsource can send in their reviews so that communities can share their experience. Although Angie’s List requires a membership fee to obtain reviews from previous customers, users can rest assure that the reviews are thoroughly screened by administrators. Crowdsourcing is an online distributed problem solving and production model used largely by online businesses. The success of the crowdsourcing model depends on the assumption that online communities have collective intelligence or crowd wisdom and empirical research supports this assumption. I proposed online community because I found that problem solving processes benefit from cognitively diverse communities, even communities of non–experts, and that ideation problems dealing with the generation of unique, creative ideas are well–suited to broadcasting to an online community for solving. Of the four types of crowdsourcing, the peer–vetted creative production approach is the best suited to online public participation processes. In this approach to crowdsourcing, an organization issues a challenge to an online community. Individuals in this community may then submit designs or solutions to address the challenge, and individuals are also able to vet the submissions of peers. Notable examples of this approach include t–shirt Company and user–generated advertising contests, such as Doritos’ Crash the Super Bowl contest. The logic of this approach is that by opening up the creative phase of a designed product to a potentially vast network of Internet users, some superior ideas will exist among the flood of submissions. Further still, the peer vetting process will simultaneously identify the best ideas and collapse the market research process into an instance of firm–consumer co–creation. It is a system where a “good” solution is also the popular solution, and a solution the market will support, not unlike the outcomes sought in public participation programs.

Therefore, a solution for evaluating the skill set and quality of the code submitted by potentially unknown users would be a service similar to Angie’s List.

Crowdsourcing may affect the budget and timeline of a design project depending on the size and timetable that companies are working with. Therefore, the size and method may affect the budget and timeline of a design project depending on the size and timetable that companies are working with. Reliable crowdsourcing takes time, and as we all know, time is money. According to (Livingston 2010), “It takes time to crowdsource effectively. Indeed, although an organization might outsource innovation, it cannot outsource the labor necessary to be successful”. By producing a well thought out plan, crowdsourcing can be a useful tool when the need arises. (Livingston 2010) also states that, “In spite of well-stated and published rules, an organization still needs to provide community management resources if it expects to sustain community interest. If there is a lack of structure, you can expect to invest even more time”. By sustaining community interest, companies can use their inputs and determine the best course of action to take. Therefore, choosing the right crowdsource audience and providing community interest can help limit budget overspending and ensure design plans follow the timeline.

Legal concerns arise when intellectual property, such as copyrights and patents, are accidently breached because a person may have unknowingly breached the legal document. According to (Yankovsky, 2012) dealing with copyright, patent, or trademark law, this risk continuum generally applies throughout intellectual property law, because the burdens of clearing potential infringement and obtaining rights in the work product generally are similar”. For example, a company sources a design for their website logo. Because they are creating this log internally, they are able to reduce the risk of infringement due to the fact the designs would be unique. When they outsource the logo design to the crowdsource, they are put into a high risk zone in which they might be utilizing a logo that has already been used in a previous project. (Yankovsky2012) also state that internally sourced designs occupy the lowest risk area of the design source continuum and theoretically have the lowest clearance risk of the categories discussed, due to the element of control a company has over its employees. In addition, the work that is created by the employee qualifies as works made for hire. Therefore, legal issues arise when intellectual property are breached which can create a cause for concern for the small company.

Social issues concerning crowdsourcing arise when matters that directly or indirectly affect a person or members of a society. Some may say that crowdsourcing reduces the number of employment opportunities and can remove a number of jobs away from the workforce. This is the due to the fact that employees are no longer needed if jobs are crowdsourced. (Rosenberg, 2011) states, “we are mainly interested in what crowdsourcing can do to help civilians contribute to social change in a way that is both useful and emotionally satisfying”. But if we look at the latter, we can see contributors may provide useless input for completing a project due to the openness of crowdsourcing.

Conclusions And Future Work

Crowdsourced data has emerged as a valuable source of information about crisis events. We believe crowdsourced information can be even more useful after data processing techniques are applied to clean up and organize the data for end users. However, additional work is needed to validate our approach. A variety of data mining techniques can be applied to crowsdsourced data including trust assessment, stop-word removal, vectoring, and feature selection. Relief organizations are a subset of the crowd interested in or affected by a particular crisis. Relief organizations acting collaboratively as a relief group can supplement crowdsourced data with group source information. To maximize efficiency and situational awareness for the relief group, an integrated response coordination system should support crowdsourcing, request collection, response, coordination, and statistics. As an integrated system is realized, future efforts should investigate how well a group sourced crisis co-ordination system will overcome traditional barriers to collaboration during relief efforts. Many challenging research questions remain to be addressed. The crowd-sourced data is often noisy and filled with repetitive entries without much relevance. Denoising is an imperative process to filter out and clean up data for analysis. In order to facilitate collaboration between organizations, we could provide a recommendation module to allow the involved organizations to perform scenario planning based on their private capabilities before committing to physical contribution. Their private capabilities include budget and priority. Other factors include the emergency level of requests, delivery cost, and the most optimized route to deliver, and so on. One of the key complaints with donors is that they are not informed of fulfilment: what is delivered, how much of the need is met, who gets what. The monitoring module can address this issue and help smooth and improve coordinated disaster relief.

Reference

Aitamurto, T., Leiponen, A., & Tee, R. (2011).The Promise of Idea Crowdsourcing. Retrievedfromhttp://www.academia.edu/963662/The_Promise_of_Idea_Crowdsourcing_Benefits_Contexts Limitations

Alsever, J. (2007). What is Crowdsourcing? Retrieved from http://www.cbsnews.com/8301-505125_162-51052961/what-is-crowdsourcing/

Gonzales, P. (2010). Crowdsourcing Erases Graphic Design as we know it.Retrieved fromhttp://www.fuelyourcreativity.com/crowdsourcing-erases-graphic-design-as-we-know-it/

Lieberstein, M., Tucker, A., &Yankovsky, A. (2012, September). Crowdsourcing: Understanding the Risks. Retrieved fromhttp://www.kilpatricktownsend.com/~/media/Files/articles/2012/MLieberstein%20NYSBA.ashx

Livingston, G. (2010, October 12). 4 Real Challenges to Crowdsourcing for Social Good.Retrieved from http://mashable.com/2010/10/12/social-good-crowdsourcing/

Phneah, E. (2013). Crowdsourcing faces ethical, legal risks. Retrieved June 16, 2013, from http://www.zdnet.com/crowdsourcing-faces-ethical-legal-risks-7000016300/

Rosenberg, T. (2011, March 28). Crowdsourcing a Better World. Retrieved from http://opinionator.blogs.nytimes.com/2011/03/28/crowdsourcing-a-better-world/

Schneiderman, B., &Plaisant, C. (2010). Designing the User Interface: Strategies for Effective Human-Computer Interaction (5th ed.). Boston, MA: Pearson.

Stevens, D. (NA). Crowdsourcing: Pros, Cons, And More. Retrieved June 16, 2013, from http://www.hongkiat.com/blog/what-is-crowdsourcing/

Thomases, H. (2012, September). How to Crowdsource Anything. Retrieved from http://www.inc.com/hollis-thomases/how-to-crowdsource-anything.html

Yankovsky, L. (2012). Angie’s List: Is the Service Site Worth Its Membership Fee Retrievedfromhttp://www.pcworld.com/article/237331/angies_list_is_the_service_site_worth_its_membership_fee.html