Publication result detail

Microworkers vs. facebook: The impact of crowdsourcing platform choice on experimental results

RIES, M.; HOSSFELD, T.; GARDLO, B.; SCHATZ, R.

Original Title

Microworkers vs. facebook: The impact of crowdsourcing platform choice on experimental results

English Title

Microworkers vs. facebook: The impact of crowdsourcing platform choice on experimental results

Type

Paper in proceedings (conference paper)

Original Abstract

Subjective laboratory tests represent a proven, reliable approach towards multimedia quality assessment. Nonetheless, in certain cases novel progressive quality of experience (QoE) assessment methods can lead to better results or enable test execution in more cost-effective ways. In this respect, crowdsourcing can be considered as emerging method enabling researchers to better explore end-user quality perception when requiring a large panel of subjects, particularly for Web application usage scenarios. However, the crowdsourcing platform chosen for recruiting participants can have an impact on the experimental results. In this paper, we examine the platform's influence on QoE results by comparing MOS scores of two otherwise identical subjective HD video quality experiments executed on one paid and one non-paid crowdsourcing platform.

English abstract

Subjective laboratory tests represent a proven, reliable approach towards multimedia quality assessment. Nonetheless, in certain cases novel progressive quality of experience (QoE) assessment methods can lead to better results or enable test execution in more cost-effective ways. In this respect, crowdsourcing can be considered as emerging method enabling researchers to better explore end-user quality perception when requiring a large panel of subjects, particularly for Web application usage scenarios. However, the crowdsourcing platform chosen for recruiting participants can have an impact on the experimental results. In this paper, we examine the platform's influence on QoE results by comparing MOS scores of two otherwise identical subjective HD video quality experiments executed on one paid and one non-paid crowdsourcing platform.

Keywords

facebook, QoE

Key words in English

facebook, QoE

Authors

RIES, M.; HOSSFELD, T.; GARDLO, B.; SCHATZ, R.

RIV year

2014

Released

05.06.2012

ISBN

978-1-4673-0724-6

Book

2012 Fourth International Workshop on Quality of Multimedia Experience

Edition

1

Pages from

35

Pages to

36

Pages count

2

BibTex

@inproceedings{BUT95042,
  author="Michal {Ries} and Tobias {Hossfeld} and Bruno {Gardlo} and Raimund {Schatz}",
  title="Microworkers vs. facebook: The impact of crowdsourcing platform choice on experimental results",
  booktitle="2012 Fourth International Workshop on Quality of Multimedia Experience",
  year="2012",
  series="1",
  number="1",
  pages="35--36",
  isbn="978-1-4673-0724-6"
}