Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Background 2.1 Amazon's Mechanical Turk
Implications of Exposing Pii
Forum Discussions Regarding Privacy
Sharing Data Online
Lesson in Value Sensitive Design
Conclusion
Recommendations
References

Mechanical Turk is Not Anonymous [ Version 1 . 0 , March 6 , 2013 ]

Profile image of Saeideh BakhshiSaeideh Bakhshi

2013

visibility

…

description

15 pages

link

1 file

Abstract

While Amazon’s Mechanical Turk (AMT) online workforce has been characterized by many people as being anonymous, we expose an aspect of AMT’s system design that can be exploited to reveal a surprising amount of information about many AMT Workers, which may include personally identifying information (PII). This risk of PII exposure may surprise many Workers and Requesters today, as well as impact current institutional review board (IRB) oversight of human subjects research involving AMT Workers as participants. We assess the potential multi-faceted impact of such PII exposure for each stakeholder group: Workers, Requesters, and AMT itself. We discuss potential remedies each group may explore, as well as the responsibility of each group with regard to privacy protection. This discussion leads us to further situate issues of crowd worker privacy amidst broader ethical, economic, and regulatory issues, and we conclude by offering a set of recommendations to each stakeholder group.

References (69)

  1. REFERENCES
  2. Amazon Mechanical Turk Forum. Discussion Thread: Worker Privacy/Anonymity. https://forums.aws.amazon.com/thread.jspa? messageID=104498&#104498. Visited March 5, 2013.
  3. Amazon.com. Amazon Mechanical Turk Privacy Notice. https://www.mturk.com/mturk/privacynotice. Last Updated: August 5, 2009. Visited March 5, 2013.
  4. Amazon.com. FAQ: Worker Web Site FAQs. https: //www.mturk.com/mturk/help?helpPage=worker. Visited March 5, 2013.
  5. Amazon.com. Participation Agreement. https://www.mturk.com/mturk/conditionsofuse. Last updated: November 1, 2012. Visited March 5, 2013.
  6. Amazon.com. Requester FAQ: Policies. https://requester.mturk.com/mturk/help? helpPage=policies. Visited March 5, 2013.
  7. Amazon.com. Some thoughts on invalid HITs. The Mechanical Turk Blog, Decemeber 17, 2010. http://mechanicalturk.typepad.com/blog/2010/ 12/some-thoughts-on-invalid-hits-.html. Visited March 5, 2013.
  8. Anonymous. The reasons why amazon mechanical turk no longer accepts international turkers. Tips For Requesters On Mechanical Turk, January 17, 2013. http://turkrequesters.blogspot.com/2013/01/ the-reasons-why-amazon-mechanical-turk.html.
  9. A. Baio. The Faces of Mechanical Turk, 2008. November 20. waxy.org/2008/11/the_faces_of_mechanical_turk.
  10. M. Barbaro, T. Zeller, and S. Hansell. A face is exposed for aol searcher no. 4417749. New York Times, August 9, 2006.
  11. R. Bell and Y. Koren. Lessons from the Netflix prize challenge. ACM SIGKDD Explorations Newsletter, 9(2):75-79, 2007.
  12. J. Bezos. Opening Keynote and Keynote Interview. September 27, 2006. techtv.mit.edu/videos/ 16180-opening-keynote-and-keynote-interview-with-jeff-bezos.
  13. J. J. Chen, N. J. Menezes, A. D. Bradley, and T. North. Opportunities for crowdsourcing research on amazon mechanical turk. In CHI Workshop on Crowdsourcing and Human Computation, 2011.
  14. E. H. Chi and M. S. Bernstein. Leveraging online populations for crowdsourcing: Guest editors' introduction to the special issue. IEEE Internet Computing, 16(5):10-12, 2012.
  15. S. Chiarella. Personal Communication. March 5, 2013.
  16. E. K. Choe, S. Consolvo, J. Jung, B. Harrison, and J. A. Kientz. Living in a glass house: a survey of private moments in the home. In Proceedings of the 13th international conference on Ubiquitous computing, UbiComp '11, pages 41-44, New York, NY, USA, 2011. ACM.
  17. E. Cushing. Amazon mechanical turk: The digital sweatshop. UTNE Reader, January/February 2013.
  18. danah boyd. What is the role of technology in human trafficking?, December 7, 2011. http://www.zephoria.org/thoughts/archives/ 2011/12/07/tech-trafficking.html.
  19. F. Data. Section 6050W of the Internal Revenue Code, November 2010. http://www.firstdata.com/ downloads/marketing-merchant/255-001_irs_ merchantuw111810fff.pdf. Visited March 5, 2013.
  20. J. Davis, J. Arderiu, H. Lin, Z. Nevins, S. Schuon, O. Gallo, and M. Yang. The HPU. In Computer Vision and Pattern Recognition Workshops (CVPRW), pages 9-16, 2010.
  21. A. Felstiner. Sweatshop or paper route?: Child labor laws and in-game work. In Proceedings of the 1st Annual Conference on the Future of Distributed Work (CrowdConf ), San Francisco, September 2010.
  22. K. Fort, G. Adda, and K. B. Cohen. Amazon mechanical turk: Gold mine or coal mine? Computational Linguistics, 37(2):413-420, 2011.
  23. E. F. Foundation. Eff demands ftc investigation and privacy reform after aol data release, August 14, 2006. https://www.eff.org/press/archives/2006/08/14.
  24. B. Friedman. Value-sensitive design. interactions, 3(6):16-23, Dec. 1996.
  25. B. Friedman, P. H. Kahn, and A. Borning. Value sensitive design and information systems. In Human-Computer Interaction and Management Information Systems: Foundations. M.E. Sharpe, pages 348-372, 2006.
  26. R. Harmanci. The googler who looked at the worst of the internet. BuzzFeed, 2012. August 21. http://www.buzzfeed.com/reyhan/ tech-confessional-the-googler-who-looks-at-the-wo.
  27. J. Heer and M. Bostock. Crowdsourcing graphical perception: using mechanical turk to assess visualization design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '10, pages 203-212, 2010.
  28. P. Heymann and H. Garcia-Molina. Turkalytics: analytics for human computation. In Proceedings of the 20th international conference on World wide web, pages 477-486. ACM, 2011.
  29. J. Howe. The rise of crowdsourcing. Wired magazine, 14(6):1-4, 2006.
  30. P. Ipeirotis. Demographics of Mechanical Turk. Technical Report CeDER-10-01, New York University, 2010.
  31. L. Irani. Idealogical work. In preparation, 2013.
  32. L. Irani and M. Silberman. Turkopticon: Interrupting worker invisibility in amazon mechanical turk. In Proceeding of the ACM SIGCHI Conference on Human Factors in Computing Systems, 2013.
  33. A. Irwin. Constructing the scientific citizen: Science and democracy in the biosciences. Public Understanding of Science, 10(1):1-18, Jan. 2001.
  34. P. G. Kelley. Conducting usable privacy & security studies with amazon Šs mechanical turk. In Symposium on Usable Privacy and Security (SOUPS)(Redmond, WA, 2010.
  35. A. Kittur, J. V. Nickerson, M. Bernstein, E. Gerber, A. Shaw, J. Zimmerman, M. Lease, and J. Horton. The Future of Crowd Work. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW), pages 1301-1318, 2013.
  36. J. Klinger and M. Lease. Enabling trust in crowd labor relations through identity sharing. In Proceedings of the 74th Annual Meeting of the American Society for Information Science and Technology (ASIS&T), pages 1-4, 2011.
  37. S. Kochhar, S. Mazzocchi, and P. Paritosh. The anatomy of a large-scale human computation engine. In Proceedings of the ACM SIGKDD Workshop on Human Computation, pages 10-17. ACM, 2010.
  38. E. Law and L. v. Ahn. Human computation. Synthesis Lectures on Artificial Intelligence and Machine Learning, 5(3):1-121, 2011.
  39. B. N. Levine, C. Shields, and N. B. Margolin. A survey of solutions to the sybil attack. Technical report, University of Massachusetts Amherst, Amherst, MA, 2006.
  40. D. Liu, R. Bias, M. Lease, and R. Kuipers. Crowdsourcing for usability testing. In Proceedings of the 75th Annual Meeting of the American Society for Information Science and Technology (ASIS&T), October 28-31 2012.
  41. W. Mason and S. Suri. Conducting behavioral research on Amazon's Mechanical Turk. Behavior Research Methods, 44(1):1-23, 2012.
  42. W. Mason and D. J. Watts. Financial incentives and the Performance of Crowds. In SIGKDD, 2009.
  43. N. Menezes. Net:Work Conference Recap. The Mechanical Turk Blog, Decemeber 9, 2010. http://mechanicalturk.typepad.com/blog/2010/ 12/network-recap.html. Visited March 5, 2013.
  44. C. Mims. How mechanical turk is broken. MIT Technology Review, January 3, 2010.
  45. S. Mitchell. Inside the online sweatshops. PC Pro Magazine, 2010. August 6. www.pcpro.co.uk/ features/360127/inside-the-online-sweatshops.
  46. mTurk Forum. Discussion Thread: Tax information i got from amazon! http://mturkforum.com/showthread.php? 5300-Tax-information-i-got-from-amazon. Visited March 5, 2013.
  47. A. Narayanan and V. Shmatikov. Robust de-anonymization of large sparse datasets. In Security and Privacy, 2008. SP 2008. IEEE Symposium on, pages 111-125. IEEE, 2008.
  48. M. K. Ohlhausen. The FTC Šs new privacy framework. Antitrust, 25:43-46, 2011.
  49. P. Ohm. Broken promises of privacy: Responding to the surprising failure of anonymization. UCLA Law Review, 57:1701-1777, 2010.
  50. A. Osterhaug. Privacy without borders: The ins and outs of outsourcing. Data Breach Examiner, 3, August 2011.
  51. G. Paolacci, J. Chandler, and P. Ipeirotis. Running experiments on amazon mechanical turk. Judgment and Decision Making, 5(5):411-419, 2010.
  52. P. Paritosh, P. Ipeirotis, M. Cooper, and S. Suri. The computer is the new sewing machine: benefits and perils of crowdsourcing. In Proceedings of the 20th international conference companion on World wide web, pages 325-326. ACM, 2011.
  53. J. Pontin. Artificial intelligence, with help from the humans. New York Times, March 25, 2007.
  54. A. J. Quinn and B. B. Bederson. Human computation: a survey and taxonomy of a growing field. In 2011 Annual ACM SIGCHI conference on Human factors in computing systems, pages 1403-1412, 2011.
  55. J. Ross, L. Irani, M. Silberman, A. Zaldivar, and B. Tomlinson. Who are the crowdworkers?: shifting demographics in mechanical turk. In Proceedings of the 28th of the international conference extended abstracts on Human factors in computing systems, pages 2863-2872. ACM, 2010.
  56. M. D. Scott. The FTC, the Unfairness Doctrine, and Data Security Breach Litigation: Has the Commission Gone Too Far. Admin. L. Rev., 60:127, 2008.
  57. D. N. Shapiro, J. Chandler, and P. A. Mueller. Using mechanical turk to study clinical populations. Clinical Psychological Science, 2013.
  58. A. Shaw. Some initial thoughts on the otey vs. crowdflower case, January 9, 2013. http://fringethoughts.wordpress.com/2013/01/09/ some-initial-thoughts-on-the-otey-vs-crowdflower-case/.
  59. M. Silberman, L. Irani, and J. Ross. Ethics and tactics of professional crowdwork. XRDS: Crossroads, The ACM Magazine for Students, 17(2):39-43, 2010.
  60. C. Sunstein. Infotopia:How Many Minds Produce Knowledge. Oxford University Press, USA, 2006.
  61. J. Surowiecki. The wisdom of crowds. Anchor, 2005.
  62. Turker Nation Forum. Dicussion Thread: *** IMPORTANT READ: HITs You Should NOT Do! ***. http://turkernation.com/showthread.php? 35-***-IMPORTANT-READ-HITs-You-Should-NOT-Do! -***. Visited March 5, 2013.
  63. Turker Nation Forum. Dicussion Thread: Privacy, shmivacy. http://turkernation.com/archive/ index.php/t-6065.html. Visited March 5, 2013.
  64. Turker Nation Forum. Dicussion Thread: Requester posting Worker IDs. http://turkernation.com/ archive/index.php/t-1384.html. Visited March 5, 2013.
  65. Turker Nation Forum. Dicussion Thread: Requesters Who Change Their Name. http://turkernation. com/archive/index.php/t-2886.html. Visited March 5, 2013.
  66. D. Vincent. China used prisoners in lucrative internet gaming work. The Guardian, May 25, 2011.
  67. L. von Ahn. Human Computation. PhD thesis, Carnegie Mellon University, 2005. Tech. Report CMU-CS-05-193.
  68. H. Wallach and J. W. Vaughan. Workshop on Computational Social Science and the Wisdom of Crowds. In NIPS, 2010.
  69. S. Wolfson and M. Lease. Look Before You Leap: Legal Pitfalls of Crowdsourcing. In Proceedings of the 74th Annual Meeting of the American Society for Information Science and Technology (ASIS&T), 2011.
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved