Five things I learned this week (future tense edition)

Kolob Reservoir, Utah (August, 2012).

For this edition of my occasional “five things” series, I’m trying out a twist on the usual theme (ideas, places, people, or things that I’ve run across in the preceding week) by discussing five things I’ll learn about next week.  So, without further ado, here are five things I am excited to encounter in the coming days…

1. CHI and CrowdCamp – I’m headed to Austin, Texas at the end of the week to present at CHI and participate in the CrowdCamp workshop. The lineup and agenda for CrowdCamp look incredibly exciting – the plan is to rapidly brainstorm, design, and (if possible) implement crowdsourcing projects. Given the past accomplishments of many of the other people who will be in the room, I’m excited!
2. New Zion Missionary Church (no website) – As part of my Austin trip, I hope to make a pilgrimage or two to as many of the regional holy sites of barbecue as I possibly can. In the case of New Zion Missionary Baptist Church (link points to a 2010 review on the Full Custom Gospel BBQ blog), I have heard that the slow smoked brisket can sometimes resemble a religious experience.
3. May Day Occupy actions in New York – Tuesday marks the first of May and, so it seems, a day of rebirth for the Occupy Movement. A few friends will be attending the New York actions and I’ll try to remember to link to anything they write or photograph.
4. The Onyx Boox M92 – Perhaps as a result of the extra attention that went to Mako’s setup a couple of weeks ago, I’ve succumbed and ordered my own e-book reader. I chose the Onyx Boox M92 because it checked all the boxes that mattered to me (linux based, large E-ink screen,  file format agnostic, vendor agnostic, and not reinforcing the Amazon empire) and because it seems to compare well against similar devices.
5. Calibre – Mako and Alan Toner kindly introduced me to Calibre – a very widely adopted and popular piece of free software to manage e-reader libraries –  this afternoon, but I won’t really start playing with it until my reader arrives next week.

Crowd labor relations revisited

I recently had a pilot version of a crowdsourcing task fail pretty spectactularly, but after discussing the failure with Mako I’ve concluded that my experience helps illustrate some interesting comparisons between labor relations in a distributed online market and more traditional sorts of employment and jobs.

The failure in this case started early: I did a mediocre job designing the task. It’s not really worth going into any details except to say that (out of laziness) I made it really easy for workers to either (a) purposefully respond with spammy results; (b) slack off and not provide responses; (c) try to complete the task but unintentionally do a bad job and therefore provide poor quality results; or (d) try to complete the task and do so successfully. I also did not do a good job incorporating any effective means of differentiating between whether the workers who did not provide accurate results were spamming, shirking, or simply failing

So why does this experience have anything to do with the nature of employment relations?

First, think about it from the employer’s (or the work “requester’s”) point of view. A major part of creating an effective crowdsourcing job consists in minimizing the likelihood or impact of (a)-(c) either by means of algorithmic estimation and/or clever task design. It’s not necessary that every worker provide you with perfect results or even perfect effort, but ideally you find some way to identify and/or remove work and workers that introduce unpredictable sources of bias into your results. Once you know what kind of results you’ve got, it’s possible to make appropriate corrections in the event that some worker has been feeding you terrible data or maybe just unintentionally sabotaging your task by doing a bad job.

In other words, low quality results can provide employer-requesters with useful information if (and only if) the employer-requester finds a way to identify it and use it to their advantage. This means that a poorly designed task is not just one that doesn’t elicit optimal performance from workers, but also one that doesn’t help an employer-requester differentiate between spammers, slackers, passive saboteurs, and those workers who really are trying and (at least most of the time) completing a given task successfully.

When I design a job I always assume that a relatively high proportion of the workers are trying to complete the task in good faith (sure, there are some spammers and slackers out there, but somehow they don’t seem to make up the majority of the labor pool when there’s a clear, well-designed, reasonably compensated task to be done). As a result, if I get predominantly crap responses back from the workers, I assume that they are (maybe somewhat less directly than I might like) providing me with negative feedback on my task design.

Now from the workers’ point of view, I suspect the situation looks a bit different. They have fewer options for dealing with employer-requesters who are trying to scam them. Most distributed labor markets lack features that would support anything resembling collective bargaining or collective action on the part of workers. Communications by workers to employer-requesters are limited and, consequently, there usually aren’t robust mechanisms for offering or coordinating feedback or complaints.

As a result, the most effective communications tool the workers possess is their work itself. Not surprisingly, some of them seem to use their work to engage acts of casual slacking and sabotage that resemble online versions of the “weapons of the weak” described by James C. Scott in his book on everyday resistance tactics among rural peasants.

The ease with which crowdsourcing workers can pursue these relatively passive forms of resistance and tacit feedback relates to a broader, more theoretically important point: in most situations, a member of an online crowd should have a much easier time quitting or resisting than workers in (for example) a factory when they decide they’re unhappy with an employment relationship for any reason. Why?  First off, crowdsourcing workers usually don’t have personal ties to a company, brand, co-workers, managers, etc. Second of all, the structure of online labor markets makes the cost of leaving any one job extraordinarily low. An office worker who (upon being confronted by, e.g., an unpleasant or unethical task) leaves her position risks giving up not only valuable resources like future wages or benefits, but also loses physical stability in her life, contact with friends and colleagues, and the respect or professional support of her superiors. In contrast, a worker in an online crowd who decides to leave her job loses almost nothing. While there is some risk associated with actively spamming or slacking (in some crowdsourcing markets, workers with low quality ratings can be banned or prevented from working on certain jobs), it’s still substantially easier to just walk away and find another task to do.

These are just some of the reasons why theoretical predictions from classical wage and employment economics – for example, that a $0.01 decrease in wages will result in some proportion of employees leaving their jobs – don’t hold up in traditional or crowdsourcing labor markets. The interesting point is that the reasons why these classical theories don’t hold up in crowdsourcing systems don’t have much to do with the complications introduced by social relations since social relations (between workers and employers as well as between workers and workers) are severely constrained in most online labor markets.

 

(Note: The first version of this post was written pretty late at night, so I didn’t include many links to sources. I’ll be trying to add them over the next few days.)

Five things I learned this week

Electronika 302 Recorder - by Daniel Gallegos

Zombie trade agreements: According to some documents acquired by the organization European Digital Rights (EDRi), it appears the G8 has decided to do a Dr. Frankenstein impression and reanimate some of the most thoughtless portions of ACTA’s Internet provisions. This latest instantiation of the ACTA agreement wants control over intellectual property, technology devices, network infrastructure, and YOUR BRAINS.

An awesome experiment on awards (published in PLoS ONE) by Michael Restivo and Arnout van de Rijt – both in the Sociology department at SUNY Stony Brook – shows that receiving an informal award (a barnstar) from a peer may have a positive effect on highly active Wikipedians’ contributions. The paper is only three pages long, but if you want to you can also read the Science Daily coverage of it.

Mako’s extensive account of his workflow tools is finally up on Uses This. The post is remarkable for many reasons. First of all, Mako puts more care and thought into his technology than anybody I know, so it’s great to see the logic behind his setup explained more or less in full. Secondly, I found it extra remarkable because I have been collaborating (and even living!) closely with Mako for a while now and I still learned a ton from reading the post. My favorite detail is unquestionably the bit about his typing eliciting a noise complaint while he was in college. As a rather loud typist myself, I have been subject to snark and snubbery from various quarters over the years, but I’ve never had anybody call the cops on me!

The Soviet Union lives on! But maybe not quite where you’d expect it. My friends and former Oakland neighbors Daniel Gallegos and Zhanara Nauruzbayeva have recently moved themselves and their incredible Artpologist project to New York. Upon arrival, they found themselves surrounded by a post soviet reality that most New Yorkers or Americans simply do not know exists at all, much less in the epicenter of finance capital. Their latest project, My American New York, chronicles this “post soviet America” through photos, stories, Daniel’s beautiful sketches, drawings, and paintings (e.g. the image at the top of this post), all wrapped up in a series of urban travelogues.

Philosophy Quantified: Kieran Healy has done a series of elegant and thoughtful guest posts on Leiter Reports in which he explores data from the 2004 and 2006 Philosophical Gourmet Report (PGR) surveys in an effort to generate some preliminary insights about the relationships between department status and areas of specialization.

Model Model Model Modeling Models

In doing some reading about collective action, cooperation, and exchange theory, I encountered (gated link) the figures below:

If you happen to be the kind of person who spends a lot of time around research combining social dilemmas, evolutionary models of cooperation, and econometric production functions, these may seem completely intuitive and you probably do not even need to read the paper to get the gist of Professor Heckathorn’s argument.

Otherwise, the images may feel a bit more like conceptual art. The Plot labeled “C” at the bottom right is my runaway favorite. I am also a big fan of the mysterious “arch” shape and the large “X” that appear in the first figure.

n.b., Professor Heckathorn does an admirable job explaining these images in the paper and my point here, is not to provide a Tuftean critique of  some rather ornate visualizations. Instead, I wanted to try to communicate the sensation I felt when I encountered these images in the context of an extraordinarily sophisticated and abstract simulation-based analysis of the social dilemmas used to analyze the theoretical conditions under which people may be more likely to cooperate and contribute to public goods.

That’s right, these figures are part of a model modeling models. Given that I am singling out this particular model for attention, they are also, you might say, part of a model model. Given that Professor Heckathorn’s work in this area is highly sophisticated and compelling, you might even say that these figures are part of a model model model (modeling models).