Swindell et al., Against Automated Education

 

Swindell, Andrew; Luke Greeley, Antony Farag, and Bailey Verdone (2024), “Against Artificial Education: Towards an Ethical Framework for Generative Artificial Intelligence (AI) Use in Education” Online Learning 28(2), 7-27.

Summary:

This interesting article argues for an ethical framework drawing on the work of Gunther Anders, Michel Foucault, Paolo Freire, Benjamin Bloom (actually, the Revised Bloom’s Taxonomy), and Hannah Arendt. In the event, Anders, Foucault, and Freire are discussed briefly for broader ethical context, but the main focus of the article is the addition of an ethical dimension to Bloom’s Taxonomy using Arendt’s hierarchy of labor, work, and action.

They apply this to the actually existing use of AI by imagining this, frankly, quite likely scenario:

Let’s consider an example of how AI might be used with current GPT technology in a classroom. A journalist, under pressure to produce more consumable content for its struggling publication, uses a GPT to write a story about the benefits and costs of electrical vehicle production and use. A teacher, excited by the labor-saving allure of an AI teaching assistant product called Brisk, uses the software extension to read the news story about electric vehicles and design a 60-minute lesson plan for their students, complete with learning goals, discussion prompts, a presentation activity, and summary quiz about the reading. The students, given carte blanche to use their school-provided Chromebooks, “read” the story using an AI platform like Perplexity, which provides summary analysis and key takeaways for them to use in their discussion and respond to the quiz. Simultaneously, they use Microsoft’s AI image generator to create a slide deck for the class to graphically represent their group’s ideas. The teacher completes the assessment cycle by having their AI assistant grade the quizzes, provide feedback to the students, and input their scores into a learning management system. (Swindell et al., 2024: 17).

[Brisk is a classic example of the stark cynicism of our current use of GAI, allowing “instructors” (the term loses meaning in this context) to automatically generate “feedback” on student essays, which you (the instructor) are then encouraged to “personalize” and present to the students as “from you.”]

The authors’ critique of such a situation:

In this scenario, the AI engages in activities of labor and consumption, while all of the parties involved advance nothing of lasting significance, and if debate or critical reflection arise amongst students it is an incidental, rather than planned, outcome of the AI-prescribed lesson. Indeed, the Brisk teaching assistant might be well programmed to incorporate into the lesson features of the RBT such as understanding, evaluating, and creating activities; but unless a human being in this process is attuned to helping learners act in the world and make it a place, using Arendt’s (1963/2006) words, “fit for human habitation,” ... the most common educational experience might become, ironically, ones in which humans are unnecessary. (ibid., 18)

They go on to propose a “Framework for Ethical AI Use in Education,” in the form of a graphic inputting insights from each of the five philosophies they are drawing on. They apply this framework in two examples, which are, unfortunately, not particularly satisfying. They begin with a list of “guiding questions” for lesson design using AI:

1. In what ways are our historical, technological, social contexts shaping how we think and act; what activity or experience can shock learners into appreciating their contingency?

2. Will the technologies we are going to use advance humanizing ends? In what ways can the technology enhance or harm the co-creation of knowledge?

3. How can we design learning activities that have benefits beyond their own sake; how are the learning activities helping students to act in the world?

4. In what ways can AI reduce the burdens of teaching and learning labor while increasing the capacity to act in the world? (ibid., 22)

[The first two questions show the influence of Foucault and the rest; the last two are primarily informed by Arendt.]

Their first proposed exercise involves a research project in which students seek to learn about their local “political landscape.” AI is used to conduct research on who the local elected officials are, what the local issues are, and what are the important fora for discussion and debate. Students then form their own positions using this knowledge: the idea is that AI performs the “labor” (Arendt’s lowest category), leaving humans free to focus on “action” (Arendt’s highest category.

However, having done exercises like this in the past without AI, this just seems like so many attempts to rationalize an “ethical” or “harmless” use of AI – namely, AI is inserted as an extra element where it is not actually needed. Local political entities, candidates, electoral bodies, and so on, have websites with all this information – it is not hard to find. Using a generative AI search tool only introduces the likelihood of errors, along with the dangerous habit of taking AI as a reliable source of information. At best, AI could be asked what websites contain this information, and then the information looked up on those websites (with the added hope that the list is correct, of course). What is more difficult is not the “labor” of looking up information, but the process of reading through debates, articles, and so on to try and evaluate and formulate issues and positions, and it is this that students are likely to use AI for – against the recommendations of Swindell et al, since after all this involves higher-level Bloom’s and corresponds with “action,” which is supposed to be left to humans.

In their second example,

students are tasked with researching a topic of their choosing both to learn about it and apply this knowledge to their own context. To facilitate this endeavor, AI acts as an agent of Socratic dialogue and questioning for the student, helping students generate research idea topics that will be specifically catered towards student interests. AI will be equipped to ask students questions regarding their level of interests and commitment, suggest other topics of potential interest based on specific student response in addition to refine students’ thinking regarding logical sequencing of topic selection and eventually argument. This personalized approach allows them to analyze how these topics manifest in their own lives and communities, gaining valuable insights. (ibid., 24)

Again, why is AI required to engage in Socratic dialogue? First of all, isn’t this the instructor’s job? (And one of Brisk’s more cynical applications is just such an automated “feedback” generator). But more deeply, isn’t this an opportunity for students to engage in Socratic interaction and mutual critique with each other? After all, the authors have been citing Freire on conscientization and the need to allow students to develop control over their learning process. The instructor could easily model Socratic questioning in class, and give students example questions and topics to guide them in developing their own practice. Delegating this to AI is an opporyunity lost.

Thus, we have yet another attempt at reasoning out an ethical use for AI in the classroom, which fails to provide any good reason for actually using AI in the first place. Seeing as the primary use of AI today is 1) to avoid having to do any actual work or difficult thinking, and 2) to avoid interacting with people, it is hard to see how a “humanist” or ethical use can gain much traction, until this situation – and the underlying causes, pre-existing the development of generative AI – are addressed.

Another limitation of the model could be the reliance on Arendt’s hierarchy of labor-work-action, which has been reasonably criticized as reproducing an arbitrary, classist distinction (cf. also Sennett 1990). It is not true that we don’t learn or gain from anything classed in this model as “labor,” or that there can in fact be a clear line drawn between the actual, complex, productive activities which Arendt has delineated into these three a priori types. More to the point, it is not the type of work, but the social context, aka the relations of production, which render some kinds of work more meaningless or alienating than others. Likewise, it is not the mere fact of automation that is problematic, but how that automation is deployed, to what ends, and in whose ultimate interests. The authors make some nods to this political-economic context (via their discussion of Freire, Foucault, etc.) but the proposed ethical framework does not much reflect this.

Beyond this, the insistence on a “humanist” framing could be a limitation (Arendt in fact called herself an “anti-humanist”). The result is yet another call to keep “humans in the loop,” as masters, rather than servants, of the technology—as if it were the relations between humans and machines, rather than those between humans and other humans, that was ultimately at stake.

What difference might a post-humanist view have on the issue? ANT, for example, could have been brought in to consider the human subject as a historically and contextually created “figure” in a larger more-than-human assemblage, and the dissolution of this figure, with the supplanting of the disciplinary society with the control society, occurring, in Foucault’s words, like the erasure of “a face drawn in sand at the edge of the sea” (Foucault 1970: 387).

Foucault, Michel (1970) The Order of Things. Vintage Books, New York.

Sennet, Richard (1990) The Conscience of the Eye: The Design and Social Life of Cities. Alfred A. Knopf, New York.

Smooth City, Chapter 2

Summary of Chapter 2: Smooth Structures

This chapter goes into more detailed discussion of the Reestraat and the redevelopment of King’s Cross Central in London, in order to illustrate the structures or “mechanisms” that produce and reproduce the “perfection” of the smooth city.

It is important to understand that the smooth city does not appear out of thin air, but is the result of all kinds of power structures, political impulses, planning processes, and design choices. (29)

One of these mechanisms is private ownership of land, and investment in redevelopment; this is tied to certain forms of state control and promotion, and results in a lack of certain [democratic mechanisms] to challenge planning and use. Together, local government and private property-holders exert a “matrix of control” (49), policing the use of space, and excluding undesirable or un-“smooth” populations, as well as unapproved practices such as loitering, putting up flyers, graffiti, etc. Boer emphasizes the role of privately-owned parks as a sort of pseudo-public space, where cryptic rules are enforced by private security guards. The generally unspoken but inferred scripts of acceptable and expected behavior in the smooth city are sometimes, in “awkward cases,” spelled out, as in a sign in King’s Cross Central commanding passersby to “Shop, Eat, Drink, Play” (52). “Smart city” policing, screening, and scripting dramatically reduce the unpredictability of encounters in the smooth city, as do the rise of delivery and e-hailing apps, which replace the chance of encounter with algorithmic manipulation, turning the smooth city into essentially the opposite of what the city has always been, and the opposite of what most urban enthusiasts and critics have tended to celebrate. However, this may very well appear “perfect” to those who can afford it, and who can follow the script.

B has some interesting passages on the way the smooth city interacts with history and place, through a sort of recasting and hollowing out – place names are preserved, as often are historic façades while the interiors are gutted and redesigned. [I was recently in just such a building in downtown Phoenix, across the street from the Footprint Center, the wind-rippled tile “skin” of which is a rare, actually beautiful (and presumably ecologically beneficial) example of contemporary architecture. Yet, much like with King’s Cross Central, a former industrial zone has been gutted and repurposed, to create a safe, smooth space for the “Shop, Eat, Drink, Play” set.] With an eye for material detail, B discusses the acid-cleaning of historical bricks, the replacement of older glass panels with up-to-date glass, and the transformation of Amsterdam streetscapes with new materials according to the rules of the “Puccini method” (24). He concludes with a discussion of how the production, maintenance, and replication of the smooth city is embedded in flows of capital, and how it relies on non-smooth or less-smooth spaces in which it exists in a hierarchy. Examples of these are the peripheral neighborhoods to which those displaced by gentrification have had to relocate; from these neighborhoods come the daily flux of workers who maintain, clean, and labor in the smooth city, and yet are excluded from local politics and decision making. This is true also on a global scale:

The complicated-looking intercom on the gasholder apartment building in King’s Cross Central was assembled in a factory near Shenzhen a few years ago, and will be disassembled by kids in the open wastelands near Accra a few years from now. (57)

The chapter also includes an inset of several pages of collages, using images presumably from the street spaces described, which highlight the attraction and repulsion exerted by the perfected/alienated space of the smooth city.

Labor and Monopoly Capital, Chapter 5

Summary of Chapter 5: The Primary Effects of Scientific Management

 

Braverman expands on the effects of scientific management covered in the previous chapter. Some interesting hauntological imagery, with the paper shadow world that mimics and supplants that of real labor. The paper shadow world is equally important for production, and this perhaps gives it an agency-stealing effect [this is my thoughts on how it links to the uber-hauntology piece]. Taylor is quoted objecting to the idea that his system turns worker into "an automaton, a wooden man."

Braverman does not use this terminology, but he talks about how Taylor's defense of the up-skilling of his system (workers move up a ladder, and new workers come in at the bottom) is really about formal subsumption masking real subsumption, and in fact scientific management practices move up, degrading and controlling workers higher and higher up the hierarchy, including white collar. He emphasizes the growing gulf between science and labor, and notes the ties that used to exist between labor and science. He shows how craft journals were critical of this process as it was happening, and their critique of Taylorism. Implicit, perhaps, is the idea that such organic critique is made less possible as the conditions change and labor becomes less and less skilled. Newer workers are unaware of the skills, etc. that preceded their entry: they are not aware of what has been lost [highly relevant to “sharing economy” app workers].

 

 

Comments on the Society of the Spectacle, Part 2

Summary of second 11 theses:

In this middle of the book, Debord sounds a lot at times like an angry, ranting conspiracy theorist. The primary point of argument is about "disinformation", which is not so much disinformation spread by the government, as the idea of disinformation, or the name, applied to any information or [articulation] that contradicts the happiness or inevitability of the spectacle. (Today, however, we can talk about “disinformation” in favor of the spectacle, as well). The concept of secrecy is repeatedly emphasized, that everything is secretly controlled [apparently a key aspect of the integrated spectacle.] Various experts are exposed as dupes, and so on. There is potentially some important insight or so in here regarding the present "post-truth" moment and the public sphere response to the loss of credibility of the state and scientific experts, although Debord would presumably deny that social media etc. (which he did not see coming so far as I can tell) counts as a "public sphere."

In a perhaps tongue-in-cheek moment he talks about conspiracy theory, as another concept used to delegitimize alternative explanations. In this he stands in a contrast with, e.g., Latour, who uses the existence of conspiracy theories to undermine the legitimacy of “critique” itself. Debord is probably right here – it is lazy and self-defeating to invoke the bugbear of the “conspiracy theorist,” but what is a better term, a better way of talking about the self-claimed “skepticism” of climate change “skeptics,” and so on? There is a temptation (which I succumbed to at the end of my previous post) to fall into the language of “real” vs. “fake,” but I’m wondering if Debord’s own insistence on clinging to an idea of the “true” which he opposes to the “falseness” of the spectacle – which seems all the weaker and outdated a strategy in the context of the integrated spectacle, and even moreso today – shows how insufficient such language is, or has become.

Rhythmanalysis in Taxicabs and Soft Cabs: a report from three North American cities

Here is the abstract of my latest publication, a chapter in Rhythmanalysis: Place, Mobility, Disruption and Performance, edited by Dawn Lyon, and just now out in print:

Rhythmanalysis in Taxicabs and Soft Cabs: a report from three North American cities

Just who is the “analyst” who practices rhythmanalysis? The extension of the name “rhythmanalyst” to other than scholarly practitioners makes possible an investigation of the relationship of rhythmanalysis to other rhythm-analytic forms of knowing and representing urban space, and the ways in which these differing but related practices may challenge, undermine, or inform each other. In this paper, drawing on years of ethnographic and autoethnographic research in three North American cities, I discuss the rhythmanalytical practice involved in cabdriving, as this is shaped by the technologies drivers use to sense the city, and by the transformation of the taxicab into the “ridesharing” or soft cab. First, I discuss the occupational knowledge and wayfinding practice of cabdrivers, and the extent to which their work requires the development, by means of a variety of tools and practices, of a sense of the city as composed of multiple interacting rhythmic movements, or polyrhythmia, with which they must strategically converge and facilitate. Second, I discuss the redelegation of the role of rhythmanalyst to predictive algorithms and mobile interfaces, as part of the reinvention of the taxicab, and its associated micropolitics and power/knowledge relations, by smartphone enabled hailing and dispatching services. Struggles over, and transformations of, these non-academic forms of rhythmanalysis may provide insight, in turn, into the contemporary politics of the production of social space.

Digital Platforms, Porosity, and Panorama

I published an article in the Platform Surveillance issue of Surveillance & Society back in March, but was so busy at the time that I neglected to post it here. Here is the abstract and link to the full text online:

https://ojs.library.queensu.ca/index.php/surveillance-and-society/article/view/12937

The concept of porosity, developed by Walter Benjamin and Asja Lacis, is proposed as a useful concept for examining the political, social, and economic impacts of digital platform surveillance on social space. As a means of characterizing and comparing how interconnected spaces are shaped through a diversity of interfaces, porosity bypasses a simplistic distinction between analog and digital technologies without losing sight of the actual material affordances, social and surveillance practices, and politics that these differing and interacting technologies enable. As part of Benjamin’s project of uncovering the tension between the present and the utopian visions that capitalism repeatedly invokes through new technologies, an attention to the politics of porosity can situate the effects of digital platforms within the ongoing history of struggle over the production and experience of urban space.

Soft Cities, Old and New

Here is the abstract for the paper I'm presenting tomorrow at the AAG meeting in DC, as part of an interesting panel on Digital Urban Revolutions:

How best to theorize the “smart” or data-driven city, without fetishizing the digital-analog divide? In this paper I turn to the image of the “soft city,” invoked by Jonathan Raban (1974) as a foil to the “hard,” planned and governed city of 20th Century modernity. For Raban the soft city is the complex and mercurial lived reality which eludes governance, and even representation. David Harvey (1990) argued that the “soft city” marked a loss of faith in grand modernist narratives, and in the “hard” technologies of governance and progress (indeed, almost all of Raban’s examples of the “hard city” are Latourian immutable mobiles). Harvey denounced Raban’s subjectivism for losing sight of the power structures shaping both hard and soft cities, and thus foreclosing the potential for revolutionary critique. In recent years the image of Raban’s “soft city” has gained renewed attention by advocates of digital platforms for managing and organizing urban space, favoring “soft” regulation by software-enabled platforms (Hill 2010, Skelton 2016). Ironically, advocates for the “new soft city” express a hopeful confidence in the new soft technologies of governance that are rolling out to replace the old hard technologies which Raban originally criticized. I argue that these multiple views of the soft city—Raban’s, Harveys, and the “new soft city”—can be taken together as one ambivalent locus of discourse, what Foucault called a problematization, which situates both the liberatory potential and the dystopian perils of the digital city within a longer history of technologies and politics in urban space. 


Click here to see the abstracts for the full panel.

Digital Mediation, Soft Cabs, and Spatial Labour

The new special issue of Digital Culture & Society on "Mobile Digital Practices" has been released, featuring my article on "Digital Mediation, Soft Cabs, and Spatial Labour:"

https://www.degruyter.com/view/j/dcs.2017.3.issue-2/issue-files/dcs.2017.3.issue-2.xml

Click here for the free repository version of the article.

Abstract

Critics of digitally mediated labour platforms (often called the “sharing” or “gig economy”) have focused on the character and extent of the control exerted by these platforms over both workers and customers, and in particular on the precarizing impact on the workers on whose labor the services depend. Less attention has been paid to the specifically spatial character of the forms of work targeted by mobile digital platforms. The production and maintenance of urban social space has always been dependent, to a large degree, on work that involves the crossing of spatial boundaries - particularly between public and private spaces, but also crossing spaces segregated by class, race, and gender. Delivery workers, cabdrivers, day labourers, home care providers, and similar boundary-crossers all perform spatial work: the work of moving between and connecting spaces physically, experientially, and through representation. Spatial work contributes to the production and reproduction of social space; it is also productive of three specific, though interrelated, products: physical movement from one place to another; the experience of this movement; and the articulation of these places, experiences, and movements with visions of society and of the social. Significantly, it is precisely such spatial work, and its products, which mobile digital platforms seek most urgently to transform. Drawing on several recent studies of “ridesharing” (or soft cab) labour platforms, I interrogate the impact of digital mediation on the actual practices involved in spatial work. I argue that the roll-out of digital labour platforms needs to be understood in terms of a struggle over the production of social space.

We need better reporting from NPR about Uber

Here is the rant I just sent to NPR, regarding their recent Marketplace segment, "Why ride sharing companies are absent from SXSW":

As a transportation scholar who has conducted research on e-hailing services, including Uber and Lyft, I was surprised and disappointed to hear Molly Woods’ one-sided reporting from the SXSW conference. Your segment, “Why ride sharing companies are absent from SXSW” is 1) misleading (there are plenty of e-hailing companies in Austin, including both taxicabs and “ridesharing” services), and 2) your segment did not actually address the question of why Uber and Lyft are absent!

Uber and Lyft voluntarily left the city to avoid regulations which the voting public approved of. Regulatory limitations on Uber and Lyft, as well as AirBnB, are based on serious considerations of economic and social welfare—but these were dismissed as “quirky” on NPR, the one network from which we expect a more critical and even-handed perspective, now more than ever.

Just as infuriating were the implications that, for daring to challenge these corporations, Austin is somehow backwards, or non-tech-friendly. While other cities are still stuck with Uber and Lyft, Austin is incubating the next generation of e-hailing services—more responsible, and more accountable than the corporate giants. 

What the world wants to know—and what NPR can more responsibly report on—is how well these new, non-Uber-and-Lyft e-hailing companies are servicing Austin. We all know that companies like Uber are unsustainable. Austin is the place where we see what will happen next—please give us some reporting on that. 

I normally try to stay away from comments or emails like this, but this time I couldn't help it. I think I showed great restraint by not even asking them why they are still calling it "ridesharing" (though they must know better by now)...

I haven't looked closely at what has been happening in Austin since my early post about "ridesharing" apps swarming into Austin, right after Uber and Lyft left. It would be great to see some real reporting on how the new, local apps are working out. For a good start at this, see this recent article on Shareable.

A Spectre is Haunting Uber: Jason Dalton’s tale of smartphone possession

We control the horizontal; we control the vertical.

Kalamazoo Uber driver and shooting suspect Jason Dalton’s story of being possessed by the Uber app is only the latest in a long history of such stories, in which people have attributed paranormal or spiritual powers to new technologies. Tales of otherworldly beings communicating through the telegraph, radio, television, or computer screen are motivated by the anxieties that arise with social and political changes driven by new forms of communication and action-at-a-distance. Today, while Uber’s PR department scrambles to keep the phrase “going Uber” from becoming an updated version of “going postal,” it is worth looking more closely at Dalton’s delusions for some insight into the particular fears and dreams of our up-and-coming app-governed existence.

In his book Haunted Media, Jeffrey Sconce describes the long history of stories of possession and paranormal activity surrounding new and unfamiliar technologies. The telegraph and radio gave rise to stories of spirit possession and the entire phenomenon of the spirit “medium:” a human who, not unlike a radio, was “tuned” to frequencies through which they could talk to the dead. Television and the internet inspired stories of mind control, alien invasion, and being trapped in worlds of illusion. In each case, the paranormal stories that have swirled around new technologies boil down to the hopes and fears these technologies inspire, and such questions as:

• how can you talk with someone who isn’t present?
• how can these images seem so real when we know they are not?
• how can we make sense of this invisible power that flows all around us, and through us?

We may laugh today at people being afraid of telephones and radios, but Dalton’s story owes more than a little to contemporary cultural anxieties over the increasing saturation of our lives with apps designed to influence, and to some extent to control, human behavior. Though exaggerated by his paranoia, each of Dalton’s crazy claims reflect the actual controls and suggestions made by the real Uber app.

In his interview with police, Dalton made these claims:

1. Dalton saw an “Eastern Star” or “devil head” in the app.

2. The app triggered Dalton's actions with colors and sounds.

3. Dalton described possession by the app as more of a “feeling” than a “telling.”

4. Dalton felt that the app was telling him where to drive.

5. Dalton felt that the app gave him special abilities or protections.

6. Through the Uber app, Dalton felt connected to some greater, inexplicable power.

Each of the quotes below (in italics) are from the interview notes made by officers Moorian and Ghiringhelli, and made available by WZZM in Kalamazoo.

1. Dalton saw an “Eastern Star” or “devil head” in the app.

Dalton said that if we only knew, it would blow our mind. Dalton then explains how when he opens up the Uber taxi App a symbol appeared and he recognized that symbol as the Eastern Star symbol. Dalton acknowledged that he recognized the Uber symbol as being that of the Eastern star and a devil head popped up on his screen and when he pressed the button on the app, that is when all the problems started.

Uber did just change its logo, but neither the old nor the new logo matches the “devil’s head” described by Dalton. Nevertheless, as Uber drivers have already started pointing out over at uberpeople.net, there are in fact upside-down five-pointed stars (as well as rightside-up ones) all over the background of the newly-designed app. Dalton seems to have fixated on this.

TruYouber: Sure, the new Uber app is covered with up and down-facing pentagrams. But isn’t it more disturbing that it is clearly modeled after the logo of the world-conquering corporation in the dystopian Dave Eggers novel, The Circle?

It was not enough for the devil’s head logo to simply be there: Dalton himself had to speak its name for it to take power over him. When he recognized the symbol and “spoke what the symbol was,” it responded (he claimed) by turning from red to black.

Dalton said that when the Uber symbol is red, it is just picking up and dropping off people, but when he recognized the symbol and spoke what the symbol was, the color changed from red to black.

Dalton said he wishes he would never have spoken what that symbol was when he saw it on his phone. Dalton described the devil figure as a horned cow head or something like that and then it would give you an assignment and it would literally take over your whole body.

Dalton said that if he wouldve never ever mentioned the Uber symbol resembling the Eastern Star, he never wouldve had any problems.

2. The app triggered Dalton's actions with colors and sounds.

Dalton was asked what was different tonight from the other nights and he said as a driver partner with Uber, the icon is red and it had changed to black tonight.

The red-to-black shift which Dalton reported seeing is a bit harder to explain. On a normal, non-possessed Uber driver app, the screen does go black—right before a ride request, after which the screen zooms in on a blue circle centered on the hailer’s location, while a ringing/beeping sound alerts the driver to touch the screen to accept the ride. Dalton reported such beeping when the app was taking control of him.

I asked Dalton why the system allowed him to stop for the officers and Dalton said that he didn’t know. Dalton then told us that he did know one thing, that when the system switched from black to red and when the officer was about to say something to him it went beep beep beep for Dalton to log back into the system. ... Dalton said that when the system switched back is when Dalton got his presence back.

The Uber app is, of course, designed to influence driver behavior through the control of information, and through certain visual and audio cues; and Uber does have a history of experimenting on driver behavior by tinkering with the app. Nevertheless, it is probably safe to assume (barring further revelations) that Dalton hallucinated this whole red-to-black shift.

Dalton said that as soon as the police officer stopped him tonight, the symbol went from black to red and he felt like he was no longer being guided. Dalton said that was the reason he didnt shoot the officer because the app went from black back to red. Dalton explained that when the symbol turns to black, it literally has control over you. I asked Dalton why didnt he just uninstall the app and he said it sort of had you at a certain point.

3. Dalton described possession by the app as more of a “feeling” than a “telling.”

Dalton said it also told him to be available all the time. ...he said it wasnt like a telling, it was more of like a control. ...Dalton said that Uber requires a driver to have a car newer than 2007 and when you plug into it, you can actually feel the presence on you.

Significantly, Dalton said that the app didn’t tell him what to do; it rather took control of him through a sort of feeling of presence. This makes sense, because this is just how algorithms influence human behavior, by feeling or intuition, rather then “telling” per se. Paranoias about receiving instructions are so last century—befitting antiquated technologies like radio or television. Today, instead of being given instructions, we rely on algorithms working in the background to guide our behavior; apps like Uber work like video games, by giving users a circumscribed freedom of action within which we intuit or “feel out” the algorithms which assign value to our actions. McKenzie Wark calls this an “intuitive relation to the algorithm;” the most successful game players, or Uber drivers, are those who have “most fully internalized” the algorithm.

Dalton certainly internalized the algorithm; unfortunately, he seems to have confused Uber’s taxi game with a FPS.

Dalton said that he could only tell us that it has the ability to take you over. We confirmed with Dalton that he was referring to the Uber app and Dalton said yes. Dalton then told us that it feels like it is coming from the phone itself and he didnt know how to describe that. ... Dalton said that as he was sitting there with us, it was almost like artificial intelligence that can tap into your body.

Dalton then said that is why he is trying to tell us it is like an artificial presence.

Dalton said that it would take you over to the point that you are like a puppet.

4. Dalton felt that the app was telling him where to drive.

This one is hardly surprising. Uber driver apps are automatically integrated with Google Maps or with Waze, and while Uber drivers are not technically required to use and follow GPS, they are strongly encouraged to do so. Dalton seems to have interpreted this suggestion as mandatory.

I asked Dalton where he was headed when he was stopped and Dalton said that the system was telling him to drive. I asked Dalton if he knew where it was telling him to drive and Dalton said that the system was literally telling him to just take turns (as he made a motion with both hands on a steering wheel making turns).

Dalton said that it starts out that you have to follow the navigation, but it gets to the point where you dont have to drive at all, the car just goes. Dalton said that as long as you have a 2007 or newer car, your phone can link through your car.

Great news for driverless car fans: there is no need to wait five or ten years for scientists to develop self-driving cars when Uber can achieve the same effect right now through the magic of spirit possession!

5. Dalton felt that the app gave him special abilities or protections.

This is one of the most interesting aspects of Dalton’s story. Just like in any deal with the devil, you lose control of yourself, but you gain certain perks in return.

Dalton then told us that when the app would turn from red to black and it was a 5 star driver that is when it was telling you you could drive just as fast as you wanted to.

This tallies with the stories told by several of Dalton’s passengers, that he drove insanely fast, and blew through stop signs and stoplights. The app, apparently, was giving him superhuman driving powers and privileges.

Dalton said that the Iphone can take you over. Dalton explained how you can drive over 100mph and go through stop signs and you can just get places.

The five-star rating system is one of the means whereby Uber (and its similar competitors) encourage drivers and passengers to feel like they have some power within the system. Dalton seems to have taken this very seriously:

Dalton explained how there is a customer service score on Uber and when he tapped the button, he could say anything he wanted to about the person and it would be anonymous. Dalton then said that he could hear other peoples phones ding and their score or rating would go down.

6. Through the Uber app, Dalton felt connected to some greater, inexplicable power.

Dalton attributed great knowledge and power to the Uber app, or some greater power that it was “attached to.”

Dalton said he was seeing himself from outside of his body. Dalton said that this thing knows where everything is through your phone. Dalton said that it knows everything and when I asked what it was he said whatever Uber is attached to.

Dalton said that there is something bigger than Uber just picking up people and dropping them off.

Isn’t this exactly what Uber’s CEO has been claiming all along?

The New Spooks

Dalton then told us that he is not a killer and he knows that he has killed.

Let’s go out on a limb here and assume that the Uber app did not make Dalton shoot all those people. He did it himself. He was bonkers, and confronted with the horror of what he himself had done, he rejected his own actions and blamed them on the conveniently available construct, the “app.” Which we all know to be an uncanny, and untrustworthy, interloper in our social relations. Jason Dalton thought he was being controlled by the app, but, in truth, he had split himself in two—one half a helpless puppet, haplessly looking on while the other half, the ghost in the machine, wrought mayhem.

Or maybe it wasn't Dalton who split himself in two. The very working of the app involves the tracking and profiling of a "data double," a spectral data-Dalton corresponding to the human Dalton, and through which the human Dalton can be tracked, profiled, and manipulated. And Dalton isn’t the only person having trouble telling where his own actions end, and algorithmic controls begin.

Apps like Uber (and Google, Instagram, etc.), through which algorithms massage us, are popular because we embrace the controls they exert on human interactions. They really do seem to know everything, or at least a lot of things. They promise us great new powers, at a (Faustian) bargain. In Uber’s case, the app provides a preprogrammed set of social roles—driver, passenger—into which actual humans can be plugged-in, interchangeably. The app promises freedom, while delivering stress, exploitation, and constant surveillance. Both YouTube and the news are full of videos of drivers having "Uber meltdowns" in which they quit the job, often spectacularly—though thankfully, not as bloodily as Dalton did.

Dalton's tale opens up all kinds of hauntological questions about the dawning algorithmic era. To what extent was it all his own paranoid delusion, and to what extent the new experience of app-enabled alienation? Haunted by our data shadows, all of our senses of individuality and identity, of agency and responsibility, may be scrambled and shuffled by the rollout of socially mediating algorithms. Will we recognize the future that is created as our own doing, or attribute it to the grotesque ideas of an algorithmic brain?

[Your Job] is a Video Game

Video games, and video game theory, provide insight into the ways ubiquitous mobile computing will be used to transform social interaction.

Cabdriving is a video game!

K-ching! K-ching!

That’s the name of the game!

– MC Mars, “Cabdriving is a Video Game”

“I’m going to Yerba Buena Gardens,” said the friendly young man. “You know where that is, right?”

Of course I know where that is, and as luck would have it, we were only a few blocks away. With a few taps of the screen, I navigated my car south and east from the Tenderloin, crossing Market at Fourth and coming up along the main entrance to Yerba Buena Gardens on Mission Street. Nothing happened; the passenger did not get out.

Damn, is this game frozen again? There should be a flag here to mark my passenger’s drop-off location. I pinched and spread the map to zoom out, and then saw it: the flag was planted way over in the intersection of Third and Howard, on the far opposite corner of the park.

Oh, of course. I was playing UberDrive, Uber’s new cabdriving (okay, “ridesharing”) simulation/game/recruiting tool. My virtual passenger, I realized, could not get out anywhere but at whatever specific location Uber’s GPS imagined to be “Yerba Buena Gardens.” Unfortunately, most of the streets around here are one-way, in the wrong direction. I maneuvered around three long city blocks, finally tapped the flagged intersection on the map, and watched the little car icon pull up. “Best ride ever!” beamed the passenger. Another five star rating!

According to Uber’s website,

UberDRIVE was designed as a fun and engaging resource for our driver-partners to hone their navigation skills if they choose to. It’s also a great way for prospective drivers to experience firsthand what it’s like to drive with Uber –– there are links to sign up and start the screening process from directly within the game.

Hone your navigation skills? Tapping intersections on a map is a far cry from the experience of actually navigating through San Francisco’s downtown streets. I suppose UberDrive could help inexperienced drivers learn which streets are one way, and how the street grids connect across Market. Experience firsthand what it’s like to drive with Uber? As a training tool, the game is hardly realistic: every single passenger was polite, no one ever cancelled, no one threw up in the back seat, or tried to squeeze in more passengers than seatbelts; everyone was at their pinned location and ready to go when I rolled up. And every single passenger gave me five stars. In other words, they were completely unlike many Uber passengers.

UberDrive is, in fact, just a poor copy of a much more interesting game – driving for Uber. The main innovation of Uber, and other smartphone-enabled “e-hailing” car services, is the insertion of a new interface into the human-to-human, on-the-street interactions between drivers and passengers. For drivers, the smartphone screen works like the little map in the corner of a first-person video game, a HUD that links the immersive environment of the city street back to the digital space of the game world. Each "ping" that alerts drivers to incoming hails is accompanied by a game-like cutscene showing concentric circles radiating from the hailer's location; drivers can log in to a "dashboard" offering "Partner Rewards" such as discounts on gas and oil changes, which drivers "unlock" by completing a set amount of quests... I mean, rides...

The interface allows the game designers at Uber, Lyft, etc., several tools for influencing the in-game behavior of both drivers and passengers: “surge pricing,” the five-star ratings system, and most importantly, the affective framing of “ridesharing” as different from (“Uber” than) the mundane experience of riding in a cab. This isn’t like those old taxicabs with their “inconvenient meatspace hailing”: it’s interactive tech, you know, more like a video game.

There is a name for this combination of storytelling appeal and software-mediated control: the allegorithm, which means the unity of allegory and algorithm. This unity comes into play when, for example, a player satisfies a game’s algorithms by hitting a series of keyboard buttons with precise timing, while, within the storyline or "allegory," they embrace the idea that they are killing orcs with a flaming sword. Or maybe they tap a smartphone screen, while imagining that they are driving passengers around SoMa. But the allegorithm really comes into its own when it is deployed with mobile interfaces into “augmented reality.” Ingress, you are already thinking; but you should really think Uber.

For the inelegance (and questionable pronounceability) of “allegorithm”, you can blame the word’s coiners, video game theorists Alexander Galloway and McKenzie Wark. But as a concept, it is of great importance for understanding how mobile interfaces are already connecting with and transforming social interaction, and how they will increasingly be used to do so in the near future.

By saying that allegorithms transform social interaction into a “video game” I do not mean that they are making the “real world” somehow “less real” or less serious. The goal of gamification is not the distortion of reality behind some kind of mystifying curtain or spectacle, but the improvement of “meatspace” reality through the deployment of design lessons learned from game development. In practice, this means carefully manipulating the kinds of information and choices available to players, studying their motivations in order to encourage desired behaviors, and inventing a compelling storyline through which players can make sense of the "game." The results can range from the benign to the sinister, from the sublime to the laughable, and we will be seeing more and more of these as the revolution in ubiquitous, mobile computing continues to roll out.

In the meantime, it can be useful to observe Uber’s allegorithmic gamification of cabdriving to assess how these initiatives will play out, and what successes or failures they are likely to encounter. The allegorithm needs, above all, a narrative which participants want to, and are able to, buy into; Uber (and its imitators) have shown great success with this so far, but how long can this be kept up? With any ubiquitously deployed allegorithm, the question arises as to how well the framing survives its insertion into the immersive “real world” environment. As I will discuss below, the success of the cabdriving-game deployed by Uber and others like it depends on the already gamelike aspects of cabdriving as a job; but risks coming apart when running up against the contradictions of cabdriving as work, and when the game designers, or the allegorithm itself, fail to predict or account for the real world complexity that players will encounter.

And as with any social transformation, we need to keep questioning. How good are these games? Are these the games we really want to play?

Crazy Taxi: from digital game-world to physical-world toy. Creative Commons photo by Tatton Partington.

Cabdriving was, of course, already a video game – Crazy Taxi – and the story I was told goes like this. Three San Francisco cabdrivers had the idea to design a board game based on their job. They designed a board loosely based on San Francisco, with pieces that looked like cabs, which the players moved about town looking for paying customers, the winner being the one who made the most money in a set number of turns. Then one day one of the designers (the driver who told me the story) was told by his partners that they had sold the idea to a game company for a few thousand dollars. A few years later, Crazy Taxi hit the arcades, though as a video game rather than a board game.

True story? It is hard to tell. Sega (producer of the game) attributes the idea to a Japanese game designer, not three San Francisco cabdrivers. Yet the original game in the franchise was clearly based on San Francisco, complete with its vertiginous hills, cable cars, and a choice of three bohemian taxi drivers as avatars. And the game captures at least some of the fun part of driving a cab in San Francisco. When I quit driving, I played the game as a way to enjoy some of the addicting aspects of the job. It was sort of like a nicotene patch for cabdriving.

UberDrive, in contrast, is designed to be a gateway drug—a free sample to get you hooked, to pull you into the deeper game of “meatspace” driving. Every few minutes the gameplay is interrupted by an appeal to start driving for Uber in real-life. “No Thanks,” I click, and go back to playing pretend cabdriver.

These games work because real-world cabdriving is, inherently, in many ways game-like. It is full of short, achievable objectives and quick rewards, won in exchange for taking calculable risks. It is more than a little like gambling, and can be as addicting, given the right personality. No matter how bad your luck gets, the player knows there are new possibilities waiting around the corner.

Driving in the city requires, besides patience and strength of will, ingenuity at puzzle-solving; as a passenger once observed while I threaded through traffic, “So you just basically play Tetris all day?” Or as San Francisco’s hip hop cabdriver MC Mars puts it:

cutting through lanes like fish veins

on Van Ness with the finesse of a sushi chef

Truly/deftly/ I’m the surfer with the motorcycle mentality/ when it gets gnarly/ I hit the slot between the herd and the Harley/ tourist with his head in a map/ kid on a raging Ducati/ cab driving is how I rock a party

                                           (Mars 2005: 144).

Cabdriving may be gamelike, but it isn’t all fun and games. I titled my 2004 cabdriving auto-ethnography Playing for Hire to emphasize the dual sense of the taxicab driving experience in San Francisco. On the one hand, drivers constantly talked about their job as “playing” – “playing” the airport, “playing” the dispatch radio, “playing” the streets. On the other hand, this “play” is also “work,” and drivers need to make money—to pay the cab company for the vehicle, to pay for gas, to pay off all the numerous gatekeepers who must be tipped in the course of a shift, and finally, if any is left over, for themselves. K-ching! K-ching! is the name of the game.

When MC Mars raps about cabdriving as a video game, he doesn’t paper over the bad or candy-coat the danger. He paints an image of cabdriving as a difficult job which demands and rewards the development of individual skill, “split-second reactions” and bravado. It's a view from the streets: a less controlled, messier, and more enjoyable experience than UberDrive, or for that matter, than Uber.

Uber, and other smartphone-interfaced cab services, build on the already-gamelike aspects of cabdriving, while relying on allegorithmic design to ameliorate or obscure the job’s less gamelike, real-world difficulties. Before Uber, cabdriving was already gamelike, and it already had its interfaces: radio and computer dispatch, even smartphone apps like Taxi Magic and Cabulous. But before Uber and Lyft the allegorithmic potential of these interfaces had not been developed. UberDrive shows us how this is supposed to work: happy drivers transporting friendly customers, and making quick money. K-ching!

If any UberDrive players are suckered in enough to click “Yes” and actually join Uber, they are in for a big surprise. UberDrive is not so much a view of the actual experience of the Uber driver, as a peek into the fantasy of the Uber designer, and their vision of how the allegorithmic narrative should be playing out: a superficially cleaned up, “tout est pour le mieux” version of reality, presenting the voyeuristic perspective of the overseer, rather than the in medias res viewpoint of the driver in the street. Having played both UberDrive and the real-world game it is based on, I have to say the designers are missing a lot. Could this blindness, in fact, be an effect of that other computer game, played by Uber personnel, looking down on the city from their computer screens at Uber HQ, playing Uber’s “God View?”

"An Icarus flying above," the Uber GodView player looks down on "the devices of Daedalus in mobile and endlesss labyrinths far below" (de Certeau 1984: 92). Image by Uber.

Perhaps UberDrive’s designers could make a better game if they got out on the streets and drove a bit. Perhaps Uber’s designers could benefit from the same exercise. Perhaps the folks at Uber HQ could make better games in general if they pondered a bit more critically how their own game’s allegorithm is shaping their experience. They do know they’re playing a video game... don’t they?

Michel de Certeau, (1984) The Practice of Everyday Life. Berkeley: University of California Press.

MC Mars, (2005) Don't Take Me The Long Way: 30 True, Truly Outrageous Cab Stories. San Francisco: Off D Edge Press.

“Shared Autonomy” or “Control Society?”

An interesting chart has been making the rounds on the internet and social media. Created by Adam Jonas, a financial analyst at Morgan Stanley, it purports to show “The Four Stages of Mobility,” outlining a transition from today’s car-centric model to tomorrow’s even more car-centric (yet “autonomous”) model.

Jonas charts a movement from 1) Today’s system of automobilty, through 2) the growing “Shared Economy” represented by companies like Uber and Lyft, and 3) a coming stage of “Owned Autonomy” made possible by self-driving cars, to 4) the final stage in which these trends come together as “Shared Autonomy,” in which nobody owns cars, but we are all instead shuttled around cities in driverless taxis straight out of a Robert Heinlein novel.

There is one big problem with this: if these trends continue, the future they are taking us towards will not be as rosy as the term “shared autonomy” suggests. Because although Jonas has correctly identified two important trends which are taking us along this path, in the chart above these trends are mislabeled – and more importantly, misunderstood. In fact, even the present is not properly represented on the chart.

Here is a more accurate version of the chart:

1. Today. Jonas, not surprisingly, chooses to trace “today’s” model back 100 years, presumably aiming for the “Model T moment” in which technological innovation and the factory system came together to put out a mass-produced, affordable automobile. This was of course an important historical event, but the car industry, and the economy as a whole, moved away from the resulting “Fordist” system a long time ago. A better description for our current system is the “Post-Fordist” condition beginning about 40 years ago. This is not nitpicking. The shift from Fordism to Post-Fordism came with an erosion of workers’ power (goodbye unions) and the resulting “flexibilization” of the workforce. Not coincidentally, this occured along with a flat-lining of real wages, leading to an increasingly unhealthy reliance on lines of credit.

Which means that that “Asset Owned” side of Jonas’s left-right continuum should instead be labelled “Commitment through Debt.” We like to think we own our cars, houses, etc., but often the truth is we owe them.

Recognizing Post-Fordism is important because it sets the stage for the two major trends which, according to the chart, are leading us to the future. These are not as new as we might be tempted to think, but are the continuation of trends which have already been in operation for decades. They represent, not a break with the present, but its continuation into the future.

2. It takes a bit of preternatural naiveté, or obstinacy, for someone to still insist, in 2015, that the so-called “Sharing Economy” has anything to do with sharing. Perhaps for this reason, Jonas prefers the term “shared economy” and highlights the “sharing” of assets made possible by pseudo-taxi companies like Lyft, Uber, and Sidecar. The obvious question arises – if using Uber counts as “sharing” assets, shouldn’t riding the bus, streetcar, or even a plain old taxi, as well? If there is anything new about the “sharing economy,” it isn’t the very old habit of “shared assets.” So what is driving this trend?

What really is going on, is a shifting of economic risk from corporations to workers; the Post-Fordist system taken to the next level. This is Precarious Society. Uber doesn’t have to own any cars, or even hire employees (if they could get away with it, they wouldn’t pay for insurance, either). While Uber, reportedly, rakes in untold riches, the drivers bear all the costs, and risks, of doing business. This is certainly a growing economic trend today, though there isn’t much about it to celebrate.

I’ve labelled this side of the left-right continuum “Access through Credit.” You may be wondering, how is this really different from “Commitment through Debt?” “Credit” and “debt,” of course, are two sides of the same coin. You can’t have one without the other. But there is one thing about coins—it is hard to see both sides at the same time.

So think of these two poles, not as a qualitative opposition, but as a reference to which side of the credit/debt coin is face up, and thus made visible. What makes services like Uber and Lyft convenient is the easy access they provide for any rider with credit. That this credit is also debt is obscured (by getting rid of the physical exchange of money at the end of the ride); even more importantly obscured is the debt drivers commit to as they drive us around in their “own” cars.

3. Jonas’s next stage is “Owned Autonomy.” Having disposed of the chimera of “ownership,” let’s focus instead on this word, “autonomy.” As attractive as it sounds, the term is being used not to refer to the “autonomy” of riders in driverless vehicles, but to that of the vehicles themselves. Apparently “they” will no longer need “us.” It’s a vision straight out of those unintentionally dystopian commercials for the “internet of things” in which all the important business is automated while humans are reduced to standing around, looking useless and disoriented, and occasionally getting in the way.

But really, “autonomy” is still not the right word for it. Just as the old-fashioned “automobile” was never truly “auto-mobile,” but relied, not only on human drivers, but an entire concrete infrastructure built into cities and smeared across the countryside, so the interconnected “autonomous vehicles” of the future will be even more dependent on the interconnected systems of which they are part. To see this as “autonomy” is to miss the deeper reality, which will be control. Which is why the important movement reflected in the chart’s up-down continuum is not away from “Human Drivers” to “Autonomous” cars, but from a relatively decentralized system (which relies on large numbers of people knowing how to drive) to an increasingly centralized system (relying on the specialized knowledge of a small number of people who design and manage the system).

And why do we need these “autonomous” vehicles in the first place? We can of course hope that driverless cars will be safer, and more ecologically sustainable than cars today, but that is not at all the reason why they are being built. Self-driving cars are just a benchmark along the path to computerized systems that can solve complex, real-world problems on the fly. The biggest reason to want to create such a system is to reduce the power of labor, by reducing the knowledge that workers are required to have to do any given job. This goal – increasing Control of Knowledge – is the whetstone that hones the cutting edge of today’s push toward automation, including the “robot car.” More important than the technology is the desired product: a de-skilled workforce provided with just enough information to complete tasks, but not enough to exert control over their own working conditions.

4. Finally, taking the “shared” from “shared economy” and the “autonomy” from “owned autonomy,” Jonas leads us to “Shared Autonomy.” This is certainly a very attractive term, invoking a utopian (“autopian,” says Jonas) blend of both individual self-governance and social responsibility. It evokes the ideal society long championed by visionaries as diverse as Karl Marx, Mikhail Bakunin, Jesus Christ, and Winnie-the-Pooh. And me too! It sounds great, bring it on!

Unfortunately, since our steps along the way, in truth, involve neither “sharing” nor “autonomy,” but instead precarization and control, the fourth stage of Jonas’s chart would be better termed the Control Society (aka "society of control"). This term was coined by French philosophers back in the 80s, and can be counted among those formerly-paranoid visions of the future which Silicon Valley is in the habit of making come eerily true. Gilles Deleuze imagined that in the society of control:

one would be able to leave one’s apartment, one’s street, one’s neighborhood, thanks to one’s (dividual) electronic card that raises a given barrier; but the card could just as easily be rejected on a given day or between certain hours; what counts is not the barrier but the computer that tracks each person’s position—licit or illicit—and effects a universal modulation. (Deleuze 1992: 7)

Gaining mobility by purchasing a car (“Commitment through debt”) is so last century. “Access through credit” is more in line with the dreams of control society. And what better way to control mobility than through self-identifying devices that we willingly purchase and carry, that buy us access into a network enabled by uninterrupted connectivity and surveillance? Meanwhile, the centralized, expensive high tech of the “autonomous” vehicle means that the provision of mobility is likely to consolidate in a few, well-funded, powerful hands. Jonas lists some likely suspects: “Google, Apple, Uber 2.0” (avert!).

Now how utopian does this sound?

The emerging tech of driverless cars makes an interesting point of entry into this question of where our society is going, but we shouldn’t blame the technology itself. The problem isn’t driverless cars, but why we think we need driverless cars (not to mention, cars), why we (in general) are so easily roped into supporting a vision of the future that, should it arrive, will benefit the few more than the many, and be founded on the same irrational, unquestioned presuppositions that underlie our current economic system. It is high time we looked more critically at emerging technologies like the driverless car (and even more importantly, ubiquitous computing), instead of taking their inevitability for granted.

Henry George, writing in 1868 about the increasing penetration, and power, of the railroads in his own time, issued a warning which is just as accurate now as then:

And this in general is the tendency of the time, and of the new era opening before us: to the great development of wealth; to concentration; to the differentiation of classes; to less personal independence among the many and the greater power of the few. (George 1868: 306)

Deleuze, Gilles, “Postscript on the Society of Control,” October, Vol. 59. (Winter, 1992), pp. 3-7.

George, Henry. “What The Railroad Will Bring Us.” The Overland Monthly 1, no. 4 (October 1868): 297–306.