Image + Affect

The overlap of cinematic theory and video game theory is perhaps most valuable when considering the image-based facet of each.  The brain’s rational, logical, and precise processes are fathomable and the algorithms we have built illustrate this, but the ways in which emotions work are far more mysterious. Images tend to operate on the affective level in ways that are sometimes hard to predict and always difficult to discuss in rational (linguistic) terms. Riffing off of Roland Barthes, Gregory Ulmer describes the concrete impact of the punctum: When viewing particular images, he asserts, “I have a very real experience in my body, an emotional pathic response, and it is at this point of the me that the discursive abstract information and my unique existence overlap” (62). The nature of this “discursive abstract information” attempts to account for the differences between more generalized cultural response to images, and the more personal and often idiosyncratic reactions based in one’s lived experience.

In a slightly different approach, Brian Massumi distinguishes between affect,which is autonomic, collective, and atmospheric, and emotion, which is personalized, namable, and consciously problematized. Unlike the studium-punctum formulation, Massumi explicitly separates image content from image effect, suggesting there is no straightforward relationship between the two. This concept leads to political quietism since the assumption of an autonomic response, one that lies below the threshold of cognition, can then be seen as idiosyncratic, such that no theory can be applied. Further, as Ruth Leys points out, content is then not ideologically imbued. In other words, this approach, by extension, removes any ethical onus from the image creators.

But even as the operational details of the process remain unclear, we do know that emotions trigger physical reactions, making viewing images an embodied experience. Thus the role of affect frequently fuels neuroscience research as the brain mediates bodily responses to vision.  Scientists are currently exploring what critical theorists have long known: not all images are equal in terms of the physical reactions they cause and their impact on identity (Arbib, Damasio). And while MRI scanning may not be able to represent human thought, it can shed light on certain emotional responses to images, especially with regard to the role of mirror neurons.

Mirror neurons are those that fire in the prefrontal lobes of the brain when one acts or is acted upon, and they also fire when one only witnesses the same act exerted upon another, especially if the other is similar to oneself.  These neurons also seem to fire when one only sees a representation of the act, implying that under some circumstances, the brain cannot distinguish the experiential from the imaginary (Goldstein).  There is some debate with regard to the role of an inciting incident and whether it must precede the activation of the mirror neuron; it is not clear if one must have experienced the act before one’s neurons will fire when only viewing a similar act, or if they fire absent that initial experience. Must I have felt the anguish of having my fingernails torn off before my neurons fire by watching it happen to someone else?  Obviously this distinction holds implications for mirror neuron response, which is seen as an empathetic one. Still, the very awareness of mirror neurons, when paired with the concept of neuroplasticity—the evidence shows that the way we use our brain effects their material structure as some synapses are pruned away, while others are strengthened—makes attending to what goes into our brains through our eyes a concern, particularly for media scholars.

When considering graphics processing then, exploring human perception via the eye becomes crucial. Thus, there is an ethical dimension to the creation and use of media in general, and of interactive media in particular.  For instance, as graphics processing advances, first person shooter games may work on brains in ways we do not, and perhaps cannot, fully appreciate. Is it ethical to foster gameplay that includes increasingly realistic images and that requires violence for success? Does virtual violence become an outlet for real world frustrations, providing catharsis, or does it create neural pathways that routinize adversarial interactions? And what about the virtual people who populate the landscape of these games? Killing differently-appearing avatars is typically the main goal as "bad guys" are visually marked, but since mirror neurons seem to fire most strongly when one views an act upon an actor who is similar to oneself, how far must an avatar vary visually from a player to make killing it an additional ethical imperative?

Slate editor Michael Thomsen argues, there is a disconnect between the realistic warfare of games such as Battlefield 3, Call of Duty: Modern Warfare, Medal of Honor and Operation Flashpoint, and the erasure of civilians in the gamespace.  Thomsen cites Battlefield 3’s executive producer Patrick Bach who, by refusing to add civilians to the game, says he is “sanitizing” the medium. Bach claims that players cannot be trusted to do good if they can do bad, which necessitates this cleansing, and he blames players for holding the medium back. Thomsen, however, indicts Bach and his fellow developers with holding the medium back, since they refuse to force players into ethical dilemmas wrought by the consequences of the act of harming innocent bystanders.  

What would it mean to make players confront such dilemmas, and would these moral quandaries impact real world actions? Restricting or censoring games is obviously not the answer, and indeed can make them more attractive by virtue of their forbidden quality.  Perhaps some regulation is viable; in 2011 the International Red Cross began talks around the possibility of recommending the application of the Geneva Conventions to gameplay. Critics invoke the triviality of such a move, since “it’s only a game.” They also invoke axioms about “mind control” and “thought police,” while studio executives espouse their reverence for their developers’ artistic vision (one might ask about the degree of finesse possible since the generic conventions of Call of Duty and Battlefield 3 are relatively rigid).  And while limiting free speech is not typically associated with a thriving public sphere, there are certainly precedents for such regulation: libel and hate speech, for example.

A fuller appreciation of the ways in which the digital and the material overlap might clarify the relative need to regulate the sale of large-scale surveillance technologies. These currently remain unchecked, and yet given that their primary raison d’etre is to aid espionage and combat, they ought to be subjected to arms control. Moreover, these surveillance technologies have transformed warfare into an ersatz video game, as remote operators pilot drones, dropping bombs from a safe distance. This places an even darker spin on Call of Duty: Modern Warfare 3, a game that grossed one billion dollars in sales during its first month on the market.

Obviously, game systems could be scripted to punish players who shoot civilians, a move that seems consonant with the ever-more realistic graphics. But even in the absence of the requirement to do so, we ought to be engaging these games far more critically. This includes exposing their producers to rhetorical scrutiny inquiring, for instance, about how they posit their audience. Discussing the extent to which they seem to have zero faith in their customers' collective moral compass may appeal to players’ righteous indignation. We might challenge players to traverse the aporia-epiphany path, wherein an impasse gives way to closure (Aarseth), which may prove far more motivating than remaining in the simplistic environment of this “sanitized” medium.

A reflective approach like this may also enrich cognitive research, as players grow increasingly aware of the impacts of gaming, and better able to articulate their experiences. Research in this area is sorely needed and one avenue that could shed light on the effects of first person games comes by way of researchers working in the burgeoning area of neurocinema; researchers scan a subjects’ brains via fMRI while they watch various films to try and gauge responses, which occur below the level of consciousness. For instance, Pia Tikka is a key researcher behind the enactive cinema project, whose various iterations have included efforts to match the cinematic narrative to the spectators’ reactions, but who have lately focused [http://www.enactivecinema.net/] measuring the responses to tenable situations and contrasting those with reactions from viewing scenes that defy the laws of physics, for instance. If the brain cannot tell the virtual from the experiential, how will it engage when faced with the impossible?



[i] See Parables for the Virtual, page 24 for a more nuanced description of embodied emotion and affect.

[ii] Though there are many accounts of the phenomenon, the best overview of recent scholarship can be found in “All Smoke and Mirror Neurons?” 2009: http://mindhacks.com/2009/05/27/all-smoke-and-mirror-neurons/

[iii] This effort follows a 2007 report issued by TRIAL, a Geneva based humanitarian organization. For an overview of the key issues, see this post in Kotaku, the prominent gamer blog:  http://kotaku.com/5863817/

[iv] A good overview amidst lots of recent press coverage in this article: http://owni.eu/2011/12/01/spyfiles-wikileaks-revelations-of-mass-internet-surveillance/

[v] The LA Times reported this number on December 11, 2011 and the game’s release was November 8, 2011, just over four weeks prior: http://latimesblogs.latimes.com/entertainmentnewsbuzz/2011/12/call-of-duty-modern-warfare-3-clocks-1-billion-dollars-in-sales.html