Introduction

In True Lies, Stephen Prince argues that with the advent of CGI, the ‘indexical’ or causal connection between what is filmed, and the real world has been broken. Today, digital imaging can make unreal things (e.g., dinosaurs, orcs) look real, and can make unreal things fit into worlds that we recognise as real. He goes on to give many examples from films like Jurassic Park, Forrest Gump and True Lies. Prince claims that the advent of CGI challenges traditional film theory assumptions about realism.

He begins by stating that realism in film theory is associated with the concept of indexicality; meaning that there is a relationship between the photographic image and the referent (or the actual thing that was photographed). This idea of indexicality is what separates realism and formalism in film theory. The use of CGI challenges some of the assumptions of realism. Charles S. Pierce argues that photographs ‘are exactly like the objects they represent … they … correspond point by point to nature’(Wollen 1976, cited in Prince, 1996, p 28). Bazin claimed that the ‘photographic image is the object itself freed from the conditions of time and space’(Bazin, 1967, cited in Prince, 1996, p 28). Stanley Cavell claimed that film was a projection of reality because it recorded what was placed in front of the lens (Cavell, 1979, cited in Prince, 1996, p 28).

Clearly these notions of realism were developed before the advent of CGI, or the more subtle ways that film and photographs can be manipulated with post-editing software. This ability to now digitally manipulate a film challenges the idea of indexically on which realism is predicated. Prince gives the example of computer-generated light and shadow, explaining that there is no need for a ‘real’ source of light to create shadows or intensity of light, but rather that it can be determined by the artist manipulating the image.

Prince (1996, p30) then asks if the ability to manipulate film has broken the link between the photograph and its referent, can films that include elements of CGI or manipulation still be contained within the ideas of realist film theory? And if it cannot, where do they fit? Film theorists such as Rudolf Arheim, Dziga Vertov, and Sergei Eisenstein use the term formalist to emphasise that one of films defining characteristics is the ability to reorganise, counter and possibly even falsify physical reality through film editing and other techniques.

Correspondence based modelling

To answer this question as to whether CGI can be contained in realist films, Prince first describes how correspondence based modelling works. He explains that there is a lot of evidence that viewers understand a film by making connections or ‘correspondences’ between what’s being displayed on the screen and their actual experience of the visual and social world they inhabit. According to Prince (1996, p 32), it is important to understand that ‘cinematic representation’ operates through the way the correspondences are structured, and the cues they produce, to connect with the viewers visual and societal experiences. Whether a viewer considers the film to be realistic, or not, is determined by how they understand these correspondences.

Prince’s key point is that instead of asking ‘whether a film is realistic or formalistic, we can ask about the kinds of linkages that connect the represented fictionalized reality of a given film to the visual and social coordinates of our own three-dimensional world’ (1996, p 32). The discussion is more about whether the viewer considers the film realistically believable in its own environment, as opposed to reflecting real world reality.  So, the question can apply to both “realist” and “fantasy” films. This means that there is no need to establish or even ask if there is indexicality as a proof of realism, since images that do not exist in the ‘real world’ (have no indexicality) can be perceptually realistic.  

Perceptual Realism

Perceptually realistic images appear this way because filmmakers build them to be so. This is achieved by structuring the display of light, colour, texture, movement, and sound in ways that match with the viewer’s understanding of these elements in normal life. Therefore, perceptual realism is not about the image’s indexicality, but rather about its structural correspondence to the viewer’s experience of the world. Correspondence-based modelling serves this purpose; to make the digital elements of a film appear as if they belong in the same environment as the live-action elements. Today, even more so than when this paper was written, there are a wide variety of techniques and technologies that enable filmmakers to connect unreal elements with real world film that are perceptually realistic. They include:

1. Camera tracking: The motion and position of the ‘real world’ camera are tracked throughout the scene. This tracking data is then used to synchronise the CGI elements with the ‘real’ live-action footage. The aligning of the digital camera in the CGI software with the real world camera, so that the CGI elements fit realistically into the scene is called match-moving.
2. Object tracking: This is used to track specific elements in a scene. This helps ensure that the CGI elements interact correctly with the real-world objects and each other. Object tracking makes sure that CGI elements don’t inadvertently run through solid objects in the ‘real’ world, or fail to react to elements in the real world.
3. Lighting and shading: This is matching, or enhancing, the digital lighting and shading to matches the real-world lighting.
4. Compositing: The final step involves compositing, where the digital and real world film elements are combined in post-production.
5. Top of Form

Prince concluded that with the advent of digital imaging, film theorists are shifting away from the idea of indexical realism to focus on how a film’s particular message or meaning is constructed through the intentional use or design of language, narrative, visual elements, structure and arrangement, irrespective of whether this is real or unreal. He argues that it is not necessary to make a distinction between whether a film ‘indexically records the world or stylistically transfigures it. Cinema does both’ (1996, p35).

He maintains that film theory has not fully explored the importance of perceptual correspondences, but technological advances in digital imaging have made clear how important it is to understand the perceptual correspondences viewers make when watching a film.

Relation to own work

In relation to the film I intend to make, this paper has relevance. It is inconceivable that the film will not have some form of manipulation of almost all elements – lighting, colour, texture, speed and so on. It is also quite possible that some elements of 3D animation will be included (not quite CGI on the Lord of the Rings scale 😊). The key point to take way is that whichever editing choices are made, they should be in service of perceptual realism, not indexicality. However, I do need to be very careful because on the one hand, the narrative film needs to serve both an educational and entertainment purpose but on the other hand, the content cannot stray so far away from the reality of the world that I will portray (the air traffic management world) that the viewer will connect with the perceptual reality of the story but not the real world reality it takes place in.

Reference:

Prince, S. (1996) True Lies: Perceptual Realism, Digital Images, and Film Theory. Film Quarterly (1996) 49 (3): 27-37.

I found an interesting article on a piece of research that was done by Queen Mary University of London and University College London to determine if the human brain reactivates mental representations of past events during new experiences to make meaning from the new experience (Fadelli, 2023). The actual research paper is a complex read and I do not have the neuroscience knowledge to follow the entire research methodology, however, I was able to follow the descriptive part of the research and conclusions and I believe that it supports the notion that film narrative can have an inadvertent, or in the case of the film I intend to make, intentional educational value.

The researchers already knew from previous research that the brain can replay events from the past, but this was usually linked to spatial navigation tasks, i.e., the ability to replay a route that has been taken previously. This particular research had been done with rodents, not humans, however it is not hard to translate this to a person being able to recall how to get to a place they have been to before by replaying the route.

The researchers already knew that the human brain chunks information into smaller pieces that can later be recalled. An example of chunks or parcels of narrative, might be how you think about the day you have just had in chunks; you drove to work, you had coffee at work, you had a meeting, you went to lunch, you worked at your computer, etc. A parcel, or chunk of information is termed an ‘event’, and switching between one event and the next, happens at ‘event boundaries’ (Hahamy, Dubossarsky and Behrens, 2023, p.1080). Event boundaries happen when the context of what is happening changes.

However, for a person to understand a narrative, to make sense of it, needs more than just chunking or parceling ‘the ongoing stream of information into events and storing them into memory. Following an ongoing narrative requires relations to be drawn between each current event and contextually relevant past events’. The researchers argue that for quite some time already it has been proposed is that ‘remote past information could be integrated with incoming sensory information’ (Hahamy, Dubossarsky and Behrens, 2023, p.1080).

The researchers designed an experiment to see if they could ‘elicit the replay of past events as observed in rodents, but during non-spatial daily experiences’ information’ (Hahamy, Dubossarsky and Behrens, 2023, p.1080). To do this, they asked the experiment participants to watch a movie or listen to audio recordings of a narrated story while recording their brain activity using a functional magnetic resonance imaging (fMRI) scanner to look for any indications in the brain that the person was replaying past events, particularly during transitions between scenes (Hahamy, Dubossarsky and Behrens, 2023). Trying to identify if this was happening between scenes, was associated with an ‘event boundary’ occurring when a film moved from one scene to the next.  Simply put the question they asked was ‘would our brains replay past information that is needed for interpreting a scene we had just perceived?”

They used movies because for them, film simulated real word experiences (apparently, they hadn’t been reading Baudrillard!) and films are made up of events that ‘should be linked together to understand the overall narrative’ (Hahamy, Dubossarsky and Behrens, 2023, p.1080). Apparently, they hadn’t read Deleuze’s Cinema 2: The Time-Image either. Nonetheless, it is understood that what they refer to as ’natural narrative’ for a film would follow a classic linear narrative structure. The film they used was Episode 1 of the BBC’s television series ‘Sherlock’.

This experiment is interesting because my film needs to serve an educational purpose, while at the same time be convincing as a story in its own right. Some of the issues that I want to address from an educational achievement perspective are concerned with mindsets and actions around subjects that may not be encountered on a daily basis, such as incidents with security/cyber-security, not making assumptions about the safety of an operation and developing habits that communicate safety concerns or suspicions. The problem with these issues is that they are not encountered on a daily basis, making them unrelatable for the target audience. When we train air traffic controllers (ATCOs) to deal with situations that don’t often (thankfully) occur in real operations, such as aircraft experiencing various different emergencies or keeping the skies safe when there is a system outage, we use very realistic simulations to expose the ATCOs to these situations and allow them to practice different responses. This training is repeated regularly in the simulator so that the ATCO can retain a mental model of how to respond.

However, the audience I am targeting for the film is a much wider Air Traffic Management (ATM) audience – effectively any personnel working in ATM operations, not just the ATCOs. It is simply not practical to provide simulation for so many different people doing so many different jobs. If people can make sense of current events, by replaying past events (in this case, a past event seen in a film) it could be a very powerful tool for creating awareness and triggering responses from ATM personnel on how to think about and respond to potentially unsafe situations.

The results of the experiment showed that while people were watching the film, they reactivated past events, in real-time, to make sense of each scene. More specifically they conclude that these reactivations are considered as the ‘candidate mechanism for binding temporally distant information into a coherent understanding of ongoing experience’(Hahamy, Dubossarsky and Behrens, 2023, p.1080). The fact that this is happening in real-time is important because it suggests, to me at least, that reactivation of a past event (even if it was in a movie) to help make sense of a current situation that potentially requires an action, can be considered as part of an educational endeavor, and even more so if the film can also provide guidance on how to deal with the new situation.

References:

Fadali,I. (2023) Study finds that the human brain reactivates mental representations of past events during new experiences. MedicalxPress. Available at: https://medicalxpress.com/news/2023-06-human-brain-reactivates-mental representations.html?utm_medium=email&utm_source=other&utm_campaign=opencourse.GdeNrll1EeSROyIACtiVvg.announcements~opencourse.GdeNrll1EeSROyIACtiVvg.1s1RHF8PRFGwztzDi5eVIQ (Accessed 8 October 2023)

Hahamy, A., Dubossarsky, H. & Behrens, T.E.J. (2023) The human brain reactivates context-specific past information at event boundaries of naturalistic experiences. Nat Neurosci 26, 1080–1089. https://doi.org/10.1038/s41593-023-01331-6

Gilles Deleuze (1925-1995) was a French philosopher who covered a wide range of subjects including film theory. I have concentrated on the work that he did that I believe may have relevance to my narrative film.

His major work on cinema is divided into two categories; the Movement-Image and the Time-Image. In the movement-image, he discusses traditional narrative cinema, which relies on cause-and-effect relationships and the continuity of time and space. Whereas, the time-image, is when there is a break from this traditional narrative structure that explores non-linear and fragmented approaches to storytelling. Both of these approaches could be relevant to my film.

The Movement-Image

The concept of the movement-image is a key way in which Deleuze analyses traditional forms of narrative film. He published this concept in a book called “Cinema 1: The Movement-Image,” which was published in 1983. He argues that movement-image is a way to explain how traditional cinema (i.e. Hollywood films) constructs and communicates narratives by manipulating time, space, and cause/effect.

The main ideas associated with this concept include:

1. Continuity and Representation: In the movement-image, cinema is primarily concerned with continuity and representation. This means that in traditional narrative cinema the aim is to create a seamless flow of images and sounds that represent a story in a cause-and-effect way. Time is linear, as is space and the narrative structure is clear with well-defined characters and plot points.
2. Action and Reaction: Movement-image relies heavily on the principle of action and reaction. It presents events in a way that shows how actions lead to consequences and how characters respond to various stimuli. This cause-and-effect structure is old-style or classic Hollywood cinema, where the story reveals itself in a logical and predictable fashion.
3. Cinematic Techniques: Deleuze also discusses the use of cinematic techniques such as composition of the shots, ways of editing, and camera movement. These techniques are used to create a coherent narrative that the audience can easily follow.
4. Sensory-Motor System: Deleuze introduces the idea of the “sensory-motor system” as the way in which classic cinema engages the viewer’s senses and motor functions. Viewers are drawn into the narrative because they start to identify with characters and react emotionally to what happens on the screen.
5. Perception of Time:  Time is typically perceived as linear and continuous with the past, present, and future tightly connected. This perception of time aligns with conventional storytelling techniques.

The Time-Image

“Cinema 2: The Time-Image” was published in 1985 and explores more complex and non-linear approaches to filmmaking and film analysis. He believes that film that does not confine itself to conventional linear storytelling (i.e. movement-image)  but rather displays a more fragmented sense of time and continuity provides new and interesting ways of telling stories. This fragmentation often involves blurring of past, present, and future, making it challenging for the viewer to work out a clear chronological order of events.

Some key points:

1. Subjectivity and Memory: The time-image film often explores the subjective experiences of characters and the way they perceive and remember events. Memory becomes a key element in how the narrative is constructed, meaning that characters may revisit or re-experience past moments in a non-linear fashion. This focus on individual subjectivity and memory allows films to be more introspective and psychological. A good explain of this is Martin Scorsese’s film Shutter Island (2010) where the story is seen through the eyes of the main protagonist who is mentally unstable, making his memories unreliable.
2. Sensations and Affects: These films have a strong emphasis on creating sensory impacts for the audience. Instead of prioritising action and plot, the time-image film tries to evoke sensations and emotions in the audience. These sensations can vary, from being disorienting, to contemplative, or dreamlike. A good example of this is the David Lynch film, Mulholland Drive (2001) that combines dreamlike sequences, narrative disruptions, and a nonlinear narrative to create a mysterious and unsettling atmosphere.
3. The Crystal-Image: The “crystal-image” is a key concept for time-image. This is when, in the film, the past, present, and future come together into a single, crystalline moment. The crystal-image represents a collapse of traditional temporal boundaries and invites viewers to contemplate the coexistence of different temporal dimensions within a single frame. A good example of the crystal image in a film is Christopher Nolan’s Inception (2010). Actually, this film is a good example of many different time-image concepts. In the film there is a scene where Dom Cobb (the protagonist) revisits his memories and confronts his own guilt and regrets about his wife, Mal who died. The moment takes place inside a dream world, but at the same time and the structure shifts to show the coexistence of multiple temporal dimensions. As Dom delves deeper into his memories and emotions, the boundaries between the past and present begin to weirdly blur, and time becomes sort of fluid and subjective. The scene is a good example of the crystal-image because it brings together the  introspection of past events and emotions, with the present. In this way it gives the audience and chance to think about the complexities of memory, guilt, and the interplay between different temporal layers.
4. Non-linearity and Discontinuity: To achieve this, directors make use techniques such as flashbacks, flash-forwards, and elliptical editing to disrupt the traditional flow of time and confuse audiences’ expectations.
5. Art Cinema and Avant-Garde Film: The time-image is often associated with art cinema and avant-garde filmmaking, where directors experiment with narrative structures and visual styles to create more open-ended stories.

Applying this to my work

On reflection, I do not believe it necessary to choose exclusively a movement-image or a time-image approach to making my film, although at this point I believe that movement-image is the most effective concept for a film that should also have educational value. Time-image films are characterised by nuanced and rich experiences for an audience that wish to untangle to the non-linearity and discontinuity of a film and contemplate the multiple-interpretations that could be made. That may be part of the appeal of a time-image film, however, for my film, I would need to give consideration to how the audience dealing with discontinuity and non-linearity would help or hinder the learning value.