Chapter 6: Captioned silences and ambient sounds

This supplemental website includes all of the media clips discussed in Reading Sounds (University of Chicago Press, 2015). Purchase your copy of Reading Sounds today in paperback or e-book.

Figure 6.1. If a car crashes into a tree but doesn’t make a sound, a silence caption may be needed.

In this clip from The Artist, smoke billows from the hood of a 1935 Cadillac Convertible Sedan that has hit a tree. The driver (Bérénice Bejo) is running away from the car and towards the viewer. The camera looks down upon the scene from a second floor vantage point. Caption: [silence]. Right before the car crash, a silent film intertitle says “BANG!” Because of its placement immediately following a scene in which George (Jean Dujardin) prepares to kill himself by putting a gun in his mouth, we are led to believe that “BANG!” applies to George’s suicide attempt when in fact it applies to the car crash. Studio 36, 2011. DVD.

Source: The Artist, 2011. DVD. Featured caption: [silence]

On Star Trek: The Next Generation, the interior scenes of the Starship Enterprise are accompanied by a continuous, low rumbling sound that is supposed to remind us subtly of the futuristic engine that powers the ship. On the engineering deck, the ambient rumbling becomes more pronounced and begins to pulse or breathe slowly. The sonic rhythm of the engineering deck can sometimes be seen in the pulsing blue light of the warp core. The iconic hum on the engineering deck is never captioned, even though it serves an important function – often just below the listener’s conscious awareness – that this sector of the starship is the driving, humming heart or living breath.

Source: Star Trek: The Next Generation, “Relics,” 1992. DVD.

Source: Star Trek: The Next Generation, “Realm of Fear,” 1992. DVD.

Table 6.1. Examples of mouthed and silenced speech captions

Source Caption Description
Monk, “Mr. Monk and the Red-Headed Stranger” (1.12), 2002 [ Mouthing Words ] Mouthed speech: “We can’t stop.”
Monk, “Mr. Monk and the Airplane” (1.13), 2002 [ Mouthing Words ] Mouthed speech: The first mouthed speech caption stands in for two utterances. Garry Marshall mouths “Very good” and Bitty Schram mouths “Thank you” in response. The exchange happens very quickly.
[ Mouthing “Hammered” ] Mouthed speech: The second mouthed speech caption puts the mouthed word(s) in the caption, which is preferred in cases where the speech can be discerned.
CSI: NY, “Unspoken” (9.4), 2012 (mouthing) Mouthed speech: “We need to talk.”
Star Trek: The Next Generation, “Starship Mine” (6.18), 1993 ( no audio ) Mouthed speech: “Find him.”
Drive Angry (2011) (INAUDIBLE) Distant speech: While it’s clear that the two men are talking to each other, it’s unclear what they are saying. “Inaudible” is appropriate in this case.
Nick and Norah’s Infinite Playlist (2008) [ALL SPEAKING INAUDIBLY] Silenced speech: It is not possible to identify what the multiple characters are saying through the car window.
The Office, “Two Weeks” (5.21), 2009 (INAUDIBLE) Silenced speech: It’s unclear what Steve Carell says in the parking lot from the camera’s vantage point a couple floors up inside the office.
Twins (1988) [ Silence ] The entire interaction takes place through jailhouse glass:
Silenced speech: “Excuse me.”
[ Silence ] Silenced speech: “Vincent. It’s you, Vincent.”
[ Silence ] Silenced speech: Intentionally unclear to hearing viewers. Played for laughs.
[ Silence ] Silenced speech: Also intentionally unclear to hearing viewers.
Oblivion (2013) (MOUTHING) Hi. Silenced speech: The silent speech is captioned verbatim, which is the preferred approach in cases where the speech can be discerned. But this speech is not technically “mouthed.” The entire flashback scene is silent except for the overlay of slow, moving, uncaptioned piano music. The next line of dialogue – “Hello.” – does not come with the same “mouthing” or inaudible disclaimer even though it is also silent.
(INAUDIBLE) Silenced speech: It’s not clear what the father says to the girl, which is why “inaudible” is appropriate here. But in all three lines of dialogue in this clip, the captions do not make it clear to viewers that the scene is done in silent style with music overlay.

Source: Monk,“Mr. Monk and the Red-Headed Stranger,” 2002. DVD. Featured caption: [ Mouthing Words ].

Source: Monk,”Mr. Monk and the Airplane,” 2002. DVD. Featured caption: [ Mouthing Words ].

Source: Monk,”Mr. Monk and the Airplane,” 2002. DVD. Featured caption: [ Mouthing “Hammered” ].

Source: CSI-NY,”Unspoken,” 2012. DVD. Featured caption: (mouthing).

Source: Star Trek: The Next Generation,”Starship Mine,” 1993. Syfy Channel. Featured caption: ( no audio ).

Source: Drive Angry, 2011. DVD. Featured caption: (INAUDIBLE).

Source: Nick and Norah’s Infinite Playlist, 2008. DVD. Featured caption: [ALL SPEAKING INAUDIBLY].

Source: The Office, “Two Weeks,” 2009. DVD. Featured caption: (INAUDIBLE).

Source: Twins, 1998. DVD. Featured caption: [ silence ].

Source: Oblivion, 2013. DVD. Featured captions: “(MOUTHING) Hi” and (INAUDIBLE).

Mouthed speech captions in We’re the Millers

Source: We’re the Millers, 2013. DVD. Uncaptioned version. The scene is silent and acquires meaning through mouthed speech, facial expressions, and common gestures.

Source: We’re the Millers, 2013. DVD. Captioned version. The scene is captioned verbatim. Each speaker’s utterances are front-loaded with two manner of speaker identifiers: [MOUTHING].

Montage sequences

Montage sequences, which may include silenced speech, can also prove challenging to caption. In a movie montage, speech and ambient sounds take a back seat to music and the dynamic interplay of images. A number of short shots are juxtaposed into a sequence that signifies the compression of space and time.

Source: South Park, “You’re Getting Old,” 2011. Comedy Central. Featured captions: Verbatim lyrics from Fleetwood Mac’s “Landslide.”

Perhaps the most famous and parodied example is the sports training montage, made famous in the Rocky films. The typical movie montage is set to music, which fuses the otherwise silent and disparate shots together. Some ambient noise may filter through, but it is more common in a montage sequence to edit out all sounds and set the scene to music.

Source: South Park, “Asspen,” 2002. Hulu. Featured captions: Verbatim lyrics from an original parody song about the sports montage.

The sports training montage was made famous in the Rocky films. In this DVD clip from Rocky IV (1985), Sly Stallone prepares to take on the Russian fighter Ivan Drago (Dolph Lundgren). In “Montage Overload,” Steve McCutchen catalogs and analyzes all of the montage and musical sequences in Rocky IV, concluding that “Nearly one-third of the entire movie is montage / musical numbers.” McCutchen rightly calls this statistic “Absolutely incredible.” Rocky IV actually contains two training montages. They are separated by seventy-five seconds but a total of nine solid minutes of training montage. If you’re going to end the Cold War by defeating a Russian boxing machine like Drago, one training montage is not enough. Here’s a short clip from the second training montage in Rocky IV (which is also the sixth montage/musical sequence in the film):

Source: Rocky IV, 1985. DVD. Featured captions: Verbatim lyrics from John Cafferty’s “Hearts on Fire.” Song artist and title are not captioned.

According to McCutchen, the third musical/montage number in Rocky IV is “perhaps the greatest montage in the history of montages.” In this montage, Rocky drives around at night after the death of his friend Apollo Creed. He reflects on the past (complete with images from Rocky I, II, and III) and considers the future (complete with fearsome images of Drago, the Russian boxer he will face):

Source: Rocky IV, 1985. DVD. Featured captions: Verbatim lyrics from Survivor’s “No Easy Way Out.” Song artist and title are not captioned.

Note that only music lyrics are ever captioned in these montage sequences from Rocky IV. Song titles and artists’ names are never captioned, nor do the captions ever indicate when speech (moving lips) has been silenced by montage music.

Animated GIFs with open captions

An animated gif from The Office featuring Jim Halpert mouthing, I'm boring myself just talking about this.
Source: Giphy.

An animated gif from The Office featuring Michael Scott mouthing the last three words of the line: I don't even consider myself part of society.
Source: Giphy.

“Continue” captions

Source: A Serious Man, 2009. DVD. Featured caption: (ROCK MUSIC CONTINUES PLAYING ON HEADPHONES). This caption is paired with (ROCK MUSIC PLAYING ON HEADPHONES), which precedes the “continues” caption by two minutes and twenty-one seconds.

Source: An Education, 2009. DVD. Featured caption: (LAUGHING CONTINUES). This caption is paired with (LAUGHS), which precedes the “continues” caption by four seconds.

Source: Argo, 2012. DVD. Featured caption: [CHANTING CONTINUES OUTSIDE]. This caption is paired with [CROWD CHANTING IN FOREIGN LANGUAGE], which precedes the “continues” caption by one minute and sixteen seconds.

Continue captions don’t tend to be followed by stop or end captions, because most of the time it is visually obvious when a continuing sound terminates. We can see the students stop laughing in An Education (2009) and then turn their attention to orchestra practice. What we can’t see is that they continue laughing off-screen when the scene cuts to the object of their amusement – hence the need for (LAUGHING CONTINUES). The same is true of (GUNFIRE CONTINUES) in Avatar (2009), a caption which is needed when it isn’t visually clear that Jake (Sam Worthington) is still being fired upon after fleeing from the top of the giant bulldozer. The difference is that (GUNFIRE CONTINUES) is not a response to a previous caption in the same way that (LAUGHS) serves as a predecessor for (LAUGHING CONTINUES). Rather, (GUNFIRE CONTINUES) is a response to a shift in focus from visible gunfire straight from the barrel of a shotgun-style weapon held by a military guy to the mere sounds of gunfire as Jake flees.

Source: Avatar, 2009. DVD. Featured caption: (GUNFIRE CONTINUES). This caption becomes necessary when we can’t see that Jake is still being fired upon.

Source: Skyfall, 2012. DVD. Featured caption: (DOGS CONTINUE BARKING). This caption is paired with (DOGS BARKING IN DISTANCE), which precedes the “continue” caption by eight seconds.

Source: Skyfall, 2012. DVD. Featured caption: (TELEGRAPH CONTINUES CLICKING). This caption is paired with (CLICKING), which precedes the “continues” caption by thirty-eight seconds.

Source: Two and a Half Men, “Ixnay on the Oggie Day,” 2010. FX Television. Featured caption: (slapping continues). This caption is paired with (slap), (loud slap), and (slapping stops).

Source: The 4400, “Rebirth,” 2005. Netflix. Featured caption: [ ANGRY SHOUTS FROM CROWD CONTINUE ]. This caption is paired with [ ANGRY SHOUTS ], which precedes the “continue” caption by seventeen seconds.

Source: Lost in Translation, 2003. DVD. Featured caption: ♪ ♪[ Continues ]. This caption is paired with music lyrics and ♪ ♪ [ Piano ].

Source: Lost in Translation, 2003. DVD. Featured captions: ♪ ♪[ Continues ], which is repeated once during this scene. This caption is paired with ♪ ♪ [ Pop ], which precedes the “continues” captions by one second and twenty seconds, respectively.

Source: The Master, 2012. DVD. Featured caption: [indistinct chatter continues]. This caption is paired with [indistinct chatter], which precedes the “continues” caption by eleven seconds.

Figure 6.2. Silence continues in The Artist.

Peppy (Bérénice Bejo) embraces George (Jean Dujardin) in a close-up over-the-shoulder shot from The Artist. Caption: [silence continues], which appears sixty-six seconds after the previous caption, [silence]. Studio 36, 2011. Blu-Ray.

Source: The Artist, 2011. DVD. Featured caption: [silence continues]

Source: The Artist, 2011. DVD. Featured caption: [barking continues] [ringing continues]

Source: The Artist, 2011. DVD. Featured caption: [whimsical, ambling melody continues]. This caption is paired with [orchestra playing whimsical, ambling melody], which precedes the “continues” caption by fifty-four seconds.

“Stop” captions

Source: Aliens vs. Predator: Requiem, 2007. DVD. Featured caption: (beeping stops). This caption is immediately preceded by (steady beeping) and (beeping accelerates).

Source: Avatar, 2009. DVD. Featured caption: (CHATTERING AND SINGING STOP). This caption is paired with (VILLAGERS CHATTERING) and (SINGING IN NA’VI). Note the layering up of multiple, sustained sounds in separate captions (chattering, singing) that are simultaneously stopped with a single caption.

Source: Moonrise Kingdom, 2012. DVD. Featured caption: (RECORD PLAYER STOPS). The record player sounds are identified in a previous caption in the form of a channel identifier nested inside a speaker ID: “BOY: (ON RECORD) So you see, the composer Benjamin Britton.”

Source: Lincoln, 2012. DVD. Featured caption: (MURMURING STOPS). This caption is paired with (CROWD MURMURING), which is presumed to be the usual sound of the chamber when legislators are hard at work. The stop caption is intended to show how the legislators are brought to a state of stunned silence when a group of African Americans enters the gallery section of the capitol.

Source: Oblivion, 2013. DVD. Featured caption: (WHIRRING STOPS). This caption is paired with (DRONE WHIRS), which precedes the stop caption by five seconds.

In the following examples from The Artist, a music stop caption is immediately followed by [silence]. In both clips, the captions follow a formula:

  1. [music playing]
  2. [music continues]
  3. [music stops]
  4. [silence]

Source: The Artist, 2011. DVD. Featured caption: [music stops abruptly]. This caption is preceded by [ominous music playing] and [ominous music continuing]. A silence caption immediately follows.

Source: The Artist, 2011. DVD. Featured caption: [jazz music ends]. This caption is preceded by [lively jazz playing], [lively jazz continues], and [lively jazz continues]. A silence caption immediately follows.

Source: Inception, 2010. DVD. Featured caption: [CHATTERING STOPS]. This caption is paired with [PEOPLE CHATTERING], which precedes the stop caption by seven seconds.

Source: The Walking Dead, “Sick,” 2012. Cable TV. Featured caption: [screaming stops]. This caption is paired with [screaming continues], which precedes the stop caption by two seconds. When the offscreen screaming stops, we know that the man has been killed by the zombies.

Source: Futurama, “Rebirth,” 2010. Cable TV. Featured caption: (screaming stops). This caption is preceded by a series of scream captions as Leela reacts to discovering she’s a robot: (screaming), (continues screaming), (continues screaming), (vibrating scream), and (screaming).

Source: Aliens vs. Predator: Requiem, 2007. DVD. Featured caption: (engine whirs to a stop). This caption is not accompanied by a “continues” caption but rather a visual context that makes it obvious that the tank is coming to a stop.

Source: Young Doctor’s Notebook, Episode 1.1, 2012. Cable TV. Featured caption: (TURNS TAP OFF), which is not preceded by a caption but rather the sight of the young doctor (Daniel Radcliffe) washing his hands in the sink.

Source: Aliens vs. Predator: Requiem, 2007. DVD. Featured caption: (fanfare ends), which is preceded by (rousing orchestral fanfare playing). These captions, along with (drumroll), describe the title sequence music for 20th Century Fox.

Source: Argo, 2012. DVD. Featured caption: [RATTLES THEN STOPS]. This stop caption is not preceded by a previous caption but rather incorporates the sound and its termination into a single caption.

Source: Knight and Day, 2010. DVD. Featured caption: (RINGTONE STOPS). This stop caption is paired with (LOUIE LOUIE RINGTONE PLAYING), which precedes the stop caption by nine seconds.

Source: Two and a Half Men, “Ixnay on the Oggie Day,” 2010. FX Television. Featured caption: (slapping stops). This caption is preceded by (slap), (loud slap), and (slapping continues).

Source: Unknown, 2011. DVD. Featured caption: [THUMPING STOPS]. This stop caption is paired with [MACHINE THUMPING], which precedes the stop caption by forty-two seconds. Note also the misspelling of “dialouge” at the forty-six second mark.

Source: State of Emergency, 2013. Netflix. Featured caption: (JIM MUTES TV). This stop caption is preceded by a visual context of speech captions emanating from the TV, followed by Jim picking up the remote control.

Source: Family Guy, “Blind Ambition,” 2005. Cable TV (Adult Swim). Featured caption: (jingling stops). This stop caption is preceded by (coins jingling rhythmically) and (coins jingling faster).

Source: Lost in Translation, 2003. DVD. Featured caption: ♪ ♪ [ Ends ]. This stop caption is preceded by ♪ ♪ [ Guitar: Heavy Metal ] and ♪ ♪ [ Video Game: Man Singing In Japanese ].

Source: Inception, 2009. DVD. Featured caption: [TURNS OFF RADIO]. This stop caption is preceded by [MAN SINGING IN FOREIGN LANGUAGE ON RADIO].

Source: Man of Steel, 2013. DVD. Featured caption: [CACOPHONY STOPS]. This stop caption is paired with [NOISE CACOPHONY], which precedes the stop caption by eight seconds.

Source: Aliens vs. Predator: Requiem, 2007. DVD. Featured caption: (car turns off). Sometimes the agent can’t (and shouldn’t) be identified, as when the driver of an approaching car is intended to be mystery in this sequence: (car approaching), (tires squealing), and then (car turns off ).

The music stops abruptly in an opening scene from American Dad

Source: American Dad, “Fartbreak Hotel,” 2011. Cable TV. Featured caption: (music stops abruptly). This caption is preceded by four music note captions, but music notes don’t provide a sufficient set-up for this stop caption.

The whirring doesn’t stop abruptly in Oblivion

The pairs of captions discussed here (continues and stop captions) tend to assume a sonic universe of binaries in which sounds are either on or off. The TV audio is either on or muted, the chattering continues or stops suddenly, the machine whirs or doesn’t, the voice screams or is silent, the slapping continues or stops abruptly, and so on. A good example of an abrupt change of state from on to off is (JIM MUTES TV) in State of Emergency (2013). While many of the sounds listed above do in fact operate as binaries (e.g., the TV audio turns off instantly when muted), not every stop caption captures the nature of the sonic changes being described. Sounds may be more complex than a simple stop caption may suggest. For example, the drone in Oblivion (2013) doesn’t stop abruptly, despite being captioned as (whirring stops). Rather, it powers down. The change from on to off is quick but still gradual, taking approximately five seconds. The process is metaphorically closer to turning off a ceiling fan than a car engine. The question, then, is how much complexity captions can support and readers need.

Source: Oblivion, 2013. DVD. Featured caption: (WHIRRING STOPS). This caption is paired with (DRONE WHIRS), which precedes the stop caption by five seconds.

Fading sounds and captions

Source: Argo, 2012. DVD. Featured caption: [RATTLES THEN STOPS]. This stop caption is not preceded by a previous caption but rather incorporates the sound and its termination into a single caption.

Source: Beasts of the Southern Wild, 2012. DVD. Featured caption: [MOTOR REVVING THEN FADING].

Source: Cloud Atlas, 2012. DVD. Featured caption: [SIGHS THEN GRUNTS]

Source: The Artist, 2011. DVD. Featured caption: [anguished music subsides]. This caption is preceded by [frenzied, anguished music swells].

Swelling and subsiding provide examples of captioned modulation in which sounds fluctuate inside a single caption, in contrast to the world of captioned binaries in which sounds are presented as either on or off.
 Two frames from The Artist (2011). In the first frame the caption is: [frenzied, anguished music swells]. In the other frame, the caption is: [anguished music subsides]. Both frames are taken from the preceding clip.

Non-speech captions tend to present sounds as either on or off. The change of state is abrupt and total. What was fully audible in a “continues” caption, for example, is now fully muted in a “stop” caption. But sounds can also be described in terms of their changing value or intensity inside a single caption. For example, consider instrumental music that is described as swelling in one caption and fading in the next, visually akin to something like an on-off switch. In contrast, captioned modulation can be visually represented as a triangle: swelling sounds move up one sloping side of the triangle and fading sounds move down the adjacent side. Time might be considered the triangle’s base.

Graphical representations of captioned binary and captioned modulation

Source: The Artist, 2011. DVD. Featured caption: [tempo slows, music fades]. This caption is preceded by [discordant, ambling melody playing].

Source: Bones, “The Wannabee in the Weeds,” 2008. Cable TV (TNT). Featured caption: (fades out): ♪ Well, we’re all in the mood for a melody… ♪ This caption signals the end of the music lyric captions for Billy Joel’s “Piano Man.”

Source: Family Guy, “Peter’s Got Woods,” 2005. Cable TV (Adult Swim). Featured caption: (laughter dies down).

In this clip from Skyfall (2012), the captions attempt to mirror the fluctuations in the bar crowd’s reactions, as James Bond (Daniel Craig) attempts to win a bet by drinking from a whiskey glass without being stung by a large scorpion perched on his drinking hand. The crowd’s (INDISTINCT CHATTER) dies down (CHATTER DIES DOWN) in anticipation of the bet before building up again (BUILDING IN VOLUME) as Bond begins to drink, and finally reaching a fever pitch of (EXCITED CHATTER) and (CHEERING) as Bond wins the bet without being stung.

Source: Skyfall, 2012. DVD. Featured captions: (CHATTER DIES DOWN) and (BUILDING IN VOLUME).

Source: Galaxy Quest, 1999. Cable TV (Comedy Central). Featured caption: [screaming dies out]. Note that the captions are delayed by approximately five seconds, making them difficult to follow.

In the DVD version of Galaxy Quest, the captioner opts for a stop caption instead:

Source: Galaxy Quest,1999. DVD. Featured caption: [Screaming Stops].

Source: Zero Dark Thirty, 2012. DVD. Featured caption: (OVERLAPPING PHONE CHATTER FADES). This caption is preceded by (OVERLAPPING PHONE CHATTER CONTINUES). The fading caption suggests that the people monitoring numerous phone conversations for potential terrorist activity have zeroed in on a single conversation of interest — i.e. (WOMAN SPEAKING IN FOREIGN LANGUAGE OVER PHONE).

Source: Zero Dark Thirty, 2012. DVD. Featured captions: (ENGINE POWERING DOWN) and (BLADES SLOWING TO A STOP). These sounds are accompanied by total darkness as we witness the attack through the eyes of the U.S. military. The screen is black in the moments following the helicopter’s (ENGINE POWERING DOWN). Listeners hear but don’t see the (WIND WHISTLING), (COW MOOS), (COW MOOS), and (DOOR CREAKS).

Source: Moneyball, 2011. Cable TV (FX). Featured caption: [TURNS VOLUME DOWN].

Source: Lincoln, 2012. DVD. Featured captions: (CHURCH BELLS CHIMING FAINTLY) and (CHIMING GETS LOUDER).

Source: Avatar, 2009. DVD. Featured caption: (ENGINE POWERING UP).

Source: Aliens vs. Predator: Requiem, 2007. DVD. Featured caption: (crackling intensifies). This caption is preceded by (splashing, electrical crackling).

Table 6.2. All twelve non-speech “silence” captions in The Artist (2011), with accompanying caption start times and movie chapter numbers.

Start time Caption Chapter
0:05:31 [silence] 1
0:09:14 [silence] 2
0:25:40 [silence] 4
0:30:35 [silence] 5
0:32:35 [silence] 5
0:42:31 [silence] 7
1:00:19 [silence] 9
1:07:35 [silence] 10
1:19:16 [silence] 11
1:26:08 [silence] 12
1:32:29 [silence] 13
1:33:35 [silence continues] 13

Figure 6.3. The twelve non-speech “silence” captions in The Artist serve as evenly spaced reminders to audiences that film sound is not functioning as expected.

Plotting the data in Table 6.2 on a timeline.
This graph plots the data in Table 6.2.

Silence overtakes an episode of Jon Benjamin has a Van

Source: Jon Benjamin Has a Van, “Breakdown,” 2011. Cable TV (Comedy Central). Featured caption: [silence].

Comparing silence captions across different programs

Silence is contextual. The [silence] of The Artist is not sonically comparable to the [silence] in Jon Benjamin. The former is a deletion of the audio track in the spirit of a silent film. The latter is a fairly loud static buzz. Jon Benjamin resorts to buzzing static instead of no audio because the transition to silence, like the breaking of the fourth wall, is so unexpected that it needs to be marked in some way. Total silence could lead hearing viewers to wonder whether something is wrong with their television or the distributor, unlike the expected silence of The Artist. While the [silence] in South Park’s “Trapper Keeper” (2000) episode is closer to the static silence in the Jon Benjamin episode, it could not serve as a compelling substitute for it.

Source: South Park, “Trapper Keeper,” 2000. Cable TV (Comedy Central). Featured caption: [SILENCE].

Curse words that are excised from the soundtrack and captioned with placeholders

In order to prepare the soundtrack of a movie for cable broadcast on American television, the soundtrack must be sanitized. A cheap and fast way to do this is to delete the offending speech, leaving silence where the curse word(s) used to be. The caption track then marks the silence in some way. For example, in the 2007 movie The Mad, which I recorded on the basic cable channel Chiller, the curse words are deleted from the soundtrack, leaving quick bursts of silence behind. On the caption track, (deleted) is used to indicate that the offending audio was removed/silenced: “Shut the (deleted) up!” Another example is from a 2004 episode of South Park (“Douche or Turd”), in which curse words are silenced and replaced on the soundtrack with [bleep]. But the [bleep] caption does not stand in for a “bleep” sound. While the original broadcaster, Comedy Central, uses a bleep sound to block curse words, the syndicator of this episode, WGN, excised the curse words, leaving silent gaps in their wake but keeping the original [bleep] captions. Put simply, these placeholders are printed words on the caption track and silences on the sound track. The captions are phantoms because they don’t correspond to any sounds.

Source: The Mad,2007. Cable TV (Chiller). Featured caption: “(in unison) SHUT THE (deleted) UP!”

Source: South Park, “Douche or Turd,” 2004. Cable TV (WGN). Featured caption: ♪ VOTE OR DIE, MOTHER[BLEEP] MOTHER[BLEEP], VOTE OR DIE ♪

Source: Foo Fighters: Live at Wembley Stadium, 2008. Cable TV (VH1). Featured caption: HOW THE […] DID THIS BAND GET THIS […] BIG?

The sponsorship message at the end of the opening theme song for American Dad

Another type of phantom caption includes the various bylines and sponsorship messages inserted at the beginning or end of TV programs that do not have any sonic equivalent. For example, no one speaks the sponsorship message that always accompanies the end of the opening theme song sequence in American Dad: “Captioning sponsored by 20th CENTURY FOX TELEVISION.” In other episodes of this show, I’ve seen an additional caption added: “and TOYOTA. Moving Forward.” This silent information is only available to caption readers. In order to squeeze it into the opening sequence, the final line of the theme song, sung by the chorus as “Good morning, USA!,” is sacrificed. The sponsor’s name takes precedence, and caption readers are denied access to the final line of the song.Two screen shots from the opening to American Dad featuring the sponsor's messages in the caption track

The sponsorship message in the opening of CBS News Sunday Morning

Source: CBS News Sunday Morning, December 1, 2013. Cable TV (CBS). Featured caption is the five-line sponsor’s message from Johnson and Johnson.

Table 6.3. One example of how captions may be hurried up to make room for a sponsor’s message.

The presentation rate for speech is compared with the presentation rate for each accompanying caption in the final five speech captions at the end of an episode of Fox’s Family Guy (“Into Fat Air,” 2012). The units of measurement are words per minute (wpm) and seconds. The final column measures how prematurely each caption appears on the screen by subtracting each caption’s start time from the accompanying speech’s start time.

Caption Speech Rate Caption Rate How Early?
There are more
important things
than a petty rivalry.
191 wpm 302 wpm 0.43 seconds
We’re just happy
that you’re okay.
196 wpm 308 wpm 1.65 seconds
PAM:
Well, thank you both.
You saved our lives.
188 wpm 309 wpm 2.25 seconds
And Peter, I’ll never
forget what you did for
us. You’re a good man.
231 wpm 336 wpm 3.4 seconds
All right, you rest
up, get better, we ate
your son. Bring it up!
253 wpm 410 wpm 4.7 seconds

Source: Family Guy, “Into Fat Air,” 2012. Cable TV (Adult Swim). An example of how the speech captions are hurried up to make room for the sponsor’s captioned ad at the end of the episode.

The captioning company’s byline at the end of The Matrix

Source: The Matrix, 1999. Cable TV (SyFy). Featured caption: CAPTIONED BY THE NATIONAL CAPTIONING INSTITUTE —www.ncicap.org—

Keynote sounds

On TV shows and in the movies, ambience creates an immersive sonic “ground” for listeners through background music, stock soundscapes, and foley sounds. The subtle keynote sounds of the subway in an episode from the TV show Castle, for example, create atmosphere and dramatic tension. On live TV news, ambience can be created by the sonic properties of spaces themselves, such as the ways in which speech sounds echo off the hard marble surfaces of the US Capitol during satellite interviews with members of Congress.

Source: Castle, “Kill Switch,” 2014. Hulu (ABC). The featured captions attempt to capture the ambient sounds of the NYC subway: [INDISTINCT TALKING OVER P.A.] and [TRAIN BEEPS]

Source: CNN, July 30, 2015. CNN.com. The speech sounds inside the capitol building not only echo off the hard marble surfaces but are framed by the echoey sounds of other people touring the building.

Ambient music in television commercials

Source: Commercial for Liberty Mutual Insurance. Recorded on October 12, 2013. Cable TV (MSNBC). The commercial uses an instrumental version of The Human League’s huge 1986 hit “Human.” The music is captioned as [HUMAN LEAGUE PLAYS “I’M ONLY HUMAN”].

Source: Commercial for United Mileage Plus Visa Card. Recorded on October 12, 2013. Cable TV (MSNBC). The commercial’s instrumental background music is captioned with two music notes at the end of the commercial: ♪ ♪

Source: Commercial for Embrel. Recorded on October 12, 2013. Cable TV (MSNBC). The commercial’s instrumental background music is not captioned.

Distance and volume indicators in non-speech captions

Source: Aliens vs. Predator: Requiem, 2007. DVD. Featured captions: (guttural croaking in distance) followed by (eerie, raspy grunting echoing from distance)

Source: Argo, 2012. DVD. Featured captions: [GUNSHOTS IN DISTANCE]. Note the “indistinct” caption in this clip too, discussed in the next section as another method of distinguishing foreground from background sounds: [CROWD SHOUTING INDISTINCTLY]

Source: Zero Dark Thirty, 2012. DVD. Featured caption: (HELICOPTER PASSING IN DISTANCE)

Nonspeech references to “passing,” as in “passing overhead” and “passing in the distance,” function similarly to place sounds away from the viewer/camera. Zero Dark Thirty (2012), a military drama about the hunt to find Osama Bin Laden, contains seven references to passing, including three consecutive passing captions: (HELICOPTER PASSING OVERHEAD), (VEHICLES PASSING), and (VEHICLES PASSING).
A compliation of frame grabs from Zero Dark Thirty featuring seven nonspeech "passing" captions.
Source: Zero Dark Thirty, 2012. DVD. This compilation includes: (HELICOPTER PASSING IN DISTANCE), (HELICOPTERS PASSING OVERHEAD), (HELICOPTER PASSING OVERHEAD), (HELICOPTER PASSING OVERHEAD), (HELICOPTER PASSING OVERHEAD), (VEHICLES PASSING), and (VEHICLES PASSING).

Source: The Master, 2012. DVD. Featured caption: [thunder rolls in distance]. While the motorcycle engine is very loud in this clip, it is only captioned as [engine starts] at the beginning of the scene. Also not accounted for in the caption track is the contrast between the loud motorcycle engine and the quiet response from Lancaster Dodd (Philip Seymour Hoffman), which tells us that the motorcycle has already traveled so far away from Dodd that it can’t be heard.

Source: District 9, 2009. DVD. Featured caption: [RAPID GUNFIRE AND MEN SHOUTING IN DISTANCE].

Source: Extract, 2009. Comedy Central. Featured captions: [DOG BARKING IN DISTANCE] and [CRICKETS CHRIPING].

The counterpart to “distance” locators in nonspeech captions are references to sounds that are “nearby”:
A compilation of framegrabs featuring references to
Source: This compilation includes frame grabs from Django Unchained (2012): [dogs barking nearby], [rooster crowing nearby], [animal yipping nearby], and [dogs barking nearby]; The Dark Knight Rises (2012): [TIRES SCREECHING NEARBY], [SIRENS WAILING NEARBY], [EXPLOSION NEARBY], and [GUNSHOTS NEARBY]; Argo (2012): [PEOPLE SHOUTING IN FOREIGN LANGUAGE NEARBY]; and Zero Dark Thirty: (BABY CRYING NEARBY).

Two main axes help to create sonic dimensionality in captioning: the axis of location (near/far captions) and the axis of volume (loud/quiet captions).

Source: The Walking Dead, “The Killer Within,” 2012. Cable TV (AMC). Featured caption: (truck engine revving, faint digging)

Source: Lost in Translation, 2003. DVD. Featured caption: ♪ ♪ [ Karaoke Continues, Singer Faint, Indistinct ]. Note the continues caption at the heart of this non-speech description, which is qualified with references to “faint” and “indistinct” singing.

Source: Lost in Translation, 2003. DVD. Featured caption: [Loud Video Game Noises]

Source: A Serious Man, 2009. DVD. Featured caption: (WAILING LOUDLY)

Source: Avatar, 2009. DVD. Featured caption: (WHISPERING IN NA’VI)

Source: Moonrise Kingdom, 2012. DVD. Featured caption: (WHISPERING INDISTINCTLY)

Source: Silver Linings Playbook, 2012. DVD. Featured caption: [inaudible whispering]

Non-speech captions that rely on the boilerplate: (indistinct chatter)

Source: Beasts of the Southern Wild, 2012. DVD. Featured caption: [CHILDREN CHATTERING INDISTINCTLY]. Note that this background chatter continues throughout the clip — it’s a sustained sound — despite being referenced only once.

Source: Argo, 2012. DVD. Featured captions: [SPEAKING INDISTINCTLY]. This caption is preceded by another kind of chatter caption: [MUFFLED CHATTERING].

Source: Argo, 2012. DVD. Featured captions: [CROWD SHOUTING INDISTINCTLY]. Also in this clip is another reference to indistinct speech: [CROWD CHANTING ON TV].

Source: Aliens vs. Predator: Requiem, 2007. DVD. Featured captions: (indistinct conversation). This caption is immediately followed by a continues caption: (conversation continues).

Source: CSI: NY, “Unspoken,” 2012. DVD. Featured caption: (muffled, indistinct chatter).

Source: District 9, 2009. DVD. Featured caption: [PEOPLE SPEAKING INDISTINCTLY]. The entire clip is spoken in the alien’s own guttural language. While there are no language identifiers in this clip, there are multiple language IDs in the movie to remind us that the aliens only speak their own language.

Source: Star Trek: The Next Generation, “Descent: Part 2,” 1993. Cable TV (BBC America). Featured caption: [INDISTINCT BORG CHATTER].

Source: Life of Pi, 2012. DVD. Featured captions: (MEN YELLING INDISTINCTLY). This caption is preceded in this scene by the very similar reference to (MEN SCREAMING INDISTINCTLY).

Source: Zero Dark Thirty, 2012. DVD. Featured captions: (LOW, INDISTINCT CONVERSATION), which is preceded ten seconds earlier by (OVERLAPPING RADIO COMMUNICATIONS). Both of these captions attempt to capture the background hum of activity in CIA headquarters as staff pore over information about a possible enemy compound.

Source: Silver Linings Playbook, 2012. DVD. Featured captions: [inaudible chatter] and [announcer chatters indistinctly] partially describe the ambient noise in the ballroom. Also included in this scene (but not in this clip) are multiple references to music [up-tempo music continues] and applause [audience applauding], which further embody the soundscape and give depth to the foreground speech of the characters.

Source: Lincoln, 2012. DVD. Featured captions: (INDISTINCT TALKING), coupled with (SHIP HORN BLOWING), describe the bustle of background activity on the river front.

Source: The Artist, 2011. DVD. Featured captions: [indistinct chattering] and [overlapping chatter] describe the background speech sounds on the set as the crew prepares to film another take of the stars’ dance routine. Some of this background chatter comes forward when captioned verbatim (see “captions equalize”).

Source: The Master, 2012. DVD. Featured caption: [indistinct chatter continues]. This caption is paired with [indistinct chatter]. Together, these captions account for the overlapping conversations of the dinner party.

Ambient sounds that become intertextually or thematically linked

Within the same program, a repeating caption may suggest themes or alert readers to forthcoming action (see chapter 5 on captioned irony). For example, in The Order (2003), starring Heath Ledger, three of the four references to [ Voices Chattering] in the first seventeen minutes and sixteen seconds are visually associated with two creepy kids called “orphans.” When the same caption is repeated a fifth time at 26 minutes and 58 seconds, we are primed to expect the reappearance of the orphans, who fulfill our expectations by reappearing moments later in the new context of the cemetery. The caption becomes intertexually linked to the prior appearances of the same caption, its meaning dependent not only on the context of the scene in which it appears but on the context of its prior appearances as well.
A compilation of five (voices chattering) captions from The Order (2003)
Source: The Order, 2003. DVD. In this compilation of five frames from The Order, [Voices Chattering] is the featured caption in each frame. The two orphan kids in frame one reappear in the moments following the repetition of the same nonspeech caption in later scenes (with the exception of frame two, which takes place in a presumably haunted church). In other words, [Voices Chattering] prefigures the kids’ reappearance. The timestamps for the five captions are: 0:02:48, 0:09:18, 0:16:29, 0:17:16, and 0:26:58.

Source: The Order, 2003. DVD. Featured caption: [Voices Chattering]. In this first appearance of [Voices Chattering], the caption is directly identified with the two orphan kids. Timestamp: 0:02:48.

Source: The Order, 2003. DVD. Featured caption: [Voices Chattering]. In this second appearance of [Voices Chattering], the orphan kids do not return, only a ghostly [Man Whispering]. Timestamp: 0:09:18.

Source: The Order, 2003. DVD. Featured caption: [Voices Chattering]. In the third and fourth appearance of [Voices Chattering], the two orphan kids feature prominently, linking the ghostly voices to the ghostly kids. Timestamps: 0:16:29 and 0:17:16.

So far, the orphans have only appeared in the same location, sitting in the foyer to the priest’s apartment. But when [Voices Chattering] appears a fifth time in the context of the cemetery, we are primed to expect the orphan kids to reappear…and they do.

Note that the chattering voices are faint and easy for listeners to miss when the captions are turned off. The captions serve to bring these faint sounds forward, thematizing them in the process.

Source: The Order, 2003. DVD. Featured caption: [Voices Chattering]. In the fifth appearance of [Voices Chattering], the two orphan kids are called forth, this time in the cemetery, and caption readers are primed to expect them. Timestamp: 0:26:58.

Ambient sounds that are reduced to discrete figures when captioned

Source: Flowers in the Attic, 2014. Cable TV (Lifetime). The featured captions are ambient bookends for this scene: [TRAIN RUMBLING OVER TRACKS] and [ENGINE CHUGGING].

Source: Skyline, 2010. Cable TV (SyFy). Featured captions: [rumbling outside], [mechanical whirring], [electrical humming].

Background ambient sounds that come forward verbatim when captioned

Source: The Happening, 2008. DVD. The featured caption is an example of backchannel speech sounds that come forward when captioned: “[Man] I just walked down a quarter mile. It was clean.”

Source: The Happening, 2008. DVD. A compilation of backchannel sounds that come forward and compete with the foreground speech when captioned verbatim.

Excavating the whispered secret at the end of Lost in Translation

Source: Lost in Translation, 2003. DVD. Featured caption: [Whispering, Indistinct]

Here’s one popular attempt to excavate the meaning of the whisper by applying some audio filters to the clip:

Buy Reading Sounds in paperback and ebook from Amazon.com