Chapter 3: Context and subjectivity in sound effects captioning

This supplemental website includes all of the media clips discussed in Reading Sounds (University of Chicago Press, 2015). Purchase your copy of Reading Sounds today in paperback or e-book.

Figure 3.1. How do you caption an ambiguous non-speech sound in a novel context?

In this clip from an episode of Futurama (“Rebirth”), the Hypnotoad sits on four legs surrounded by a background of undifferentiated whiteness. The Hypnotoad is a large cartoon horny toad with unusually large round eyes. These eyes vibrate and change color when animated, giving the toad his powers of hypnosis. He is also wearing a dog collar. His head is positioned to the left of his bumpy body, allowing us to see his multi-colored brown body in profile while his eyes gaze directly at us, hypnotizing his audience just as television hypnotizes its viewers. Caption: (LOUD BUZZING DRONE). 20th Century Fox Television, 2010. DVD.

Source: Futurama, “Rebirth,” DVD, 2010. Featured caption: (LOUD BUZZING DRONE).

Sample sound

Sound can only be effectively captioned in context. Listening to a discrete sound out of context may not provide enough information about what it is or how to caption it. Context matters. Hearing readers might take a moment right now and listen carefully to the sample sound below. How should it be captioned? Out of context, the sound could be described as a violent rumbling, an industrial engine, or angry white noise. The source is unmistakably mechanical and unnatural, the output of some kind of large engine or factory motor, perhaps. Visualized in an audio editing program such as Audacity, the sound wave appears uniformly jagged, with only small differences in the heights of the sharp peaks of the wave. In other words, there are no characteristic breaks in the wave that one would find in visualized speech, breaks indicative of pauses between words. The wave’s uniformity – the absence of a clear start or end point – means that the sound could easily be set to play on an endless loop. It also means that one second in the wave could most likely stand in for any other second. The rumbling engine sound is continuous.

Audio and caption source: Futurama, “Bender should not be allowed on TV,” DVD, 2004. Soundwave created with Audacity; video created with Adobe After Effects.

Hypnotoad — Eyeballs Thrumming Loudly

Source: Futurama, “Bender should not be allowed on TV,” DVD, 2004. Featured caption: [Eyeballs Thrumming Loudly].

Hypnotoad — Every appearance, 2001-2010

“The Day the Earth Stood Stupid”

  • Season 3, Episode 7 (2001)
  • Plot info: “The Day the Earth Stood Stupid
  • Hypnotoad appears at: 01:52 and 04:55.
  • Source: Netflix (videos 1 and 2) and Cable TV (videos 3 and 4)
  • DVD captions are the same as the Netflix captions.
  • Netflix/DVD captions: [ Electronic Humming Sound ], [ Electronic Humming ]
  • Cable TV captions: (low humming), None

“Bender Should Not Be Allowed on Television”

  • Season 4, Episode 6 (2004)
  • Plot info: “Bender Should Not Be Allowed on Television
  • Hypnotoad appears at: 03:57
  • Source: DVD (video 1) and Cable TV (video 2).
  • DVD captions are the same as the Netflix captions.
  • Netflix/DVD captions: [ Eyeballs Thrumming Loudly ]
  • Cable TV captions: ( sustained electrical buzzing )

“Bender’s Big Score”

  • 89-minute film (2007)
  • Plot info: “Bender’s Big Score
  • Hypnotoad appears at: 01:05:37
  • Source: DVD (video 1) and Netflix (video 2).
  • DVD caption: (MECHANICAL GRINDING)
  • Netflix caption: (droning mechanical sputtering)

“Everybody Loves Hypnotoad: Amazon Adventure”

  • 22-minute “extra” on the DVD for Bender’s Big Score (2007)
  • Plot info: “Everybody Loves Hypnotoad
  • Hynptoad appears throughout the 22-minute “episode” (only the first minute is included below)
  • Source: DVD
  • Uncaptioned (typical for DVD extras)

“Into the Wild Green Yonder”

  • 84-minute film (2009)
  • Plot info: “Into the Wild Green Yonder
  • Hypnotoad appears at: 27:15
  • Source: DVD
  • DVD captions are the same as the Netflix captions.
  • Uncaptioned (Hypnotoad appears briefly in background)

“Rebirth”

  • Season 6, Episode 1 (2010)
  • Plot info: “Rebirth
  • Hypnotoad appears at: 0:00 (episode opens with Hypnotoad)
  • Source: DVD (video 1) an d Cable TV (video 2)
  • DVD captions are the same as the Netflix captions.
  • Netflix/DVD captions: (LOUD BUZZING DRONE)
  • Cable TV captions: (deep, distorted electronic tones blaring)

“Attack of the Killer App”

  • Season 6, Episode 3 (2010)
  • Plot info: “Attack of the Killer App
  • Hypnotoad appears at: 07:03
  • Source: DVD (video 1) and Cable TV (video 2)
  • DVD captions are the same as the Netflix captions.
  • Netflix/DVD captions: None
  • Cable TV captions: (mechanical humming)

“Lrrreconcilable Ndndifferences”

  • Season 6, Episode 11 (2010)
  • Plot info: “Lrrreconcilable Ndndifferences
  • Hypnotoad appears at: 09:32
  • Source: DVD (video 1) and Cable TV (video 2)
  • DVD captions are the same as the Netflix captions.
  • Netflix/DVD captions: None
  • Cable TV captions: (electronic static)

Table 3.1. Closed captions for the Hypnotoad’s sound on DVD, Cable TV, and Netflix streaming, 2001-2010

Futurama Title Hypnotoad Caption Source
The Day the Earth Stood Stupid (2001) [ Electronic Humming Sound ]
[ Electronic Humming ]
Netflix and DVD
( low humming )
None
Cable TV
Bender Should Not Be Allowed on Television (2004) [ Eyeballs Thrumming Loudly ] Netflix and DVD
( sustained electrical buzzing ) Cable TV
Bender’s Big Score (2007 DVD movie) (droning mechanical sputtering) Netflix and Cable TV
(MECHANICAL GRINDING) DVD
Everybody Loves Hypnotoad (DVD special feature on Bender’s Big Score) None DVD
Into the Wild Green Yonder (2009 DVD movie) None Netflix and DVD
Rebirth (2010) (LOUD BUZZING DRONE) Netflix and DVD
(deep, distorted electronic tones blaring) Cable TV
Attack of the Killer App (2010) None Netflix and DVD
(mechanical humming) Cable TV
Lrrreconcilable Ndndifferences (2010) None Netflix and DVD
(electronic static) Cable TV

Table 3.2. Is it captioned? Differences between formats in the Futurama episode “The Day the Earth Stood Stupid”

Hypnotoad’s Sound Audience’s Reaction
DVD/Netflix Yes
[ Electronic Humming ]
No
Cable TV No Yes
( clapping in sync )

Applause sounds and captions

Source: 21 Jump Street, 2012, DVD. Featured caption: (APPLAUSE).

Source: Lincoln, 2012, DVD. Featured caption: (SCATTERED APPLAUSE).

Source: The Master, 2012, DVD. Featured caption: [clapping in rhythm].

Source: Zero Dark Thirty, 2012, DVD. Featured caption: (LIGHT APPLAUSE).

Source: Argo, 2012, DVD. Featured caption: [CROWD CHEERING AND APPLAUDING].

Loud sounds and captions

Describing a sound as “loud” is always a popular option, but “loud” can also become an easy fallback option when a more creative or distinctive term might be called for (e.g. “blaring”). In this example from 21 Jump Street (2012), (LOUD MUSIC PLAYING) doesn’t tell us very much about the genre of music or the mood the music is intended to convey. We might guess, given the visual context of young people serving up beer from a keg at a house party, that the music is pop or dance music. The caption should confirm or correct our initial assumptions. Instead, the caption tells us nothing we didn’t already guess about the sounds of teen house parties.

Source: 21 Jump Street, 2012, DVD. Featured caption: (LOUD MUSIC PLAYING)

Source: Lincoln, 2012, DVD. Featured caption: (LOUD PROTESTS)

Source: Moonrise Kingdom, 2012, DVD. Featured caption: (LOUD RUMBLING)

Source: Skyfall, 2012, DVD. Featured caption: (GROANS LOUDLY)

Source: Star Wars Episode 1: The Phantom Menace, 1999, DVD. Featured caption: (LOUD BLUBBERING)

Source: Twilight, 2008, DVD. Featured caption: (LOUD WHOOSH)

Source: Silver Linings Playbook,2012, DVD. Featured caption: [loud clattering]

A line graph that compares how often the Hypnotoad action terms appear in Google’s corpus of over 5 million digitized books, 1900-2010

A partial screenshot of Google's Ngram viewer showing results for hypnotic droning, humming, buzzing, and thrumming

Thrumming sounds and captions

Source: Zero Dark Thirty,2012, DVD. Featured captions: (BLADES THRUMMING), (BLADES THRUMMING LOUDLY), (THRUMMING FADES)

Flatulence in South Park

Source: South Park, “Ass Burgers,” 2011, Hulu Plus. Featured caption: [flatulence]

Table 3.3. Build your own hypnocaptions

The three-column table in the book has been translated here into a form with three drop-down lists. As in the original table, the terms have been culled from the available options. First terms have been placed into drop-down list one, second terms into list two, and third terms into list three. For the sake of efficiency, I’ve reduced (deep, distorted electronic tones blaring) to make the terms fit into three columns. Use any combination of terms to create one-, two-, or three-word hypnocaptions. Make your choices from the columns and use the Submit button to send your choices to me. You can also send your user-generated captions to me directly via email (seanzdenek@gmail.com).

Build your own hypnocaptions

Figure 3.2. Peter hurts his knee in a recurring gag on Fox’s Family Guy

A side-by-side comparison of two identical frames from an episode of Family Guy (“Wasted Talent’) that is part of the recurring hurt knee gag on the show. Peter sits on the sidewalk outside his house, grabbing his left knee with both hands. His eyes are closed, his mouth is open. He looks like he’s in pain. The left frame shows the plain text caption track (Caption: “AHH!”) and the right frame shows the bitmap caption track (Caption: [Howling in pain]). 20th Century Fox Television, 2000. DVD.
Two identical frames from an episode of Family Guy  featuring different captions: AHH! in the text caption track and [Howling in pain]  in the bitmap caption track.

The hurt knee gag on Fox’s Family Guy

Source: Family Guy, “Wasted Talent,” 2000, DVD. Bitmap caption track.

Source: Family Guy, “Wasted Talent,” 2000, DVD. Text caption track.

Source: Family Guy, “Wasted Talent,” 2000, DVD. Bitmap caption track.

Source: Family Guy, “Wasted Talent,” 2000, DVD. Text caption track.

Source: Family Guy, “Brian Goes Back to College,” 2005, DVD. Bitmap caption track.

Source: Family Guy, “Fox-y Lady,” 2009, DVD. Bitmap caption track.

Source: Family Guy, “Something, Something, Something, Dark Side,” 2010, DVD. Bitmap caption track.

Table 3.4. A comparison of the DVD text captions and DVD bitmap captions for the five instances of the hurt knee gag in Family Guy

Episode Bitmap Captions Plain Text Captions
“Wasted Talent” (2.20, 2000) Run home, Peter! Run as fast as you can!
 
[Cheerful instrumental music]
 
[Howling in pain]
RUN HOME, PETER! RUN AS FAST AS YOU CAN!
 
 
AHH!
[Whimpers]
AHH!
[Whimpers]
AHH!
[Whimpers]
AHH!
AHH!
AHH!
AHH!
AHH!
“Wasted Talent” (2.20, 2000) CHUMBA WUMBAS: [Singing]
“Chumba Wumba gobble”
 
[Howling in pain]
♪ CHUMBA WUMBA GOBBLE ♪
 
AHH!
AHH!
[Whimpers]
AHH!
“Brian Goes Back to College” (4.15,
2005)
(WINCING)
 
(GROANING)
Same
“Fox-y Lady” (7.10, 2009) Now run home, Lois! Run as fast as you can!
 
(THE GOLDEN TICKET PLAYING)
 
(EXCLAIMING IN PAIN)
Same
“Something, Something, Something, Dark Side” (8.20, 2010) (EXCLAIMING)
 
(GROANING)
Same

A side-by-side comparison of two episodes from Family Guy featuring the hurt knee gag: the bitmap and text caption tracks from “Wasted Talent,” and the bitmap caption track from “Fox-y Lady.”

Sources: Family Guy, “Wasted Talent” (2000) and “Fox-y Lady” (2009). DVD. Created with Adobe After Effects.

“I’ve got a golden ticket”

When do captions need to cite the cultural touchstones or “prior texts” that inform them? For example, “I’ve Got a Golden Ticket,” a song made famous in Willy Wonka and the Chocolate Factory (1971), connects two instances of the hurt knee gag together. In “Wasted Talent,” an instrumental version of the song is captioned in the bitmap track as [Cheerful instrumental music], but not captioned at all in the text track. In “Fox-y Lady” (2009), the extended gag is duplicated almost frame for frame but this time with Lois Griffin (voiced by Alex Borstein) running down the street and falling in front of their house. On both caption tracks for this episode, the song is captioned as (THE GOLDEN TICKET PLAYING). While “The Golden Ticket” is not technically the title of the song, this caption, unlike the vague reference to cheerful music in “Wasted Talent,” sufficiently alerts viewers to interpret the sequence as a spoof of the scene in Willy Wonka when young Charlie Bucket (Peter Gardner Ostrum) finds the last remaining Golden Ticket hidden inside a chocolate bar wrapper. After a crowd forms around Charlie, a passerby yells: “Run for it, Charlie. Run straight home, and don’t stop ‘til you get there.” Family Guy streamlines this imperative as: “[Now] Run home, Peter [Lois]! Run as fast as you can!” Without access to the name of the song, it’s possible that caption readers will miss the deeper significance of the scene as a parody of a classic movie scene. Recognizing the song as more than cheerful music requires cultural literacy, a topic I take up in Chapter 7.
A comparison of how

Charlie finds a golden ticket in Willy Wonka and the Chocolate Factory (1971)

While lyrics are captioned verbatim on the DVD version of the movie, neither song titles nor instrumental music is described in the captions. Thus,”I’ve got a golden ticket,” the iconic instrumental music that plays as Charlie runs straight home, is not captioned at all.

Source: Charlie and the Chocolate Factory, 1971. DVD.

Gasping sounds in Twilight

For example, of the 190 non-speech descriptions in Twilight, thirteen involve gasping, and twelve of these are associated with Bella. Gasping is used in the caption track as an all-purpose placeholder for the audible intake of air when Bella is scared (during the attempted sexual assault in Port Angeles), startled (when she wakes to find Edward [Robert Pattinson] watching her sleep), aroused (when she and Edward are kissing), excited (when she is lifted by Edward high into the trees), and dying (when Bella gasps three times within a span of thirty-five seconds following the nearly fatal vampire attack at the end of the film). In fact, when we analyze Bella’s character through the non-speech descriptions associated with her, we have to conclude that gasping is one of her key personality traits, at least as far as the captioner was concerned. Bella gasps.

Source: Twilight, 2008. DVD. Featured captions: (SIGHS SOFTLY), (ANXIOUS LAUGH), (BREATH TREMBLING), (GASPS), (LAUGHS SOFTLY), (SIGHS), (BELLA GASPS QUIETLY), (BELLA PANTING), (CHUCKLES SOFTLY), (GRUNTS SOFTLY), (GRUNTS LOUDLY), (BOTH PANTING), (PANTS), (GROANS), (GRUNTING), (RAPID GASPING).

We could continue this line of analysis, identifying the non-speech sounds associated with Edward, Jacob Black (Taylor Lautner), the evil vampires, etc. We could also analyze any non-speech descriptors that are associated with and bond together multiple characters. At one point, for example, when Edward and Bella are kissing for the first time, Bella (GASPS SOFTLY), but Edward breathes heavily too during this scene. In fact, it’s possible that the caption was meant for him. It’s hard to tell because the DVD captions for Twilight do not use placement to indicate who is speaking. In this moment, they are bound together around a single caption spanning multiple breathy sounds between them as they embrace.

Source: Twilight, 2008. DVD. Featured captions: (GASPS SOFTLY), (GRUNTS SOFTLY), (BOTH PANTING).

Captioned thematics of “spritzing” in Sunshine Cleaning

Source: Sunshine Cleaning, 2008. DVD. Featured captions: (spritzes) and “Spritz.”

Captioned thematics of “buzzing” in Hick

Source: Hick, 2012. Showtime. Featured caption: (Buzzing).

“Guttural croaking” as a leitmotif for the Predator in Aliens vs. Predator: Requiem

In Aliens vs. Predator: Requiem (2007), (guttural croaking) is repeated twenty-two times over the course of the movie and always in the context of the Predator’s animalistic noise-making. No one in the movie utters the word guttural or croaking; these words only exist in the captions. Sometimes, viewers encounter the captioned description before they see the Predator. Given that the Predator is usually couched in darkness and sometimes resembles little more than a foreboding dark outline, (guttural croaking) becomes an identity marker as much as a description of a non-speech sound. The recurring caption is the Predator’s leitmotif, as it repeatedly announces his presence in the narrative.

Source: Aliens vs. Predator: Requiem, 2007. DVD. Featured caption: (guttural croaking).

A visual timeline marking each reference to (guttural croaking) in the caption track for Aliens vs. Predator: Requiem

Every instance of (guttural croaking) in Aliens vs. Predator: Requiem (2007). DVD.
Caption No. Start Time Caption
22 00:02:46,321 (low, guttural croaking)
68 00:06:18,366 (guttural croaking)
121 00:09:52,179 (guttural croaking in distance)
286 00:20:14,902 (guttural croaking)
292 00:20:41,528 (low, guttural croaking)
300 00:21:25,038 (guttural croaking)
420 00:31:07,153 (guttural croaking)
422 00:31:13,660 (guttural croaking)
443 00:32:29,936 (guttural croaking)
446 00:32:50,123 (guttural croaking)
452 00:33:26,159 (guttural croaking)
510 00:37:57,930 (guttural croaking)
542 00:40:40,593 (quiet, guttural croaking)
726 00:53:07,506 (guttural croaking)
730 00:53:32,264 (guttural croaking)
783 00:57:33,705 (guttural croaking)
811 00:59:36,261 (guttural croaking)
1012 01:09:07,498 (guttural croaking)
1079 01:13:14,478 (guttural croaking)
1175 01:20:10,293 (guttural croaking)
1302 01:27:15,618 (guttural croaking)
1335 01:28:46,075 (guttural croaking)

Epilogue — Hypnotoad in “Decision 3012”

Source: Futurama, “Decision 3012,” 2012. DVD. The Hypnotoad’s characteristic sound is uncaptioned.

Source: Futurama, “Decision 3012,” 2012. DVD. The Hypnotoad’s characteristic sound is uncaptioned.

Buy Reading Sounds in paperback and ebook from Amazon.com