Why We Can Give the Deaf Sound, but Not Music

The cochlear implant is a miracle, but advances in pitch processing could give a better quality of life to the hearing impaired.

National Journal
Brian Resnick
Add to Briefcase
See more stories about...
Brian Resnick
Oct. 30, 2013, 1 a.m.

The faces spark with the switch. For some hear­ing noise for the first time, it’s a look of joy­ful awe; for oth­ers it’s star­tle­ment at the sud­den col­or­ing of the vi­bra­tions that al­ways sur­roun­ded them.

This child’s face can ex­plain the mir­acle of the coch­lear im­plant — a device that by­passes a deaf per­son’s dam­aged nerve cells, sim­u­lat­ing a sense of sound in the brain — bet­ter than I can.

That child will prob­ably grow up with a near-nor­mal abil­ity to de­cipher speech in re­l­at­ive quiet. He prob­ably will have a nat­ur­al sound­ing voice. But as of now, he will not be able to hear mu­sic — or at least all the as­pects of it. He’ll hear the rhythm, the beats of the mu­sic, but he won’t be able to dis­cern pitch or the melody. He won’t be able to dif­fer­en­ti­ate between a flute and an oboe. To him, it will all be noise. But maybe not for his whole life: Sci­ence is get­ting closer to giv­ing mu­sic to the deaf.

“You do polls of people who have coch­lear im­plants, the first thing they want to do is hear speech,” says Jay Ru­bin­stein, a sur­geon who in­stalls the im­plants (he’s also fit­tingly the broth­er of Jon Ru­bin­stein, a coin­vent­or of the iPod). “The second thing they want to do gen­er­ally is hear speech in back­ground noise. The third thing is mu­sic.” But we can really only give them the first. Even our best soft­ware can’t dis­tin­guish one voice among many — just try giv­ing Siri a com­mand in a noisy room. And the re­duced pitch in­form­a­tion makes un­der­stand­ing ton­al lan­guages like Chinese much harder. “All the elec­tric­al is all done rather crudely com­pared to nor­mal listen­ing, quite crudely,” says Les At­las, an au­di­olo­gic­al-sys­tems re­search­er at the Uni­versity of Wash­ing­ton.

Let’s take a listen. This is what a per­son with a tra­di­tion­al im­plant might hear when listen­ing to a song (these are all sim­u­la­tions, as it is im­possible to ac­tu­ally hear what an­oth­er hu­man hears, save for out­right tele­pathy).

This is what the rest of us hear.

The dif­fer­ence is in­com­par­able. The first is creepy noise, the second is mu­sic. And that’s the gap that At­las, Ru­bin­stein, and their col­lab­or­at­ors are try­ing to bridge with a new ap­proach to coch­lear-im­plant pro­cessing.

In mod­u­lat­ing the sounds com­ing through the im­plant, At­las and his team have been able to in­crease the per­cep­tion of mu­sic among eight test sub­jects. “Ima­gine that in­stead of play­ing a pi­ano with your fin­gers, you are play­ing it with your fore­arms,” At­las says of the mu­sic that comes through with a tra­di­tion­al pro­cessor. Now “ima­gine you can take those pi­ano keys and you can push them up and down really fast, at the rate of the pitch. It’s 100 times per second, 200 times, 300 times. That’s what we ad­ded.” And in ad­just­ing the noise in that man­ner, those fore­arm smashes of the keys be­gin to sound more like fist smashes of the keys. There’s a bit more res­ol­u­tion to the sound.

Here’s that same file from above, but with At­las’s new pro­cess.

Ob­vi­ously, it isn’t all the way there. But it’s the ori­gins of melody. “They’d like to take it home,” Ru­bin­stein said of the re­ac­tions of the eight sub­jects tested with the new pro­cess. Sev­en of them showed a sub­stan­tial abil­ity to dif­fer­en­ti­ate mu­sic­al  in­stru­ments. Three of the sub­jects showed bet­ter re­cog­ni­tion of melody (though not stat­ist­ic­ally sig­ni­fic­ant in the small sample size.) “But two out of those three showed huge im­prove­ments in melody per­cep­tion.”

This isn’t a su­per­fi­cial en­hance­ment so im­plant users can listen to the latest Top 40s on their Beats by Dre. Life with less stim­u­lus is less stim­u­lat­ing. Be­cause of the de­graded sig­nals that come through the coch­lear im­plants, it takes more men­tal power to pro­cess speech, which can be isol­at­ing.

Hear­ing loss, write re­search­ers in the In­ter­na­tion­al Journ­al of Au­di­ology, of­ten leads to “with­draw­al from so­cial activ­it­ies, re­jec­tion of in­vit­a­tions to parties, and no more vis­its to theatres, cinemas, churches, lec­tures, etc. This, in turn, leads to re­duced in­tel­lec­tu­al and cul­tur­al stim­u­la­tion, and an in­creas­ingly pass­ive and isol­ated cit­izen.” At­las knows this firsthand, hav­ing watched his fath­er and grand­fath­er struggle with hear­ing loss in a mu­sic­al fam­ily. “They seemed at first isol­ated be­cause it was hard to un­der­stand people,” he says, “but felt even more isol­ated be­cause they couldn’t en­joy mu­sic, which was a pretty big part of our fam­ily.” I know it too, re­mem­ber­ing my nearly deaf grandpa re­treat­ing to the world between the couch and closed-cap­tion tele­vi­sion. Hear­ing loss as we age relates to cog­nit­ive de­cline, with those hard of hear­ing de­clin­ing at a rate 40 per­cent faster.

For now, At­las and Ru­bin­stein’s de­vel­op­ments will be con­fined to the lab. “It’s too com­pu­ta­tion­ally com­plex to do on a wear­able speech pro­cessor,” Ru­bin­stein said. They’ll turn now to de­vis­ing a sim­pli­fied ap­proach.

Hear­ing mu­sic for a first time can be a pro­found ex­per­i­ence, be­cause it is so in­teg­ral to the hu­man ex­per­i­ence. “We hu­mans are a mu­sic­al spe­cies no less than a lin­guist­ic one,” writes the neur­o­lo­gist and au­thor Oliv­er Sacks. “And to this largely un­con­scious struc­tur­al ap­pre­ci­ation of mu­sic is ad­ded an of­ten in­tense and pro­found emo­tion­al re­ac­tion to mu­sic.”

Aus­tin Chap­man didn’t have im­plants, but when he got an ad­vanced pair of hear­ing aids that al­lowed him to hear mu­sic for the first time, it was a rev­el­a­tion — in an al­most spir­itu­al sense. “Be­ing able to hear the mu­sic for the first time ever was un­real,” he wrote. “When Moz­art’s “Lac­rimosa” came on, I was blown away by the beauty of it. At one point of the song, it soun­ded like an­gels singing and I sud­denly real­ized that this was the first time I was able to ap­pre­ci­ate mu­sic. Tears rolled down my face and I tried to hide it.”

What We're Following See More »
Spicer Excludes NYT, CNN, BuzzFeed from Press Gaggle
11 hours ago
NRA Chief: Leftist Protesters Are Paid
12 hours ago
Trump Still on Campaign Rhetoric
15 hours ago
Trump Rails On Obamacare
15 hours ago

After spending a few minutes re-litigating the Democratic primary, Donald Trump turned his focus to Obamacare. “I inherited a mess, believe me. We also inherited a failed healthcare law that threatens our medical system with absolute and total catastrophe” he said. “I’ve been watching and nobody says it, but Obamacare doesn’t work.” He finished, "so we're going to repeal and replace Obamacare."

Trump Goes After The Media
15 hours ago

Donald Trump lobbed his first attack at the “dishonest media” about a minute into his speech, saying that the media would not appropriately cover the standing ovation that he received. “We are fighting the fake news,” he said, before doubling down on his previous claim that the press is “the enemy of the people." However, he made a distinction, saying that he doesn't think all media is the enemy, just the "fake news."


Welcome to National Journal!

You are currently accessing National Journal from IP access. Please login to access this feature. If you have any questions, please contact your Dedicated Advisor.