Why We Can Give the Deaf Sound, but Not Music

The cochlear implant is a miracle, but advances in pitch processing could give a better quality of life to the hearing impaired.

National Journal
Brian Resnick
Add to Briefcase
See more stories about...
Brian Resnick
Oct. 30, 2013, 1 a.m.

The faces spark with the switch. For some hear­ing noise for the first time, it’s a look of joy­ful awe; for oth­ers it’s star­tle­ment at the sud­den col­or­ing of the vi­bra­tions that al­ways sur­roun­ded them.

This child’s face can ex­plain the mir­acle of the coch­lear im­plant — a device that by­passes a deaf per­son’s dam­aged nerve cells, sim­u­lat­ing a sense of sound in the brain — bet­ter than I can.

That child will prob­ably grow up with a near-nor­mal abil­ity to de­cipher speech in re­l­at­ive quiet. He prob­ably will have a nat­ur­al sound­ing voice. But as of now, he will not be able to hear mu­sic — or at least all the as­pects of it. He’ll hear the rhythm, the beats of the mu­sic, but he won’t be able to dis­cern pitch or the melody. He won’t be able to dif­fer­en­ti­ate between a flute and an oboe. To him, it will all be noise. But maybe not for his whole life: Sci­ence is get­ting closer to giv­ing mu­sic to the deaf.

“You do polls of people who have coch­lear im­plants, the first thing they want to do is hear speech,” says Jay Ru­bin­stein, a sur­geon who in­stalls the im­plants (he’s also fit­tingly the broth­er of Jon Ru­bin­stein, a coin­vent­or of the iPod). “The second thing they want to do gen­er­ally is hear speech in back­ground noise. The third thing is mu­sic.” But we can really only give them the first. Even our best soft­ware can’t dis­tin­guish one voice among many — just try giv­ing Siri a com­mand in a noisy room. And the re­duced pitch in­form­a­tion makes un­der­stand­ing ton­al lan­guages like Chinese much harder. “All the elec­tric­al is all done rather crudely com­pared to nor­mal listen­ing, quite crudely,” says Les At­las, an au­di­olo­gic­al-sys­tems re­search­er at the Uni­versity of Wash­ing­ton.

Let’s take a listen. This is what a per­son with a tra­di­tion­al im­plant might hear when listen­ing to a song (these are all sim­u­la­tions, as it is im­possible to ac­tu­ally hear what an­oth­er hu­man hears, save for out­right tele­pathy).

This is what the rest of us hear.

The dif­fer­ence is in­com­par­able. The first is creepy noise, the second is mu­sic. And that’s the gap that At­las, Ru­bin­stein, and their col­lab­or­at­ors are try­ing to bridge with a new ap­proach to coch­lear-im­plant pro­cessing.

In mod­u­lat­ing the sounds com­ing through the im­plant, At­las and his team have been able to in­crease the per­cep­tion of mu­sic among eight test sub­jects. “Ima­gine that in­stead of play­ing a pi­ano with your fin­gers, you are play­ing it with your fore­arms,” At­las says of the mu­sic that comes through with a tra­di­tion­al pro­cessor. Now “ima­gine you can take those pi­ano keys and you can push them up and down really fast, at the rate of the pitch. It’s 100 times per second, 200 times, 300 times. That’s what we ad­ded.” And in ad­just­ing the noise in that man­ner, those fore­arm smashes of the keys be­gin to sound more like fist smashes of the keys. There’s a bit more res­ol­u­tion to the sound.

Here’s that same file from above, but with At­las’s new pro­cess.

Ob­vi­ously, it isn’t all the way there. But it’s the ori­gins of melody. “They’d like to take it home,” Ru­bin­stein said of the re­ac­tions of the eight sub­jects tested with the new pro­cess. Sev­en of them showed a sub­stan­tial abil­ity to dif­fer­en­ti­ate mu­sic­al  in­stru­ments. Three of the sub­jects showed bet­ter re­cog­ni­tion of melody (though not stat­ist­ic­ally sig­ni­fic­ant in the small sample size.) “But two out of those three showed huge im­prove­ments in melody per­cep­tion.”

This isn’t a su­per­fi­cial en­hance­ment so im­plant users can listen to the latest Top 40s on their Beats by Dre. Life with less stim­u­lus is less stim­u­lat­ing. Be­cause of the de­graded sig­nals that come through the coch­lear im­plants, it takes more men­tal power to pro­cess speech, which can be isol­at­ing.

Hear­ing loss, write re­search­ers in the In­ter­na­tion­al Journ­al of Au­di­ology, of­ten leads to “with­draw­al from so­cial activ­it­ies, re­jec­tion of in­vit­a­tions to parties, and no more vis­its to theatres, cinemas, churches, lec­tures, etc. This, in turn, leads to re­duced in­tel­lec­tu­al and cul­tur­al stim­u­la­tion, and an in­creas­ingly pass­ive and isol­ated cit­izen.” At­las knows this firsthand, hav­ing watched his fath­er and grand­fath­er struggle with hear­ing loss in a mu­sic­al fam­ily. “They seemed at first isol­ated be­cause it was hard to un­der­stand people,” he says, “but felt even more isol­ated be­cause they couldn’t en­joy mu­sic, which was a pretty big part of our fam­ily.” I know it too, re­mem­ber­ing my nearly deaf grandpa re­treat­ing to the world between the couch and closed-cap­tion tele­vi­sion. Hear­ing loss as we age relates to cog­nit­ive de­cline, with those hard of hear­ing de­clin­ing at a rate 40 per­cent faster.

For now, At­las and Ru­bin­stein’s de­vel­op­ments will be con­fined to the lab. “It’s too com­pu­ta­tion­ally com­plex to do on a wear­able speech pro­cessor,” Ru­bin­stein said. They’ll turn now to de­vis­ing a sim­pli­fied ap­proach.

Hear­ing mu­sic for a first time can be a pro­found ex­per­i­ence, be­cause it is so in­teg­ral to the hu­man ex­per­i­ence. “We hu­mans are a mu­sic­al spe­cies no less than a lin­guist­ic one,” writes the neur­o­lo­gist and au­thor Oliv­er Sacks. “And to this largely un­con­scious struc­tur­al ap­pre­ci­ation of mu­sic is ad­ded an of­ten in­tense and pro­found emo­tion­al re­ac­tion to mu­sic.”

Aus­tin Chap­man didn’t have im­plants, but when he got an ad­vanced pair of hear­ing aids that al­lowed him to hear mu­sic for the first time, it was a rev­el­a­tion — in an al­most spir­itu­al sense. “Be­ing able to hear the mu­sic for the first time ever was un­real,” he wrote. “When Moz­art’s “Lac­rimosa” came on, I was blown away by the beauty of it. At one point of the song, it soun­ded like an­gels singing and I sud­denly real­ized that this was the first time I was able to ap­pre­ci­ate mu­sic. Tears rolled down my face and I tried to hide it.”

What We're Following See More »
Morning Consult Poll: Clinton Decisively Won Debate
22 hours ago

"According to a new POLITICO/Morning Consult poll, the first national post-debate survey, 43 percent of registered voters said the Democratic candidate won, compared with 26 percent who opted for the Republican Party’s standard bearer. Her 6-point lead over Trump among likely voters is unchanged from our previous survey: Clinton still leads Trump 42 percent to 36 percent in the race for the White House, with Libertarian nominee Gary Johnson taking 9 percent of the vote."

Trump Draws Laughs, Boos at Al Smith Dinner
1 days ago

After a lighthearted beginning, Donald Trump's appearance at the Al Smith charity dinner in New York "took a tough turn as the crowd repeatedly booed the GOP nominee for his sharp-edged jokes about his rival Hillary Clinton."

McMullin Leads in New Utah Poll
1 days ago

Evan McMul­lin came out on top in a Emer­son Col­lege poll of Utah with 31% of the vote. Donald Trump came in second with 27%, while Hillary Clin­ton took third with 24%. Gary John­son re­ceived 5% of the vote in the sur­vey.

Quinnipiac Has Clinton Up by 7
1 days ago

A new Quin­nipi­ac Uni­versity poll finds Hillary Clin­ton lead­ing Donald Trump by seven percentage points, 47%-40%. Trump’s “lead among men and white voters all but” van­ished from the uni­versity’s early Oc­to­ber poll. A new PPRI/Brook­ings sur­vey shows a much bigger lead, with Clinton up 51%-36%. And an IBD/TIPP poll leans the other way, showing a vir­tu­al dead heat, with Trump tak­ing 41% of the vote to Clin­ton’s 40% in a four-way match­up.

Trump: I’ll Accept the Results “If I Win”
1 days ago

Welcome to National Journal!

You are currently accessing National Journal from IP access. Please login to access this feature. If you have any questions, please contact your Dedicated Advisor.