Why We Can Give the Deaf Sound, but Not Music

The cochlear implant is a miracle, but advances in pitch processing could give a better quality of life to the hearing impaired.

National Journal
Brian Resnick
Add to Briefcase
See more stories about...
Brian Resnick
Oct. 30, 2013, 1 a.m.

The faces spark with the switch. For some hear­ing noise for the first time, it’s a look of joy­ful awe; for oth­ers it’s star­tle­ment at the sud­den col­or­ing of the vi­bra­tions that al­ways sur­roun­ded them.

This child’s face can ex­plain the mir­acle of the coch­lear im­plant — a device that by­passes a deaf per­son’s dam­aged nerve cells, sim­u­lat­ing a sense of sound in the brain — bet­ter than I can.

That child will prob­ably grow up with a near-nor­mal abil­ity to de­cipher speech in re­l­at­ive quiet. He prob­ably will have a nat­ur­al sound­ing voice. But as of now, he will not be able to hear mu­sic — or at least all the as­pects of it. He’ll hear the rhythm, the beats of the mu­sic, but he won’t be able to dis­cern pitch or the melody. He won’t be able to dif­fer­en­ti­ate between a flute and an oboe. To him, it will all be noise. But maybe not for his whole life: Sci­ence is get­ting closer to giv­ing mu­sic to the deaf.

“You do polls of people who have coch­lear im­plants, the first thing they want to do is hear speech,” says Jay Ru­bin­stein, a sur­geon who in­stalls the im­plants (he’s also fit­tingly the broth­er of Jon Ru­bin­stein, a coin­vent­or of the iPod). “The second thing they want to do gen­er­ally is hear speech in back­ground noise. The third thing is mu­sic.” But we can really only give them the first. Even our best soft­ware can’t dis­tin­guish one voice among many — just try giv­ing Siri a com­mand in a noisy room. And the re­duced pitch in­form­a­tion makes un­der­stand­ing ton­al lan­guages like Chinese much harder. “All the elec­tric­al is all done rather crudely com­pared to nor­mal listen­ing, quite crudely,” says Les At­las, an au­di­olo­gic­al-sys­tems re­search­er at the Uni­versity of Wash­ing­ton.

Let’s take a listen. This is what a per­son with a tra­di­tion­al im­plant might hear when listen­ing to a song (these are all sim­u­la­tions, as it is im­possible to ac­tu­ally hear what an­oth­er hu­man hears, save for out­right tele­pathy).

This is what the rest of us hear.

The dif­fer­ence is in­com­par­able. The first is creepy noise, the second is mu­sic. And that’s the gap that At­las, Ru­bin­stein, and their col­lab­or­at­ors are try­ing to bridge with a new ap­proach to coch­lear-im­plant pro­cessing.

In mod­u­lat­ing the sounds com­ing through the im­plant, At­las and his team have been able to in­crease the per­cep­tion of mu­sic among eight test sub­jects. “Ima­gine that in­stead of play­ing a pi­ano with your fin­gers, you are play­ing it with your fore­arms,” At­las says of the mu­sic that comes through with a tra­di­tion­al pro­cessor. Now “ima­gine you can take those pi­ano keys and you can push them up and down really fast, at the rate of the pitch. It’s 100 times per second, 200 times, 300 times. That’s what we ad­ded.” And in ad­just­ing the noise in that man­ner, those fore­arm smashes of the keys be­gin to sound more like fist smashes of the keys. There’s a bit more res­ol­u­tion to the sound.

Here’s that same file from above, but with At­las’s new pro­cess.

Ob­vi­ously, it isn’t all the way there. But it’s the ori­gins of melody. “They’d like to take it home,” Ru­bin­stein said of the re­ac­tions of the eight sub­jects tested with the new pro­cess. Sev­en of them showed a sub­stan­tial abil­ity to dif­fer­en­ti­ate mu­sic­al  in­stru­ments. Three of the sub­jects showed bet­ter re­cog­ni­tion of melody (though not stat­ist­ic­ally sig­ni­fic­ant in the small sample size.) “But two out of those three showed huge im­prove­ments in melody per­cep­tion.”

This isn’t a su­per­fi­cial en­hance­ment so im­plant users can listen to the latest Top 40s on their Beats by Dre. Life with less stim­u­lus is less stim­u­lat­ing. Be­cause of the de­graded sig­nals that come through the coch­lear im­plants, it takes more men­tal power to pro­cess speech, which can be isol­at­ing.

Hear­ing loss, write re­search­ers in the In­ter­na­tion­al Journ­al of Au­di­ology, of­ten leads to “with­draw­al from so­cial activ­it­ies, re­jec­tion of in­vit­a­tions to parties, and no more vis­its to theatres, cinemas, churches, lec­tures, etc. This, in turn, leads to re­duced in­tel­lec­tu­al and cul­tur­al stim­u­la­tion, and an in­creas­ingly pass­ive and isol­ated cit­izen.” At­las knows this firsthand, hav­ing watched his fath­er and grand­fath­er struggle with hear­ing loss in a mu­sic­al fam­ily. “They seemed at first isol­ated be­cause it was hard to un­der­stand people,” he says, “but felt even more isol­ated be­cause they couldn’t en­joy mu­sic, which was a pretty big part of our fam­ily.” I know it too, re­mem­ber­ing my nearly deaf grandpa re­treat­ing to the world between the couch and closed-cap­tion tele­vi­sion. Hear­ing loss as we age relates to cog­nit­ive de­cline, with those hard of hear­ing de­clin­ing at a rate 40 per­cent faster.

For now, At­las and Ru­bin­stein’s de­vel­op­ments will be con­fined to the lab. “It’s too com­pu­ta­tion­ally com­plex to do on a wear­able speech pro­cessor,” Ru­bin­stein said. They’ll turn now to de­vis­ing a sim­pli­fied ap­proach.

Hear­ing mu­sic for a first time can be a pro­found ex­per­i­ence, be­cause it is so in­teg­ral to the hu­man ex­per­i­ence. “We hu­mans are a mu­sic­al spe­cies no less than a lin­guist­ic one,” writes the neur­o­lo­gist and au­thor Oliv­er Sacks. “And to this largely un­con­scious struc­tur­al ap­pre­ci­ation of mu­sic is ad­ded an of­ten in­tense and pro­found emo­tion­al re­ac­tion to mu­sic.”

Aus­tin Chap­man didn’t have im­plants, but when he got an ad­vanced pair of hear­ing aids that al­lowed him to hear mu­sic for the first time, it was a rev­el­a­tion — in an al­most spir­itu­al sense. “Be­ing able to hear the mu­sic for the first time ever was un­real,” he wrote. “When Moz­art’s “Lac­rimosa” came on, I was blown away by the beauty of it. At one point of the song, it soun­ded like an­gels singing and I sud­denly real­ized that this was the first time I was able to ap­pre­ci­ate mu­sic. Tears rolled down my face and I tried to hide it.”

What We're Following See More »
Nuclear Bombers Preparing to Go Back on 24-Hour Alert
40 minutes ago

"The U.S. Air Force is preparing to put nuclear-armed bombers back on 24-hour ready alert, a status not seen since the Cold War ended in 1991...Putting the B-52s back on alert is just one of many decisions facing the Air Force as the U.S. military responds to a changing geopolitical environment that includes North Korea’s rapidly advancing nuclear arsenal, President Trump’s confrontational approach to Pyongyang, and Russia’s increasingly potent and active armed forces."

Senate Intel Postpones Testimony by Cohen
2 days ago
Senate Rejects Effort to Nix SALT Tax Changes
3 days ago

"Senate Democrats on Thursday failed in their first attempt to save the state and local tax deduction, which helps many residents of California and other high-cost states reduce their federal income tax bills. The Republican-controlled Senate voted 52-47 to reject an amendment that would have prevented the Senate from considering any bill that repeals or limits the deduction as part of a planned tax overhaul."

Lewandowski Meets with Senate Intelligence Committee
3 days ago

"President Donald Trump's former campaign manager Corey Lewandowski appeared on Capitol Hill for a closed-door interview with the Senate intelligence committee Wednesday, according to a source familiar with the matter. Lewandowski is the latest senior official in Trump's orbit who has met with the committee as part of its investigation into Russian election meddling and possible collusion with the Trump campaign."

Some Members Seek to Wrap Up Russia Investigations by Year’s End
4 days ago

"A growing number of key Republicans are sending this message to the leaders of the congressional committees investigating potential Trump campaign collusion with the Russians: Wrap it up soon. In the House and Senate, several Republicans who sit on key committees are starting to grumble that the investigations have spanned the better part of the past nine months, contending that the Democratic push to extend the investigation well into next year could amount to a fishing expedition."


Welcome to National Journal!

You are currently accessing National Journal from IP access. Please login to access this feature. If you have any questions, please contact your Dedicated Advisor.