Why We Can Give the Deaf Sound, but Not Music

The cochlear implant is a miracle, but advances in pitch processing could give a better quality of life to the hearing impaired.

National Journal
Brian Resnick
Add to Briefcase
See more stories about...
Brian Resnick
Oct. 30, 2013, 1 a.m.

The faces spark with the switch. For some hear­ing noise for the first time, it’s a look of joy­ful awe; for oth­ers it’s star­tle­ment at the sud­den col­or­ing of the vi­bra­tions that al­ways sur­roun­ded them.

This child’s face can ex­plain the mir­acle of the coch­lear im­plant — a device that by­passes a deaf per­son’s dam­aged nerve cells, sim­u­lat­ing a sense of sound in the brain — bet­ter than I can.

That child will prob­ably grow up with a near-nor­mal abil­ity to de­cipher speech in re­l­at­ive quiet. He prob­ably will have a nat­ur­al sound­ing voice. But as of now, he will not be able to hear mu­sic — or at least all the as­pects of it. He’ll hear the rhythm, the beats of the mu­sic, but he won’t be able to dis­cern pitch or the melody. He won’t be able to dif­fer­en­ti­ate between a flute and an oboe. To him, it will all be noise. But maybe not for his whole life: Sci­ence is get­ting closer to giv­ing mu­sic to the deaf.

“You do polls of people who have coch­lear im­plants, the first thing they want to do is hear speech,” says Jay Ru­bin­stein, a sur­geon who in­stalls the im­plants (he’s also fit­tingly the broth­er of Jon Ru­bin­stein, a coin­vent­or of the iPod). “The second thing they want to do gen­er­ally is hear speech in back­ground noise. The third thing is mu­sic.” But we can really only give them the first. Even our best soft­ware can’t dis­tin­guish one voice among many — just try giv­ing Siri a com­mand in a noisy room. And the re­duced pitch in­form­a­tion makes un­der­stand­ing ton­al lan­guages like Chinese much harder. “All the elec­tric­al is all done rather crudely com­pared to nor­mal listen­ing, quite crudely,” says Les At­las, an au­di­olo­gic­al-sys­tems re­search­er at the Uni­versity of Wash­ing­ton.

Let’s take a listen. This is what a per­son with a tra­di­tion­al im­plant might hear when listen­ing to a song (these are all sim­u­la­tions, as it is im­possible to ac­tu­ally hear what an­oth­er hu­man hears, save for out­right tele­pathy).

This is what the rest of us hear.

The dif­fer­ence is in­com­par­able. The first is creepy noise, the second is mu­sic. And that’s the gap that At­las, Ru­bin­stein, and their col­lab­or­at­ors are try­ing to bridge with a new ap­proach to coch­lear-im­plant pro­cessing.

In mod­u­lat­ing the sounds com­ing through the im­plant, At­las and his team have been able to in­crease the per­cep­tion of mu­sic among eight test sub­jects. “Ima­gine that in­stead of play­ing a pi­ano with your fin­gers, you are play­ing it with your fore­arms,” At­las says of the mu­sic that comes through with a tra­di­tion­al pro­cessor. Now “ima­gine you can take those pi­ano keys and you can push them up and down really fast, at the rate of the pitch. It’s 100 times per second, 200 times, 300 times. That’s what we ad­ded.” And in ad­just­ing the noise in that man­ner, those fore­arm smashes of the keys be­gin to sound more like fist smashes of the keys. There’s a bit more res­ol­u­tion to the sound.

Here’s that same file from above, but with At­las’s new pro­cess.

Ob­vi­ously, it isn’t all the way there. But it’s the ori­gins of melody. “They’d like to take it home,” Ru­bin­stein said of the re­ac­tions of the eight sub­jects tested with the new pro­cess. Sev­en of them showed a sub­stan­tial abil­ity to dif­fer­en­ti­ate mu­sic­al  in­stru­ments. Three of the sub­jects showed bet­ter re­cog­ni­tion of melody (though not stat­ist­ic­ally sig­ni­fic­ant in the small sample size.) “But two out of those three showed huge im­prove­ments in melody per­cep­tion.”

This isn’t a su­per­fi­cial en­hance­ment so im­plant users can listen to the latest Top 40s on their Beats by Dre. Life with less stim­u­lus is less stim­u­lat­ing. Be­cause of the de­graded sig­nals that come through the coch­lear im­plants, it takes more men­tal power to pro­cess speech, which can be isol­at­ing.

Hear­ing loss, write re­search­ers in the In­ter­na­tion­al Journ­al of Au­di­ology, of­ten leads to “with­draw­al from so­cial activ­it­ies, re­jec­tion of in­vit­a­tions to parties, and no more vis­its to theatres, cinemas, churches, lec­tures, etc. This, in turn, leads to re­duced in­tel­lec­tu­al and cul­tur­al stim­u­la­tion, and an in­creas­ingly pass­ive and isol­ated cit­izen.” At­las knows this firsthand, hav­ing watched his fath­er and grand­fath­er struggle with hear­ing loss in a mu­sic­al fam­ily. “They seemed at first isol­ated be­cause it was hard to un­der­stand people,” he says, “but felt even more isol­ated be­cause they couldn’t en­joy mu­sic, which was a pretty big part of our fam­ily.” I know it too, re­mem­ber­ing my nearly deaf grandpa re­treat­ing to the world between the couch and closed-cap­tion tele­vi­sion. Hear­ing loss as we age relates to cog­nit­ive de­cline, with those hard of hear­ing de­clin­ing at a rate 40 per­cent faster.

For now, At­las and Ru­bin­stein’s de­vel­op­ments will be con­fined to the lab. “It’s too com­pu­ta­tion­ally com­plex to do on a wear­able speech pro­cessor,” Ru­bin­stein said. They’ll turn now to de­vis­ing a sim­pli­fied ap­proach.

Hear­ing mu­sic for a first time can be a pro­found ex­per­i­ence, be­cause it is so in­teg­ral to the hu­man ex­per­i­ence. “We hu­mans are a mu­sic­al spe­cies no less than a lin­guist­ic one,” writes the neur­o­lo­gist and au­thor Oliv­er Sacks. “And to this largely un­con­scious struc­tur­al ap­pre­ci­ation of mu­sic is ad­ded an of­ten in­tense and pro­found emo­tion­al re­ac­tion to mu­sic.”

Aus­tin Chap­man didn’t have im­plants, but when he got an ad­vanced pair of hear­ing aids that al­lowed him to hear mu­sic for the first time, it was a rev­el­a­tion — in an al­most spir­itu­al sense. “Be­ing able to hear the mu­sic for the first time ever was un­real,” he wrote. “When Moz­art’s “Lac­rimosa” came on, I was blown away by the beauty of it. At one point of the song, it soun­ded like an­gels singing and I sud­denly real­ized that this was the first time I was able to ap­pre­ci­ate mu­sic. Tears rolled down my face and I tried to hide it.”

What We're Following See More »
John McCain Paid Secret Visit To Syria
3 hours ago

Senator John McCain paid a secret visit to Northern Syria over the weekend during his trip abroad. McCain reportedly went "to speak with American officials and Kurdish fighters leading the charge to push ISIS militants out of Raqqa, the jihadist group’s stronghold." The trip was organized with the help of U.S. military.

Trump Budget to Call for Major Cuts
3 hours ago

"The Trump administration will deliver its first budget to Congress in mid-March, and the president confirmed Wednesday it will contain major cuts for federal agencies." The blueprint, expected to be released in mid-March, will not include the kinds of specifics usually seen in White House budgets, but rather will instruct the heads of agencies to "do more with less."

More Republicans Trust Trump than GOP Members
5 hours ago
Sanders Acolytes Taking the Movement Local
7 hours ago

"While Democrats nationwide have put the focus on President Trump, the Sanders wing of the party has engaged in an intramural fight to remake the party in a more populist, liberal mold." From Washington state to California to Florida, Sanders loyalists are making good on their promise to remake the party from the ground up. And just last week, a "group of former Sanders campaign aides launched a super PAC with the explicit goal of mounting primary challenges to Democratic incumbents."

McMaster Requires Congressional Approval
7 hours ago

Congress will need to vote on Donald Trump's pick of Lt. General H.R. McMaster to be his next national security adviser, but not for the reason you think. The position of NSA doesn't require Senate approval, but since McMaster currently holds a three-star military position, Congress will need to vote to allow him to keep his position instead of forcing him to drop one star and become a Major General, which could potentially affect his pension.


Welcome to National Journal!

You are currently accessing National Journal from IP access. Please login to access this feature. If you have any questions, please contact your Dedicated Advisor.