Why We Can Give the Deaf Sound, but Not Music

The cochlear implant is a miracle, but advances in pitch processing could give a better quality of life to the hearing impaired.

National Journal
Brian Resnick
Add to Briefcase
Brian Resnick
Oct. 30, 2013, 1 a.m.

The faces spark with the switch. For some hear­ing noise for the first time, it’s a look of joy­ful awe; for oth­ers it’s star­tle­ment at the sud­den col­or­ing of the vi­bra­tions that al­ways sur­roun­ded them.

This child’s face can ex­plain the mir­acle of the coch­lear im­plant — a device that by­passes a deaf per­son’s dam­aged nerve cells, sim­u­lat­ing a sense of sound in the brain — bet­ter than I can.

That child will prob­ably grow up with a near-nor­mal abil­ity to de­cipher speech in re­l­at­ive quiet. He prob­ably will have a nat­ur­al sound­ing voice. But as of now, he will not be able to hear mu­sic — or at least all the as­pects of it. He’ll hear the rhythm, the beats of the mu­sic, but he won’t be able to dis­cern pitch or the melody. He won’t be able to dif­fer­en­ti­ate between a flute and an oboe. To him, it will all be noise. But maybe not for his whole life: Sci­ence is get­ting closer to giv­ing mu­sic to the deaf.

“You do polls of people who have coch­lear im­plants, the first thing they want to do is hear speech,” says Jay Ru­bin­stein, a sur­geon who in­stalls the im­plants (he’s also fit­tingly the broth­er of Jon Ru­bin­stein, a coin­vent­or of the iPod). “The second thing they want to do gen­er­ally is hear speech in back­ground noise. The third thing is mu­sic.” But we can really only give them the first. Even our best soft­ware can’t dis­tin­guish one voice among many — just try giv­ing Siri a com­mand in a noisy room. And the re­duced pitch in­form­a­tion makes un­der­stand­ing ton­al lan­guages like Chinese much harder. “All the elec­tric­al is all done rather crudely com­pared to nor­mal listen­ing, quite crudely,” says Les At­las, an au­di­olo­gic­al-sys­tems re­search­er at the Uni­versity of Wash­ing­ton.

Let’s take a listen. This is what a per­son with a tra­di­tion­al im­plant might hear when listen­ing to a song (these are all sim­u­la­tions, as it is im­possible to ac­tu­ally hear what an­oth­er hu­man hears, save for out­right tele­pathy).

This is what the rest of us hear.

The dif­fer­ence is in­com­par­able. The first is creepy noise, the second is mu­sic. And that’s the gap that At­las, Ru­bin­stein, and their col­lab­or­at­ors are try­ing to bridge with a new ap­proach to coch­lear-im­plant pro­cessing.

In mod­u­lat­ing the sounds com­ing through the im­plant, At­las and his team have been able to in­crease the per­cep­tion of mu­sic among eight test sub­jects. “Ima­gine that in­stead of play­ing a pi­ano with your fin­gers, you are play­ing it with your fore­arms,” At­las says of the mu­sic that comes through with a tra­di­tion­al pro­cessor. Now “ima­gine you can take those pi­ano keys and you can push them up and down really fast, at the rate of the pitch. It’s 100 times per second, 200 times, 300 times. That’s what we ad­ded.” And in ad­just­ing the noise in that man­ner, those fore­arm smashes of the keys be­gin to sound more like fist smashes of the keys. There’s a bit more res­ol­u­tion to the sound.

Here’s that same file from above, but with At­las’s new pro­cess.

Ob­vi­ously, it isn’t all the way there. But it’s the ori­gins of melody. “They’d like to take it home,” Ru­bin­stein said of the re­ac­tions of the eight sub­jects tested with the new pro­cess. Sev­en of them showed a sub­stan­tial abil­ity to dif­fer­en­ti­ate mu­sic­al  in­stru­ments. Three of the sub­jects showed bet­ter re­cog­ni­tion of melody (though not stat­ist­ic­ally sig­ni­fic­ant in the small sample size.) “But two out of those three showed huge im­prove­ments in melody per­cep­tion.”

This isn’t a su­per­fi­cial en­hance­ment so im­plant users can listen to the latest Top 40s on their Beats by Dre. Life with less stim­u­lus is less stim­u­lat­ing. Be­cause of the de­graded sig­nals that come through the coch­lear im­plants, it takes more men­tal power to pro­cess speech, which can be isol­at­ing.

Hear­ing loss, write re­search­ers in the In­ter­na­tion­al Journ­al of Au­di­ology, of­ten leads to “with­draw­al from so­cial activ­it­ies, re­jec­tion of in­vit­a­tions to parties, and no more vis­its to theatres, cinemas, churches, lec­tures, etc. This, in turn, leads to re­duced in­tel­lec­tu­al and cul­tur­al stim­u­la­tion, and an in­creas­ingly pass­ive and isol­ated cit­izen.” At­las knows this firsthand, hav­ing watched his fath­er and grand­fath­er struggle with hear­ing loss in a mu­sic­al fam­ily. “They seemed at first isol­ated be­cause it was hard to un­der­stand people,” he says, “but felt even more isol­ated be­cause they couldn’t en­joy mu­sic, which was a pretty big part of our fam­ily.” I know it too, re­mem­ber­ing my nearly deaf grandpa re­treat­ing to the world between the couch and closed-cap­tion tele­vi­sion. Hear­ing loss as we age relates to cog­nit­ive de­cline, with those hard of hear­ing de­clin­ing at a rate 40 per­cent faster.

For now, At­las and Ru­bin­stein’s de­vel­op­ments will be con­fined to the lab. “It’s too com­pu­ta­tion­ally com­plex to do on a wear­able speech pro­cessor,” Ru­bin­stein said. They’ll turn now to de­vis­ing a sim­pli­fied ap­proach.

Hear­ing mu­sic for a first time can be a pro­found ex­per­i­ence, be­cause it is so in­teg­ral to the hu­man ex­per­i­ence. “We hu­mans are a mu­sic­al spe­cies no less than a lin­guist­ic one,” writes the neur­o­lo­gist and au­thor Oliv­er Sacks. “And to this largely un­con­scious struc­tur­al ap­pre­ci­ation of mu­sic is ad­ded an of­ten in­tense and pro­found emo­tion­al re­ac­tion to mu­sic.”

Aus­tin Chap­man didn’t have im­plants, but when he got an ad­vanced pair of hear­ing aids that al­lowed him to hear mu­sic for the first time, it was a rev­el­a­tion — in an al­most spir­itu­al sense. “Be­ing able to hear the mu­sic for the first time ever was un­real,” he wrote. “When Moz­art’s “Lac­rimosa” came on, I was blown away by the beauty of it. At one point of the song, it soun­ded like an­gels singing and I sud­denly real­ized that this was the first time I was able to ap­pre­ci­ate mu­sic. Tears rolled down my face and I tried to hide it.”

What We're Following See More »
Why Yes, Mueller Is Looking into Trump Businesses
2 hours ago

In addition to ties between Russia and the Trump campaign, Robert Mueller's team is also "examining a broad range of transactions involving Trump’s businesses as well as those of his associates, according to a person familiar with the probe. FBI investigators and others are looking at Russian purchases of apartments in Trump buildings, Trump’s involvement in a controversial SoHo development in New York with Russian associates, the 2013 Miss Universe pageant in Moscow, and Trump’s sale of a Florida mansion to a Russian oligarch in 2008, the person said. The investigation also has absorbed a money-laundering probe begun by federal prosecutors in New York into Trump’s former campaign chairman Paul Manafort."

House Reauthorizes DHS
2 hours ago

"The House voted Thursday to reauthorize the Department of Homeland Security. The bipartisan measure passed easily by a vote of 386-41, with nine Republicans and 32 Democrats voting in opposition. If the bill makes it through the Senate, it would be the first-ever reauthorization of the Department of Homeland Security (DHS) since it was created in the wake of the Sept. 11, 2001, terrorist attacks." Among the provisions it contains is a mandate that the Senate confirm the Secret Service director. It also boosts funding for the Urban Area Security Initiative by $195 million per year.

AFT’s Weingarten Likens Voucher Support to Segregation
2 hours ago

In remarks scheduled to be delivered today at the American Federation of Teachers' summer conference, President Randi Weingarten "likens U.S. Education Secretary Betsy DeVos to a climate-change denier" and "says the Trump administration's school choice plans are secretly intended to starve funding from public schools. She calls taxpayer-funded private school vouchers, tuition tax credits and the like 'only slightly more polite cousins of segregation.'" The pro-voucher Center for Education Reform said teachers should "consider inviting Weingarten’s resignation."

Trump Has Confidence in Sessions
3 hours ago

"President Trump has confidence in Attorney General Jeff Sessions, despite his criticism of the Justice Department head's decision to recuse himself from the Russia probe, the White House said Thursday. 'Clearly he has confidence in him or he would not be the attorney general,' spokeswoman Sarah Huckabee Sanders told reporters at an off-camera briefing."

ACLU Suing Trump Administration for Planned Executive Order
5 hours ago

"The American Civil Liberties Union is suing the Trump administration for records on an executive order President Trump reportedly planned to release targeting the rights of lesbian, gay, bisexual and transgender people. In a lawsuit filed in the U.S. District Court for the Southern District of New York on Thursday, ACLU claimed the departments of Health and Human Services, Justice, Labor, and Treasury violated the Freedom of Information Act by failing to release the records it requested on the reported draft order."


Welcome to National Journal!

You are currently accessing National Journal from IP access. Please login to access this feature. If you have any questions, please contact your Dedicated Advisor.