How the Vote Ratings Are Calculated

None

See more stories about...
Feb. 5, 2014, 11:59 p.m.

For the past three dec­ades, Na­tion­al Journ­al has rated mem­bers of Con­gress based on se­lec­ted roll-call votes from the pre­vi­ous year to see how they com­pared with each oth­er on an ideo­lo­gic­al scale. Un­like in­terest groups that rate law­makers, Na­tion­al Journ­al does not at­tempt to say how mem­bers should have voted. Our goal is to de­scribe how they voted in com­par­is­on with one an­oth­er.

The rat­ings sys­tem was de­vised in 1981 un­der the dir­ec­tion of Bill Schneider, a polit­ic­al ana­lyst and long­time con­trib­ut­or to Na­tion­al Journ­al.

For the 2013 rat­ings, Na­tion­al Journ­al ex­amined all of the roll-call votes in the first ses­sion of the 113th Con­gress — 641 in the House and 291 in the Sen­ate — and iden­ti­fied the ones that show ideo­lo­gic­al dis­tinc­tions between mem­bers. Many votes did not make the cut — those that in­volve non­con­tro­ver­sial is­sues or that fall along re­gion­al lines, for in­stance. In the end, 117 votes in the Sen­ate and 111 votes in the House were se­lec­ted and were cat­egor­ized as eco­nom­ic, for­eign, or so­cial.

As in oth­er years re­cently, eco­nom­ic is­sues dom­in­ated the House’s at­ten­tion; there were few­er votes on so­cial is­sues (such as abor­tion rights or gun con­trol) and for­eign is­sues (such as war fund­ing and for­eign aid). The Sen­ate voted on more so­cial is­sues than the House be­cause of its con­sid­er­a­tion of im­mig­ra­tion and gun-con­trol meas­ures.

Lists were down­loaded from the House and Sen­ate web­sites show­ing how all the mem­bers voted on the se­lec­ted votes. The votes in each is­sue area were then sub­jec­ted to a prin­cip­al-com­pon­ents ana­lys­is, a stat­ist­ic­al pro­ced­ure de­signed to de­term­ine the de­gree to which each vote re­sembled oth­er votes in the same cat­egory (the same mem­bers tend­ing to vote to­geth­er).

The ana­lys­is also re­vealed which yea votes cor­rel­ated with which nay votes with­in each is­sue area (mem­bers vot­ing yea on cer­tain is­sues ten­ded to vote nay on oth­ers). The yea and nay po­s­i­tions on each roll call were then iden­ti­fied as con­ser­vat­ive or lib­er­al.

Each roll-call vote was as­signed a weight from 1 (low­est) to 3 (highest), based on the de­gree to which it cor­rel­ated with oth­er votes in the same is­sue area. A high­er weight means that a vote was more strongly cor­rel­ated with oth­er votes and was, there­fore, a bet­ter test of eco­nom­ic, so­cial, or for­eign policy ideo­logy. The votes in each is­sue area were com­bined in an in­dex (lib­er­al or con­ser­vat­ive votes as a per­cent­age of total votes cast, with each vote weighted 1, 2, or 3).

Ab­sences and ab­sten­tions were not coun­ted; in­stead, the per­cent­age base was ad­jus­ted to com­pensate for missed roll calls. A mem­ber who missed more than half of the votes in any is­sue cat­egory was scored as “miss­ing” in that cat­egory (shown as an as­ter­isk [*] in the vote-rat­ing tables).

Mem­bers were then ranked from the most lib­er­al to the most con­ser­vat­ive in each is­sue area. These rank­ings were used to as­sign lib­er­al and con­ser­vat­ive per­cent­ile rat­ings to all mem­bers of Con­gress.

The lib­er­al per­cent­ile score means that the mem­ber voted more lib­er­al than that per­cent­age of his or her col­leagues in that is­sue area in 2013. The con­ser­vat­ive fig­ure means that the mem­ber voted more con­ser­vat­ive than that per­cent­age of his or her col­leagues.

For ex­ample, a House mem­ber in the 30th per­cent­ile of lib­er­als and the 60th per­cent­ile of con­ser­vat­ives on eco­nom­ic is­sues voted more lib­er­al than 30 per­cent of the House and more con­ser­vat­ive than 60 per­cent of the House on those is­sues, and was tied with the re­main­ing 10 per­cent. The scores do not mean that the mem­ber voted lib­er­al 30 per­cent of the time and voted con­ser­vat­ive 60 per­cent of the time.

Per­cent­ile scores can range from a min­im­um of 0 to a max­im­um of 100. Some mem­bers, however, voted either con­sist­ently lib­er­al or con­sist­ently con­ser­vat­ive on every roll call. As a res­ult, there are ties at both the lib­er­al and the con­ser­vat­ive ends of each scale. For that reas­on, the max­im­um per­cent­iles are usu­ally less than 100. This was most ap­par­ent in the so­cial and for­eign cat­egor­ies in the Sen­ate last year. Many mem­bers had the same scores be­cause they voted alike. In the eco­nom­ic cat­egory, there were few­er ties.

Mem­bers also re­ceive a com­pos­ite lib­er­al score and a com­pos­ite con­ser­vat­ive score de­term­ined by their is­sue-based scores. Mem­bers who missed more than half of the votes in any of the three is­sue cat­egor­ies do not re­ceive com­pos­ite scores (shown as an as­ter­isk [*] in the vote-rat­ing tables).

To de­term­ine a mem­ber’s com­pos­ite lib­er­al score, for ex­ample, first add the lib­er­al scores in all three is­sue areas. Next, in each is­sue area, cal­cu­late 100 minus the mem­ber’s con­ser­vat­ive score and add the three res­ults to­geth­er. The two fig­ures are then com­bined and di­vided by 6 (the num­ber of in­di­vidu­al scores).

Crit­ics have some­times ac­cused Na­tion­al Journ­al of rig­ging the vote rat­ings so that cer­tain mem­bers of Con­gress are ranked as the most lib­er­al or most con­ser­vat­ive. The cri­ti­cism is un­foun­ded. When we se­lect the votes, we have no idea how an in­di­vidu­al mem­ber of Con­gress will be ranked.

Keep in mind that no single meas­ure of vot­ing be­ha­vi­or is likely to be per­fect. For in­stance, con­sider the hy­po­thet­ic­al ex­ample of a vote in the House on cut­ting do­mest­ic spend­ing. Let’s say the bill passed with over­whelm­ing sup­port from House Re­pub­lic­ans and over­whelm­ing op­pos­i­tion from House Demo­crats. A vote for the bill would be coun­ted as con­ser­vat­ive and a vote against the bill would be coun­ted as lib­er­al. But let’s say a hand­ful of House Re­pub­lic­an con­ser­vat­ives voted against the bill on the grounds that the budget cuts didn’t go far enough. In so do­ing, they voted against most con­ser­vat­ives and with most lib­er­als. Their votes would be coun­ted as lib­er­al be­cause they voted with lib­er­als. It’s bey­ond the ca­pa­city of a vote-rat­ings sys­tem to de­term­ine why a mem­ber voted the way he or she did on any par­tic­u­lar piece of le­gis­la­tion. For that reas­on, some high-pro­file votes that have con­ser­vat­ives vot­ing against a meas­ure be­cause it isn’t con­ser­vat­ive enough and lib­er­als vot­ing against the same meas­ure be­cause it isn’t lib­er­al enough are of­ten omit­ted from the vote rat­ings.

Na­tion­al Journ­al‘s an­nu­al vote rat­ings, like any oth­er vote rat­ings, should be viewed as a tool in as­sess­ing a mem­ber of Con­gress but not the only tool. Oth­er vote rat­ings should also be taken in­to con­sid­er­a­tion, as should at­trib­utes bey­ond the cap­ab­il­ity of a rat­ing sys­tem to as­sess, such as lead­er­ship and ef­fect­ive­ness.

What We're Following See More »
TAKING A LONG VIEW TO SOUTHERN STATES
In Dropout Speech, Santorum Endorses Rubio
3 days ago
THE DETAILS

As expected after earlier reports on Wednesday, Rick Santorum ended his presidential bid. But less expected: he threw his support to Marco Rubio. After noting he spoke with Rubio the day before for an hour, he said, “Someone who has a real understanding of the threat of ISIS, real understanding of the threat of fundamentalist Islam, and has experience, one of the things I wanted was someone who has experience in this area, and that’s why we decided to support Marco Rubio.” It doesn’t figure to help Rubio much in New Hampshire, but the Santorum nod could pay dividends down the road in southern states.

Source:
‘PITTING PEOPLE AGAINST EACH OTHER’
Rubio, Trump Question Obama’s Mosque Visit
3 days ago
WHY WE CARE

President Obama’s decision to visit a mosque in Baltimore today was never going to be completely uncontroversial. And Donald Trump and Marco Rubio proved it. “Maybe he feels comfortable there,” Trump told interviewer Greta van Susteren on Fox News. “There are a lot of places he can go, and he chose a mosque.” And in New Hampshire, Rubio said of Obama, “Always pitting people against each other. Always. Look at today – he gave a speech at a mosque. Oh, you know, basically implying that America is discriminating against Muslims.”

Source:
THE TIME IS NOW, TED
Cruz Must Max Out on Evangelical Support through Early March
3 days ago
WHY WE CARE

For Ted Cruz, a strong showing in New Hampshire would be nice, but not necessary. That’s because evangelical voters only make up 21% of the Granite State’s population. “But from the February 20 South Carolina primary through March 15, there are nine states (South Carolina, Alabama, Arkansas, Georgia, Oklahoma, Tennessee, Kentucky, Mississippi, and North Carolina) with an estimated white-Evangelical percentage of the GOP electorate over 60 percent, and another four (Texas, Kansas, Louisiana, and Missouri) that come in over 50 percent.” But after that, he better be in the catbird’s seat, because only four smaller states remain with evangelical voter majorities.

Source:
CHRISTIE, BUSH TRYING TO TAKE HIM DOWN
Rubio Now Winning the ‘Endorsement Primary’
2 days ago
WHY WE CARE

Since his strong third-place finish in Iowa, Marco Rubio has won endorsement by two sitting senators and two congressmen, putting him in the lead for the first time of FiveThirtyEight‘s Endorsement Tracker. “Some politicians had put early support behind Jeb Bush — he had led [their] list since August — but since January the only new endorsement he has received was from former presidential candidate Sen. Lindsey Graham.” Meanwhile, the New York Times reports that fueled by resentment, “members of the Bush and Christie campaigns have communicated about their mutual desire to halt … Rubio’s rise in the polls.”

Source:
7 REPUBLICANS ON STAGE
Carly Fiorina Will Not Be Allowed to Debate on Saturday
2 days ago
THE LATEST

ABC News has announced the criteria for Saturday’s Republican debate, and that means Carly Fiorina won’t be a part of it. The network is demanding candidates have “a top-three finish in Iowa, a top-six standing in an average of recent New Hampshire polls or a top-six placement in national polls in order for candidates to qualify.” And there will be no “happy hour” undercard debate this time. “So that means no Fiorina vs. Jim Gilmore showdown earlier in the evening for the most ardent of campaign 2016 junkies.

Source:
×