How the Vote Ratings Are Calculated

Add to Briefcase
See more stories about...
Feb. 5, 2014, 11:59 p.m.

For the past three dec­ades, Na­tion­al Journ­al has rated mem­bers of Con­gress based on se­lec­ted roll-call votes from the pre­vi­ous year to see how they com­pared with each oth­er on an ideo­lo­gic­al scale. Un­like in­terest groups that rate law­makers, Na­tion­al Journ­al does not at­tempt to say how mem­bers should have voted. Our goal is to de­scribe how they voted in com­par­is­on with one an­oth­er.

The rat­ings sys­tem was de­vised in 1981 un­der the dir­ec­tion of Bill Schneider, a polit­ic­al ana­lyst and long­time con­trib­ut­or to Na­tion­al Journ­al.

For the 2013 rat­ings, Na­tion­al Journ­al ex­amined all of the roll-call votes in the first ses­sion of the 113th Con­gress — 641 in the House and 291 in the Sen­ate — and iden­ti­fied the ones that show ideo­lo­gic­al dis­tinc­tions between mem­bers. Many votes did not make the cut — those that in­volve non­con­tro­ver­sial is­sues or that fall along re­gion­al lines, for in­stance. In the end, 117 votes in the Sen­ate and 111 votes in the House were se­lec­ted and were cat­egor­ized as eco­nom­ic, for­eign, or so­cial.

As in oth­er years re­cently, eco­nom­ic is­sues dom­in­ated the House’s at­ten­tion; there were few­er votes on so­cial is­sues (such as abor­tion rights or gun con­trol) and for­eign is­sues (such as war fund­ing and for­eign aid). The Sen­ate voted on more so­cial is­sues than the House be­cause of its con­sid­er­a­tion of im­mig­ra­tion and gun-con­trol meas­ures.

Lists were down­loaded from the House and Sen­ate web­sites show­ing how all the mem­bers voted on the se­lec­ted votes. The votes in each is­sue area were then sub­jec­ted to a prin­cip­al-com­pon­ents ana­lys­is, a stat­ist­ic­al pro­ced­ure de­signed to de­term­ine the de­gree to which each vote re­sembled oth­er votes in the same cat­egory (the same mem­bers tend­ing to vote to­geth­er).

The ana­lys­is also re­vealed which yea votes cor­rel­ated with which nay votes with­in each is­sue area (mem­bers vot­ing yea on cer­tain is­sues ten­ded to vote nay on oth­ers). The yea and nay po­s­i­tions on each roll call were then iden­ti­fied as con­ser­vat­ive or lib­er­al.

Each roll-call vote was as­signed a weight from 1 (low­est) to 3 (highest), based on the de­gree to which it cor­rel­ated with oth­er votes in the same is­sue area. A high­er weight means that a vote was more strongly cor­rel­ated with oth­er votes and was, there­fore, a bet­ter test of eco­nom­ic, so­cial, or for­eign policy ideo­logy. The votes in each is­sue area were com­bined in an in­dex (lib­er­al or con­ser­vat­ive votes as a per­cent­age of total votes cast, with each vote weighted 1, 2, or 3).

Ab­sences and ab­sten­tions were not coun­ted; in­stead, the per­cent­age base was ad­jus­ted to com­pensate for missed roll calls. A mem­ber who missed more than half of the votes in any is­sue cat­egory was scored as “miss­ing” in that cat­egory (shown as an as­ter­isk [*] in the vote-rat­ing tables).

Mem­bers were then ranked from the most lib­er­al to the most con­ser­vat­ive in each is­sue area. These rank­ings were used to as­sign lib­er­al and con­ser­vat­ive per­cent­ile rat­ings to all mem­bers of Con­gress.

The lib­er­al per­cent­ile score means that the mem­ber voted more lib­er­al than that per­cent­age of his or her col­leagues in that is­sue area in 2013. The con­ser­vat­ive fig­ure means that the mem­ber voted more con­ser­vat­ive than that per­cent­age of his or her col­leagues.

For ex­ample, a House mem­ber in the 30th per­cent­ile of lib­er­als and the 60th per­cent­ile of con­ser­vat­ives on eco­nom­ic is­sues voted more lib­er­al than 30 per­cent of the House and more con­ser­vat­ive than 60 per­cent of the House on those is­sues, and was tied with the re­main­ing 10 per­cent. The scores do not mean that the mem­ber voted lib­er­al 30 per­cent of the time and voted con­ser­vat­ive 60 per­cent of the time.

Per­cent­ile scores can range from a min­im­um of 0 to a max­im­um of 100. Some mem­bers, however, voted either con­sist­ently lib­er­al or con­sist­ently con­ser­vat­ive on every roll call. As a res­ult, there are ties at both the lib­er­al and the con­ser­vat­ive ends of each scale. For that reas­on, the max­im­um per­cent­iles are usu­ally less than 100. This was most ap­par­ent in the so­cial and for­eign cat­egor­ies in the Sen­ate last year. Many mem­bers had the same scores be­cause they voted alike. In the eco­nom­ic cat­egory, there were few­er ties.

Mem­bers also re­ceive a com­pos­ite lib­er­al score and a com­pos­ite con­ser­vat­ive score de­term­ined by their is­sue-based scores. Mem­bers who missed more than half of the votes in any of the three is­sue cat­egor­ies do not re­ceive com­pos­ite scores (shown as an as­ter­isk [*] in the vote-rat­ing tables).

To de­term­ine a mem­ber’s com­pos­ite lib­er­al score, for ex­ample, first add the lib­er­al scores in all three is­sue areas. Next, in each is­sue area, cal­cu­late 100 minus the mem­ber’s con­ser­vat­ive score and add the three res­ults to­geth­er. The two fig­ures are then com­bined and di­vided by 6 (the num­ber of in­di­vidu­al scores).

Crit­ics have some­times ac­cused Na­tion­al Journ­al of rig­ging the vote rat­ings so that cer­tain mem­bers of Con­gress are ranked as the most lib­er­al or most con­ser­vat­ive. The cri­ti­cism is un­foun­ded. When we se­lect the votes, we have no idea how an in­di­vidu­al mem­ber of Con­gress will be ranked.

Keep in mind that no single meas­ure of vot­ing be­ha­vi­or is likely to be per­fect. For in­stance, con­sider the hy­po­thet­ic­al ex­ample of a vote in the House on cut­ting do­mest­ic spend­ing. Let’s say the bill passed with over­whelm­ing sup­port from House Re­pub­lic­ans and over­whelm­ing op­pos­i­tion from House Demo­crats. A vote for the bill would be coun­ted as con­ser­vat­ive and a vote against the bill would be coun­ted as lib­er­al. But let’s say a hand­ful of House Re­pub­lic­an con­ser­vat­ives voted against the bill on the grounds that the budget cuts didn’t go far enough. In so do­ing, they voted against most con­ser­vat­ives and with most lib­er­als. Their votes would be coun­ted as lib­er­al be­cause they voted with lib­er­als. It’s bey­ond the ca­pa­city of a vote-rat­ings sys­tem to de­term­ine why a mem­ber voted the way he or she did on any par­tic­u­lar piece of le­gis­la­tion. For that reas­on, some high-pro­file votes that have con­ser­vat­ives vot­ing against a meas­ure be­cause it isn’t con­ser­vat­ive enough and lib­er­als vot­ing against the same meas­ure be­cause it isn’t lib­er­al enough are of­ten omit­ted from the vote rat­ings.

Na­tion­al Journ­al‘s an­nu­al vote rat­ings, like any oth­er vote rat­ings, should be viewed as a tool in as­sess­ing a mem­ber of Con­gress but not the only tool. Oth­er vote rat­ings should also be taken in­to con­sid­er­a­tion, as should at­trib­utes bey­ond the cap­ab­il­ity of a rat­ing sys­tem to as­sess, such as lead­er­ship and ef­fect­ive­ness.

What We're Following See More »
House to Vote on Iran Sanctions Renewal in Lame Duck
2 minutes ago

Republican House leaders are planning on taking up a vote to renew the Iran Sanctions Act as soon as the lame-duck session begins in mid-November. The law, which expires on Dec. 31, permits a host of sanctions against Iran's industries, defense, and government. The renewal will likely pass the House, but its status is unclear once it reaches the Senate, and a spokesman from the White House refused to say whether President Obama would sign it into law.

Trump Stops Holding Fundraisers to Benefit GOP
7 minutes ago

Just two weeks from Nov. 8, Donald Trump's campaign is not scheduling anymore high-dollar fundraisers, the type which usually benefit the Republican Party as a whole. The move comes as a surprise and could be a big blow to the GOP's turnout operations. Many down-ballot candidates are relying on the party apparatus to turn out voters in their districts and/or states, something that could be compromised. The last formal fundraiser occurred on Wednesday, Oct. 19.

Republican Polling Shows Close Race
Roundup: National Polling Remains Inconsistent
16 hours ago

The national polls, once again, tell very different stories: Clinton leads by just one point in the IBD, Rasmussen, and LA Times tracking polls, while she shows a commanding 12 point lead in the ABC news poll and a smaller but sizable five point lead in the CNN poll. The Republican Remington Research Group released a slew of polls showing Trump up in Ohio, Nevada, and North Carolina, a tie in Florida, and Clinton leads in Pennsylvania, Wisconsin, and Virginia. However, an independent Siena poll shows Clinton up 7 in North Carolina, while a Monmouth poll shows Trump up one in Arizona

Colin Powell to Vote for Clinton
18 hours ago
Clinton Reaching Out to GOP Senators
23 hours ago

If you need a marker for how confident Hillary Clinton is at this point of the race, here's one: CNN's Jeff Zeleny reports "she's been talking to Republican senators, old allies and new, saying that she is willing to work with them and govern."


Welcome to National Journal!

You are currently accessing National Journal from IP access. Please login to access this feature. If you have any questions, please contact your Dedicated Advisor.