How the Vote Ratings Are Calculated

See more stories about...
Feb. 5, 2014, 11:59 p.m.

For the past three dec­ades, Na­tion­al Journ­al has rated mem­bers of Con­gress based on se­lec­ted roll-call votes from the pre­vi­ous year to see how they com­pared with each oth­er on an ideo­lo­gic­al scale. Un­like in­terest groups that rate law­makers, Na­tion­al Journ­al does not at­tempt to say how mem­bers should have voted. Our goal is to de­scribe how they voted in com­par­is­on with one an­oth­er.

The rat­ings sys­tem was de­vised in 1981 un­der the dir­ec­tion of Bill Schneider, a polit­ic­al ana­lyst and long­time con­trib­ut­or to Na­tion­al Journ­al.

For the 2013 rat­ings, Na­tion­al Journ­al ex­amined all of the roll-call votes in the first ses­sion of the 113th Con­gress — 641 in the House and 291 in the Sen­ate — and iden­ti­fied the ones that show ideo­lo­gic­al dis­tinc­tions between mem­bers. Many votes did not make the cut — those that in­volve non­con­tro­ver­sial is­sues or that fall along re­gion­al lines, for in­stance. In the end, 117 votes in the Sen­ate and 111 votes in the House were se­lec­ted and were cat­egor­ized as eco­nom­ic, for­eign, or so­cial.

As in oth­er years re­cently, eco­nom­ic is­sues dom­in­ated the House’s at­ten­tion; there were few­er votes on so­cial is­sues (such as abor­tion rights or gun con­trol) and for­eign is­sues (such as war fund­ing and for­eign aid). The Sen­ate voted on more so­cial is­sues than the House be­cause of its con­sid­er­a­tion of im­mig­ra­tion and gun-con­trol meas­ures.

Lists were down­loaded from the House and Sen­ate web­sites show­ing how all the mem­bers voted on the se­lec­ted votes. The votes in each is­sue area were then sub­jec­ted to a prin­cip­al-com­pon­ents ana­lys­is, a stat­ist­ic­al pro­ced­ure de­signed to de­term­ine the de­gree to which each vote re­sembled oth­er votes in the same cat­egory (the same mem­bers tend­ing to vote to­geth­er).

The ana­lys­is also re­vealed which yea votes cor­rel­ated with which nay votes with­in each is­sue area (mem­bers vot­ing yea on cer­tain is­sues ten­ded to vote nay on oth­ers). The yea and nay po­s­i­tions on each roll call were then iden­ti­fied as con­ser­vat­ive or lib­er­al.

Each roll-call vote was as­signed a weight from 1 (low­est) to 3 (highest), based on the de­gree to which it cor­rel­ated with oth­er votes in the same is­sue area. A high­er weight means that a vote was more strongly cor­rel­ated with oth­er votes and was, there­fore, a bet­ter test of eco­nom­ic, so­cial, or for­eign policy ideo­logy. The votes in each is­sue area were com­bined in an in­dex (lib­er­al or con­ser­vat­ive votes as a per­cent­age of total votes cast, with each vote weighted 1, 2, or 3).

Ab­sences and ab­sten­tions were not coun­ted; in­stead, the per­cent­age base was ad­jus­ted to com­pensate for missed roll calls. A mem­ber who missed more than half of the votes in any is­sue cat­egory was scored as “miss­ing” in that cat­egory (shown as an as­ter­isk [*] in the vote-rat­ing tables).

Mem­bers were then ranked from the most lib­er­al to the most con­ser­vat­ive in each is­sue area. These rank­ings were used to as­sign lib­er­al and con­ser­vat­ive per­cent­ile rat­ings to all mem­bers of Con­gress.

The lib­er­al per­cent­ile score means that the mem­ber voted more lib­er­al than that per­cent­age of his or her col­leagues in that is­sue area in 2013. The con­ser­vat­ive fig­ure means that the mem­ber voted more con­ser­vat­ive than that per­cent­age of his or her col­leagues.

For ex­ample, a House mem­ber in the 30th per­cent­ile of lib­er­als and the 60th per­cent­ile of con­ser­vat­ives on eco­nom­ic is­sues voted more lib­er­al than 30 per­cent of the House and more con­ser­vat­ive than 60 per­cent of the House on those is­sues, and was tied with the re­main­ing 10 per­cent. The scores do not mean that the mem­ber voted lib­er­al 30 per­cent of the time and voted con­ser­vat­ive 60 per­cent of the time.

Per­cent­ile scores can range from a min­im­um of 0 to a max­im­um of 100. Some mem­bers, however, voted either con­sist­ently lib­er­al or con­sist­ently con­ser­vat­ive on every roll call. As a res­ult, there are ties at both the lib­er­al and the con­ser­vat­ive ends of each scale. For that reas­on, the max­im­um per­cent­iles are usu­ally less than 100. This was most ap­par­ent in the so­cial and for­eign cat­egor­ies in the Sen­ate last year. Many mem­bers had the same scores be­cause they voted alike. In the eco­nom­ic cat­egory, there were few­er ties.

Mem­bers also re­ceive a com­pos­ite lib­er­al score and a com­pos­ite con­ser­vat­ive score de­term­ined by their is­sue-based scores. Mem­bers who missed more than half of the votes in any of the three is­sue cat­egor­ies do not re­ceive com­pos­ite scores (shown as an as­ter­isk [*] in the vote-rat­ing tables).

To de­term­ine a mem­ber’s com­pos­ite lib­er­al score, for ex­ample, first add the lib­er­al scores in all three is­sue areas. Next, in each is­sue area, cal­cu­late 100 minus the mem­ber’s con­ser­vat­ive score and add the three res­ults to­geth­er. The two fig­ures are then com­bined and di­vided by 6 (the num­ber of in­di­vidu­al scores).

Crit­ics have some­times ac­cused Na­tion­al Journ­al of rig­ging the vote rat­ings so that cer­tain mem­bers of Con­gress are ranked as the most lib­er­al or most con­ser­vat­ive. The cri­ti­cism is un­foun­ded. When we se­lect the votes, we have no idea how an in­di­vidu­al mem­ber of Con­gress will be ranked.

Keep in mind that no single meas­ure of vot­ing be­ha­vi­or is likely to be per­fect. For in­stance, con­sider the hy­po­thet­ic­al ex­ample of a vote in the House on cut­ting do­mest­ic spend­ing. Let’s say the bill passed with over­whelm­ing sup­port from House Re­pub­lic­ans and over­whelm­ing op­pos­i­tion from House Demo­crats. A vote for the bill would be coun­ted as con­ser­vat­ive and a vote against the bill would be coun­ted as lib­er­al. But let’s say a hand­ful of House Re­pub­lic­an con­ser­vat­ives voted against the bill on the grounds that the budget cuts didn’t go far enough. In so do­ing, they voted against most con­ser­vat­ives and with most lib­er­als. Their votes would be coun­ted as lib­er­al be­cause they voted with lib­er­als. It’s bey­ond the ca­pa­city of a vote-rat­ings sys­tem to de­term­ine why a mem­ber voted the way he or she did on any par­tic­u­lar piece of le­gis­la­tion. For that reas­on, some high-pro­file votes that have con­ser­vat­ives vot­ing against a meas­ure be­cause it isn’t con­ser­vat­ive enough and lib­er­als vot­ing against the same meas­ure be­cause it isn’t lib­er­al enough are of­ten omit­ted from the vote rat­ings.

Na­tion­al Journ­al‘s an­nu­al vote rat­ings, like any oth­er vote rat­ings, should be viewed as a tool in as­sess­ing a mem­ber of Con­gress but not the only tool. Oth­er vote rat­ings should also be taken in­to con­sid­er­a­tion, as should at­trib­utes bey­ond the cap­ab­il­ity of a rat­ing sys­tem to as­sess, such as lead­er­ship and ef­fect­ive­ness.

What We're Following See More »
GOOGLE SEARCHES SPIKE
Libertarians Getting a Second Look?
5 hours ago
WHY WE CARE
VEEPSTAKES
Trump Floats Gingrich, Kasich as Running Mates
5 hours ago
THE LATEST

Newt Gringrich is actively positioning himself as a possible VP nominee for Donald Trump, according to National Review. After a New York Times piece mentioned him as a possible running mate, he said, "It is an honor to be mentioned. We need a new Contract with America to outline a 100-day plan to take back Washington from the lobbyists, bureaucrats, unions, and leftists. After helping in 1980 with Reagan and 1995 as speaker I know we have to move boldly and decisively before the election results wear off and the establishment starts fighting us. That is my focus." Meanwhile, Trump told CNN he'd be "interested in vetting" John Kasich as well.

NO MORE CUTS
House Dems Push on Puerto Rico, Citing Zika
7 hours ago
THE LATEST

"House Democrats are stepping up pressure on Republicans to advance legislation addressing Puerto Rico’s worsening debt crisis by issuing a report arguing that austerity cuts can’t be sustained and have made the island more vulnerable to the mosquito-borne Zika virus." Democrats on the House Natural Resources Committee released a report yesterday that argued "further sharp reductions in government spending can’t be a part of a legislative solution"—especially with a rainy season boosting the mosquito population and stressing an island health system already struggling to deal with the Zika virus.

Source:
USING LOCAL CELLS
Clapper: ISIS Can Stage Attacks in U.S.
7 hours ago
THE DETAILS

"ISIS has the capability to stage a Paris-style attack in the U.S. using local cells to strike in multiple locations and inflict dozens of casualties, according to the Obama administration's top U.S. intelligence official." Director of National Intelligence James Clapper told CNN's Peter Bergen that such a scenario is "something we worry about a lot in the United States, that they could conjure up a raid like they did in Paris or Brussels."

Source:
WILL ANNOUNCE PICK BEFORE CONVENTION
Trump to Name VP Search Committee
8 hours ago
THE LATEST

"Donald J. Trump said on Wednesday that he expected to reveal his vice presidential pick sometime in July—before the Republican National Convention in Cleveland—but added that he would soon announce a committee to handle the selection process, which would include Dr. Ben Carson." He said he's inclined to name a traditional political figure, unlike himself.

Source:
×