How the Vote Ratings Are Calculated

None

See more stories about...
Feb. 5, 2014, 11:59 p.m.

For the past three dec­ades, Na­tion­al Journ­al has rated mem­bers of Con­gress based on se­lec­ted roll-call votes from the pre­vi­ous year to see how they com­pared with each oth­er on an ideo­lo­gic­al scale. Un­like in­terest groups that rate law­makers, Na­tion­al Journ­al does not at­tempt to say how mem­bers should have voted. Our goal is to de­scribe how they voted in com­par­is­on with one an­oth­er.

The rat­ings sys­tem was de­vised in 1981 un­der the dir­ec­tion of Bill Schneider, a polit­ic­al ana­lyst and long­time con­trib­ut­or to Na­tion­al Journ­al.

For the 2013 rat­ings, Na­tion­al Journ­al ex­amined all of the roll-call votes in the first ses­sion of the 113th Con­gress — 641 in the House and 291 in the Sen­ate — and iden­ti­fied the ones that show ideo­lo­gic­al dis­tinc­tions between mem­bers. Many votes did not make the cut — those that in­volve non­con­tro­ver­sial is­sues or that fall along re­gion­al lines, for in­stance. In the end, 117 votes in the Sen­ate and 111 votes in the House were se­lec­ted and were cat­egor­ized as eco­nom­ic, for­eign, or so­cial.

As in oth­er years re­cently, eco­nom­ic is­sues dom­in­ated the House’s at­ten­tion; there were few­er votes on so­cial is­sues (such as abor­tion rights or gun con­trol) and for­eign is­sues (such as war fund­ing and for­eign aid). The Sen­ate voted on more so­cial is­sues than the House be­cause of its con­sid­er­a­tion of im­mig­ra­tion and gun-con­trol meas­ures.

Lists were down­loaded from the House and Sen­ate web­sites show­ing how all the mem­bers voted on the se­lec­ted votes. The votes in each is­sue area were then sub­jec­ted to a prin­cip­al-com­pon­ents ana­lys­is, a stat­ist­ic­al pro­ced­ure de­signed to de­term­ine the de­gree to which each vote re­sembled oth­er votes in the same cat­egory (the same mem­bers tend­ing to vote to­geth­er).

The ana­lys­is also re­vealed which yea votes cor­rel­ated with which nay votes with­in each is­sue area (mem­bers vot­ing yea on cer­tain is­sues ten­ded to vote nay on oth­ers). The yea and nay po­s­i­tions on each roll call were then iden­ti­fied as con­ser­vat­ive or lib­er­al.

Each roll-call vote was as­signed a weight from 1 (low­est) to 3 (highest), based on the de­gree to which it cor­rel­ated with oth­er votes in the same is­sue area. A high­er weight means that a vote was more strongly cor­rel­ated with oth­er votes and was, there­fore, a bet­ter test of eco­nom­ic, so­cial, or for­eign policy ideo­logy. The votes in each is­sue area were com­bined in an in­dex (lib­er­al or con­ser­vat­ive votes as a per­cent­age of total votes cast, with each vote weighted 1, 2, or 3).

Ab­sences and ab­sten­tions were not coun­ted; in­stead, the per­cent­age base was ad­jus­ted to com­pensate for missed roll calls. A mem­ber who missed more than half of the votes in any is­sue cat­egory was scored as “miss­ing” in that cat­egory (shown as an as­ter­isk [*] in the vote-rat­ing tables).

Mem­bers were then ranked from the most lib­er­al to the most con­ser­vat­ive in each is­sue area. These rank­ings were used to as­sign lib­er­al and con­ser­vat­ive per­cent­ile rat­ings to all mem­bers of Con­gress.

The lib­er­al per­cent­ile score means that the mem­ber voted more lib­er­al than that per­cent­age of his or her col­leagues in that is­sue area in 2013. The con­ser­vat­ive fig­ure means that the mem­ber voted more con­ser­vat­ive than that per­cent­age of his or her col­leagues.

For ex­ample, a House mem­ber in the 30th per­cent­ile of lib­er­als and the 60th per­cent­ile of con­ser­vat­ives on eco­nom­ic is­sues voted more lib­er­al than 30 per­cent of the House and more con­ser­vat­ive than 60 per­cent of the House on those is­sues, and was tied with the re­main­ing 10 per­cent. The scores do not mean that the mem­ber voted lib­er­al 30 per­cent of the time and voted con­ser­vat­ive 60 per­cent of the time.

Per­cent­ile scores can range from a min­im­um of 0 to a max­im­um of 100. Some mem­bers, however, voted either con­sist­ently lib­er­al or con­sist­ently con­ser­vat­ive on every roll call. As a res­ult, there are ties at both the lib­er­al and the con­ser­vat­ive ends of each scale. For that reas­on, the max­im­um per­cent­iles are usu­ally less than 100. This was most ap­par­ent in the so­cial and for­eign cat­egor­ies in the Sen­ate last year. Many mem­bers had the same scores be­cause they voted alike. In the eco­nom­ic cat­egory, there were few­er ties.

Mem­bers also re­ceive a com­pos­ite lib­er­al score and a com­pos­ite con­ser­vat­ive score de­term­ined by their is­sue-based scores. Mem­bers who missed more than half of the votes in any of the three is­sue cat­egor­ies do not re­ceive com­pos­ite scores (shown as an as­ter­isk [*] in the vote-rat­ing tables).

To de­term­ine a mem­ber’s com­pos­ite lib­er­al score, for ex­ample, first add the lib­er­al scores in all three is­sue areas. Next, in each is­sue area, cal­cu­late 100 minus the mem­ber’s con­ser­vat­ive score and add the three res­ults to­geth­er. The two fig­ures are then com­bined and di­vided by 6 (the num­ber of in­di­vidu­al scores).

Crit­ics have some­times ac­cused Na­tion­al Journ­al of rig­ging the vote rat­ings so that cer­tain mem­bers of Con­gress are ranked as the most lib­er­al or most con­ser­vat­ive. The cri­ti­cism is un­foun­ded. When we se­lect the votes, we have no idea how an in­di­vidu­al mem­ber of Con­gress will be ranked.

Keep in mind that no single meas­ure of vot­ing be­ha­vi­or is likely to be per­fect. For in­stance, con­sider the hy­po­thet­ic­al ex­ample of a vote in the House on cut­ting do­mest­ic spend­ing. Let’s say the bill passed with over­whelm­ing sup­port from House Re­pub­lic­ans and over­whelm­ing op­pos­i­tion from House Demo­crats. A vote for the bill would be coun­ted as con­ser­vat­ive and a vote against the bill would be coun­ted as lib­er­al. But let’s say a hand­ful of House Re­pub­lic­an con­ser­vat­ives voted against the bill on the grounds that the budget cuts didn’t go far enough. In so do­ing, they voted against most con­ser­vat­ives and with most lib­er­als. Their votes would be coun­ted as lib­er­al be­cause they voted with lib­er­als. It’s bey­ond the ca­pa­city of a vote-rat­ings sys­tem to de­term­ine why a mem­ber voted the way he or she did on any par­tic­u­lar piece of le­gis­la­tion. For that reas­on, some high-pro­file votes that have con­ser­vat­ives vot­ing against a meas­ure be­cause it isn’t con­ser­vat­ive enough and lib­er­als vot­ing against the same meas­ure be­cause it isn’t lib­er­al enough are of­ten omit­ted from the vote rat­ings.

Na­tion­al Journ­al‘s an­nu­al vote rat­ings, like any oth­er vote rat­ings, should be viewed as a tool in as­sess­ing a mem­ber of Con­gress but not the only tool. Oth­er vote rat­ings should also be taken in­to con­sid­er­a­tion, as should at­trib­utes bey­ond the cap­ab­il­ity of a rat­ing sys­tem to as­sess, such as lead­er­ship and ef­fect­ive­ness.

What We're Following See More »
‘PULLING A TRUMP’
GOP Budget Chiefs Won’t Invite Administration to Testify
23 hours ago
THE DETAILS

The administration will release its 2017 budget blueprint tomorrow, but the House and Senate budget committees won’t be inviting anyone from the White House to come talk about it. “The chairmen of the House and Senate Budget committees released a joint statement saying it simply wasn’t worth their time” to hear from OMB Director Shaun Donovan. Accusing the members of pulling a “Donald Trump,” White House spokesman Josh Earnest said the move “raises some questions about how confident they are about the kinds of arguments that they could make.”

Source:
A DARK CLOUD OVER TRUMP?
Snowstorm Could Impact Primary Turnout
19 hours ago
THE LATEST

A snowstorm is supposed to hit New Hampshire today and “linger into Primary Tuesday.” GOP consultant Ron Kaufman said lower turnout should help candidates who have spent a lot of time in the state tending to retail politicking. Donald Trump “has acknowledged that he needs to step up his ground-game, and a heavy snowfall could depress his figures relative to more organized candidates.”

Source:
IN CASE OF EMERGENCY
A Shake-Up in the Offing in the Clinton Camp?
14 hours ago
THE DETAILS

Anticipating a primary loss in New Hampshire on Tuesday, Hillary and Bill Clinton “are considering staffing and strategy changes” to their campaign. Sources tell Politico that the Clintons are likely to layer over top officials with experienced talent, rather than fire their staff en masse.

Source:
THE LAST ROUND OF NEW HAMPSHIRE POLLS
Trump Is Still Ahead, but Who’s in Second?
1 hours ago
THE LATEST

We may not be talking about New Hampshire primary polls for another three-and-a-half years, so here goes:

  • American Research Group’s tracking poll has Donald Trump in the lead with 30% support, followed by Marco Rubio and John Kasich tying for second place at 16%. On the Democratic side, Bernie Sanders leads Hillary Clinton 53%-41%.
  • The 7 News/UMass Lowell tracking poll has Trump way out front with 34%, followed by Rubio and Ted Cruz with 13% apiece. Among the Democrats, Sanders is in front 56%-40%.
  • A Gravis poll puts Trump ahead with 28%, followed by Kasich with 17% and Rubio with 15%.
×