How the Vote Ratings Are Calculated

Add to Briefcase
See more stories about...
Feb. 5, 2014, 11:59 p.m.

For the past three dec­ades, Na­tion­al Journ­al has rated mem­bers of Con­gress based on se­lec­ted roll-call votes from the pre­vi­ous year to see how they com­pared with each oth­er on an ideo­lo­gic­al scale. Un­like in­terest groups that rate law­makers, Na­tion­al Journ­al does not at­tempt to say how mem­bers should have voted. Our goal is to de­scribe how they voted in com­par­is­on with one an­oth­er.

The rat­ings sys­tem was de­vised in 1981 un­der the dir­ec­tion of Bill Schneider, a polit­ic­al ana­lyst and long­time con­trib­ut­or to Na­tion­al Journ­al.

For the 2013 rat­ings, Na­tion­al Journ­al ex­amined all of the roll-call votes in the first ses­sion of the 113th Con­gress — 641 in the House and 291 in the Sen­ate — and iden­ti­fied the ones that show ideo­lo­gic­al dis­tinc­tions between mem­bers. Many votes did not make the cut — those that in­volve non­con­tro­ver­sial is­sues or that fall along re­gion­al lines, for in­stance. In the end, 117 votes in the Sen­ate and 111 votes in the House were se­lec­ted and were cat­egor­ized as eco­nom­ic, for­eign, or so­cial.

As in oth­er years re­cently, eco­nom­ic is­sues dom­in­ated the House’s at­ten­tion; there were few­er votes on so­cial is­sues (such as abor­tion rights or gun con­trol) and for­eign is­sues (such as war fund­ing and for­eign aid). The Sen­ate voted on more so­cial is­sues than the House be­cause of its con­sid­er­a­tion of im­mig­ra­tion and gun-con­trol meas­ures.

Lists were down­loaded from the House and Sen­ate web­sites show­ing how all the mem­bers voted on the se­lec­ted votes. The votes in each is­sue area were then sub­jec­ted to a prin­cip­al-com­pon­ents ana­lys­is, a stat­ist­ic­al pro­ced­ure de­signed to de­term­ine the de­gree to which each vote re­sembled oth­er votes in the same cat­egory (the same mem­bers tend­ing to vote to­geth­er).

The ana­lys­is also re­vealed which yea votes cor­rel­ated with which nay votes with­in each is­sue area (mem­bers vot­ing yea on cer­tain is­sues ten­ded to vote nay on oth­ers). The yea and nay po­s­i­tions on each roll call were then iden­ti­fied as con­ser­vat­ive or lib­er­al.

Each roll-call vote was as­signed a weight from 1 (low­est) to 3 (highest), based on the de­gree to which it cor­rel­ated with oth­er votes in the same is­sue area. A high­er weight means that a vote was more strongly cor­rel­ated with oth­er votes and was, there­fore, a bet­ter test of eco­nom­ic, so­cial, or for­eign policy ideo­logy. The votes in each is­sue area were com­bined in an in­dex (lib­er­al or con­ser­vat­ive votes as a per­cent­age of total votes cast, with each vote weighted 1, 2, or 3).

Ab­sences and ab­sten­tions were not coun­ted; in­stead, the per­cent­age base was ad­jus­ted to com­pensate for missed roll calls. A mem­ber who missed more than half of the votes in any is­sue cat­egory was scored as “miss­ing” in that cat­egory (shown as an as­ter­isk [*] in the vote-rat­ing tables).

Mem­bers were then ranked from the most lib­er­al to the most con­ser­vat­ive in each is­sue area. These rank­ings were used to as­sign lib­er­al and con­ser­vat­ive per­cent­ile rat­ings to all mem­bers of Con­gress.

The lib­er­al per­cent­ile score means that the mem­ber voted more lib­er­al than that per­cent­age of his or her col­leagues in that is­sue area in 2013. The con­ser­vat­ive fig­ure means that the mem­ber voted more con­ser­vat­ive than that per­cent­age of his or her col­leagues.

For ex­ample, a House mem­ber in the 30th per­cent­ile of lib­er­als and the 60th per­cent­ile of con­ser­vat­ives on eco­nom­ic is­sues voted more lib­er­al than 30 per­cent of the House and more con­ser­vat­ive than 60 per­cent of the House on those is­sues, and was tied with the re­main­ing 10 per­cent. The scores do not mean that the mem­ber voted lib­er­al 30 per­cent of the time and voted con­ser­vat­ive 60 per­cent of the time.

Per­cent­ile scores can range from a min­im­um of 0 to a max­im­um of 100. Some mem­bers, however, voted either con­sist­ently lib­er­al or con­sist­ently con­ser­vat­ive on every roll call. As a res­ult, there are ties at both the lib­er­al and the con­ser­vat­ive ends of each scale. For that reas­on, the max­im­um per­cent­iles are usu­ally less than 100. This was most ap­par­ent in the so­cial and for­eign cat­egor­ies in the Sen­ate last year. Many mem­bers had the same scores be­cause they voted alike. In the eco­nom­ic cat­egory, there were few­er ties.

Mem­bers also re­ceive a com­pos­ite lib­er­al score and a com­pos­ite con­ser­vat­ive score de­term­ined by their is­sue-based scores. Mem­bers who missed more than half of the votes in any of the three is­sue cat­egor­ies do not re­ceive com­pos­ite scores (shown as an as­ter­isk [*] in the vote-rat­ing tables).

To de­term­ine a mem­ber’s com­pos­ite lib­er­al score, for ex­ample, first add the lib­er­al scores in all three is­sue areas. Next, in each is­sue area, cal­cu­late 100 minus the mem­ber’s con­ser­vat­ive score and add the three res­ults to­geth­er. The two fig­ures are then com­bined and di­vided by 6 (the num­ber of in­di­vidu­al scores).

Crit­ics have some­times ac­cused Na­tion­al Journ­al of rig­ging the vote rat­ings so that cer­tain mem­bers of Con­gress are ranked as the most lib­er­al or most con­ser­vat­ive. The cri­ti­cism is un­foun­ded. When we se­lect the votes, we have no idea how an in­di­vidu­al mem­ber of Con­gress will be ranked.

Keep in mind that no single meas­ure of vot­ing be­ha­vi­or is likely to be per­fect. For in­stance, con­sider the hy­po­thet­ic­al ex­ample of a vote in the House on cut­ting do­mest­ic spend­ing. Let’s say the bill passed with over­whelm­ing sup­port from House Re­pub­lic­ans and over­whelm­ing op­pos­i­tion from House Demo­crats. A vote for the bill would be coun­ted as con­ser­vat­ive and a vote against the bill would be coun­ted as lib­er­al. But let’s say a hand­ful of House Re­pub­lic­an con­ser­vat­ives voted against the bill on the grounds that the budget cuts didn’t go far enough. In so do­ing, they voted against most con­ser­vat­ives and with most lib­er­als. Their votes would be coun­ted as lib­er­al be­cause they voted with lib­er­als. It’s bey­ond the ca­pa­city of a vote-rat­ings sys­tem to de­term­ine why a mem­ber voted the way he or she did on any par­tic­u­lar piece of le­gis­la­tion. For that reas­on, some high-pro­file votes that have con­ser­vat­ives vot­ing against a meas­ure be­cause it isn’t con­ser­vat­ive enough and lib­er­als vot­ing against the same meas­ure be­cause it isn’t lib­er­al enough are of­ten omit­ted from the vote rat­ings.

Na­tion­al Journ­al‘s an­nu­al vote rat­ings, like any oth­er vote rat­ings, should be viewed as a tool in as­sess­ing a mem­ber of Con­gress but not the only tool. Oth­er vote rat­ings should also be taken in­to con­sid­er­a­tion, as should at­trib­utes bey­ond the cap­ab­il­ity of a rat­ing sys­tem to as­sess, such as lead­er­ship and ef­fect­ive­ness.

What We're Following See More »
A CANDIDATE TO BE ‘PROUD’ OF
Chicago Tribune Endorses Gary Johnson
2 hours ago
THE LATEST

No matter that his recall of foreign leaders leaves something to be desired, Gary Johnson is the choice of the Chicago Tribune's editorial board. The editors argue that Donald Trump couldn't do the job of president, while hitting Hillary Clinton for "her intent to greatly increase federal spending and taxation, and serious questions about honesty and trust." Which leaves them with Johnson. "Every American who casts a vote for him is standing for principles," they write, "and can be proud of that vote. Yes, proud of a candidate in 2016."

FUNERAL FOR ISRAELI LEADER
Obama Compares Peres to ‘Giants of the 20th Century’
2 hours ago
THE DETAILS

Speaking at the funeral of former Israeli Prime Minister Shimon Peres, President Obama "compared Peres to 'other giants of the 20th century' such as Nelson Mandela and Queen Elizabeth who 'find no need to posture or traffic in what's popular in the moment.'" Among the 6,000 mourners at the service was Palestinian President Mahmoud Abbas. Obama called Abbas's presence a sign of the "unfinished business of peace" in the region.

Source:
THE QUESTION
How Many New Voters Does the Clinton Campaign Aim to Register?
2 hours ago
THE ANSWER

Three million—a number that lays "bare the significant gap between Donald Trump’s bare-bones operation and the field program that Clinton and her hundreds of aides have been building for some 17 months."

Source:
“STANDING FOR PRINCIPLES”
Chicago Tribune Endorses Johnson
3 hours ago
THE LATEST

In a somewhat shocking move, the Chicago Tribune has endorsed Libertarian candidate Gary Johnson for president, saying a vote for him is one that voters "can be proud of." The editorial barely touches on Donald Trump, who the paper has time and again called "unfit to be president," before offering a variety of reasons for why it can't endorse Hillary Clinton. Johnson has been in the news this week for being unable to name a single world leader who he admires, after earlier this month being unable to identify "Aleppo," a major Syrian city in the middle of the country's ongoing war.

Source:
NEVER TRUMP
USA Today Weighs in on Presidential Race for First Time Ever
16 hours ago
THE DETAILS

"By all means vote, just not for Donald Trump." That's the message from USA Today editors, who are making the first recommendation on a presidential race in the paper's 34-year history. It's not exactly an endorsement; they make clear that the editorial board "does not have a consensus for a Clinton endorsement." But they state flatly that Donald Trump is, by "unanimous consensus of the editorial board, unfit for the presidency."

Source:
×