Last month Technology Review profiled M.I.T. economist Josh Angrist, who's known for conducting "natural experiments."
That's an academic way of describing research that occurs through observing the world, as opposing to being controlled in a lab. From his morning commute in Cambridge (via bike) to his interest in urban charter schools (analyzing their effectiveness), it's clear that Angrist's powers of observation are largely focused on cities.
Most of Angrist's recent attention has been on urban education in Massachusetts. Because over-subscribed charter schools use a lottery to determine who gets in and who doesn't, they offer a "natural" point of comparison between student populations. Different schools in comparable communities also provide a solid basis for study. Tech Review summarizes two of Angrist's most significant findings:
In a 2009 report, they found that certain Boston charter schools had produced an average gain of roughly 15 percentile points for middle-school students on the state math exams.
Two years later, however, Angrist and colleagues found that in Massachusetts districts outside Boston, charter-¬school students did no better on average than students at other public schools.
In an upcoming issue [PDF] of the American Economic Journal: Applied Economics, Angrist and colleagues expand on the difference between urban and non-urban charter schools. They argue that urban charter schools in Massachusetts outperform urban public schools when it comes to student achievement. They also argue that non-urban charter schools may actually reduce achievement from a baseline level.
So what's happening to make urban charters so effective? Well, part of the explanation is simple demographics. Urban charter schools tend to boost performance for minority students from parts of the city with low achievement scores. In other words, they help students who enter school with lots of room for improvement.
At the same time, the non-urban charter schools evaluated by Angrist and company struggled to boost academic achievement at all. In fact, the researchers think urban charters would outperform non-urban charters even if the student populations were (in a demographic sense) similar. True, non-urban charter students typically enter school with a higher baseline, but the schools themselves don't seem to be elevating the level of achievement:
The resulting estimates show that students at urban charters are typical of the urban student population, and that urban charter attendance boosts achievement well beyond ambient non-charter levels. Student demographics and baseline scores play a role in this — schools work best for minority students and students with low baseline scores — but non-urban charters appear to be ineffective for most subgroups.
A major part of the urban charter success has to do with pedagogical approach. Angrist and colleagues report that a "No Excuses" education strategy accounts for much of the benefit discovered in certain city charters. No Excuses schooling is characterized by strict discipline, basic math and reading skills, instruction time, cold-calling in the classroom, and selective hiring (especially graduates of the Teach for America program).
Angrist is aware of the limits of his research. As Tech Review notes, he admits that charter schools "may be too different from one another to justify sweeping conclusions about whether they provide better education." Additionally, what's true of the Boston metropolitan area, and urban Massachusetts more broadly, may not be true for other cities across the country.
Still, Angrist's work continues to inform the public discussion on charter school expansion. Many states have laws capping the number of charter schools; in 2010, thanks in part to new research, Massachusetts passed a law that relaxed its caps for "proven providers." As Angrist and colleagues argue in their upcoming paper, policies that favor schools with "documented effectiveness" could go a long way toward reducing achievement gaps.
Eric Jaffe is a contributing writer to The Atlantic Cities and the author of The King's Best Highway: The Lost History of the Boston Post Road, the Route That Made America. He lives in New York.