Five years ago Harvard's Caroline Hoxby, a rising star in economics, wrote a paper that reached an unusual conclusion: Cities with more streams tended to have schools with higher test scores.
Today her work is a widely cited landmark in the fierce national debate over free-market competition in public schools. And it's at the center of a bitter dispute with another economist that is riveting social scientists across the country.
Her adversary is Jesse Rothstein, a young professor at Princeton, who says her study is full of flaws. In a rebuttal to her critic, Dr. Hoxby wrote of his work: "Every claim is wrong." She has also accused him of ideological bias. Dr. Rothstein, in turn, says she resorts to "name-calling" and "ad hominem attacks" on him.
The unusual spat has put a prominent economist in the awkward position of having to defend one of her most influential studies. Along the way, it has spotlighted the challenges economists face as they study possible solutions to one of the nation's most pressing problems: the poor performance of some public schools. Despite a vast array of statistical tools, economists have had a very hard time coming up with clear answers.
"They're fighting over streams," marvels John Witte, a University of Wisconsin-Madison professor of political science and veteran of a brawl over school vouchers in Milwaukee in the 1990s. "It's almost to the point where you can't really determine what's going on."
Milton Friedman, the Nobel Prize-winning economist known for his free-market views, proposed 50 years ago that to improve schools, parents could be given vouchers -- tickets they could spend to shop for a better education for their kids. He theorized that the resulting competition among schools would spark improvements in the system. Free-market advocates loved the idea. Teachers' unions hated it, arguing that it could drain resources from some public schools and direct resources to religious institutions.
Research on these programs turns up evidence of benefits from school choice. But it hasn't proved strongly convincing, and testing the hypothesis is anything but simple. In the mid-1990s, researchers battled over how to interpret studies of voucher use in Milwaukee. In 2003, they tried to evaluate voucher experiments in New York and ended up squabbling over the right way to decide if a child was African-American. Last year, in assessing charter schools -- institutions that are publicly funded but not bound by traditional rules -- they argued over how to take into account differing backgrounds of the children who attend.
Analysts have searched as far away as New Zealand for evidence about the effects of competition in education -- and disagreed about what was found there, too. Now there is Hoxby vs. Rothstein.
Dr. Hoxby, 39 years old, is one of only two women tenured in Harvard's economics department, a distinction she achieved just seven years after earning a doctorate from Massachusetts Institute of Technology. Other universities, such as Stanford, have tried to lure her away. Harvard, in turn, has given her a prestigious endowed chair.
Although her father, Steven Minter, was an official in the Carter administration Education Department, she has become a favorite in Republican circles for producing statistical evidence that competition improves schools. "This is a person who is smart, who is logical, who is committed and who is dedicated," says Rod Paige, President Bush's first Secretary of Education. Dr. Hoxby also is a distinguished visiting fellow at the Hoover Institution, the right-leaning research center affiliated with Stanford.
Dr. Rothstein, 31, is the son of Richard Rothstein, a former textile-union organizer who's now a lecturer at Columbia. Father and son have both worked closely with the left-leaning Economic Policy Institute in Washington. The son got interested in the streams paper while studying for his doctorate at the University of California, Berkeley. He is now an assistant professor at Princeton, not yet eligible for tenure. His Berkeley thesis adviser, David Card, describes Dr. Rothstein, who had majored in math as a Harvard undergraduate, as "tenacious" and having "very good technical skills."
In her 2000 paper, published in the prestigious American Economic Review, Dr. Hoxby explored competition among public schools. She noticed that some metropolitan areas, like Boston, had dozens of school districts, while others, such as Las Vegas, were dominated by just one. She reasoned that if pro-competition economists were right, school systems with many districts should produce better results, because parents in those cities would have more choices about where to live and educate their children, creating a more competitive environment.
To test this notion she might have simply counted the number of school districts in cities. But there were factors that muddied the waters. Sometimes the quality of the school districts influenced their number. That is, in some cases, it appeared cities had numerous districts partly because some were bad -- so bad they couldn't be closed or merged with others. It was the kind of chicken-and-egg problem that often trips up economic research.
Dr. Hoxby tried to find a way around this. She noticed that the number of school districts seemed related to geography. Streams were natural boundaries around which districts were formed many years ago. Cities with lots of streams had more school districts than cities with few streams.
An Opportunity
Testing a hypothesis in economics isn't as straightforward as, say, testing a drug, where researchers can randomly assign some subjects to receive a placebo. Many economists believe they can approach scientific rigor, however, by taking advantage of random events like draft lotteries and judicial assignments. For Dr. Hoxby, streams offered such an opportunity: Cities with lots of streams had been randomly chosen by nature to have more school districts and more school competition, while cities with few streams were naturally home to fewer districts and less competition.
"By using the variation in the number of school districts in a metropolitan area that is driven by streams, we can isolate the effect that interests us: the causal effect of more districts on achievement," she said in an interview via email.
When she found that metro areas with more streams tended to have more districts, and also higher student achievement, many academics thought she had come up with an ingenious way of testing Dr. Friedman's competition thesis. "Caroline had a great idea with that paper," says David Figlio, an education economist at the University of Florida. "It is incontrovertible that it was a brilliant insight."
Dr. Rothstein says it doesn't stand up to scrutiny. He makes several technical challenges, but his main attack is on the way the author counted streams.
A problem she faced at the outset was that some streams can affect more than school-district borders. Large, navigable ones affect commerce and wealth in an area and the kind of population it attracts -- influences that could distort her test. Small streams wouldn't have this problem, Dr. Hoxby said. She divided her streams into larger and smaller ones and entered them into her equations separately to make the distinction clear. Studying detailed maps published by the U.S. Geological Survey, she measured dimensions of water bodies in hundreds of metropolitan areas.
Dr. Rothstein said Dr. Hoxby never laid out exactly how she measured a stream's width, which, he noted, can vary at different points in its course. In the case of Fort Lauderdale, Fla., which she found to have five large streams, Dr. Rothstein said when he tried to count them her way in the map room at Princeton's Fine Hall Library, he came up with 12. His research assistant got 15, he wrote in "A Comment on Hoxby," which the National Bureau of Economic Research published in March (Read an abstract). The American Economic Review is now preparing to publish a version of that paper.
"The exercise makes clear that Hoxby's larger-streams variable is subjective and unverifiable," Dr. Rothstein's paper added. In an interview, he says more bluntly: "Fort Lauderdale is a swamp. The idea that streams were a relevant thing in this area is just crazy."
He tried other ways to count streams. Using government databases, he distinguished between larger and smaller ones by their recorded length, and also by whether they flowed through more than one county. With these other measures, he wrote, he found a link between streams and school performance, but too small to be statistically significant.
Dr. Rothstein also complained that for years Dr. Hoxby ignored his requests for the data she used. Economists frequently argue about the availability of data. In this case, Dr. Hoxby said her ability to circulate all her data was limited because some came from the National Center for Education Statistics, which restricts public access to some of its information to protect the identities of students and school districts. Dr. Rothstein contends that the data she ultimately made available don't match up to the data used in her paper.
Dr. Hoxby responded in a National Bureau paper called "Competition Among Public Schools: A Reply to Rothstein," also slated for publication in the American Economic Review. By tossing aside her hand counts of streams, Dr. Hoxby wrote, Dr. Rothstein was proposing "the destruction of important information" and "promoting less informed measurement." (Read an abstract)
She added that "it is dismaying to see a great deal of my work (with strategic changes made by him) appearing on Rothstein's Web site, presented as though it were mainly if not wholly his work." And she wondered: "Why, especially when he has the benefit of hindsight, careful explanations, and nearly all of the data work done for him, does he make bad decisions repeatedly?"
It may be, Dr. Hoxby wrote, that Dr. Rothstein "was determined to generate results that would contradict those in [her original paper] at whatever cost."
Defenders Irked
The rejoinder irked his defenders. "Her nasty, vicious response is really about shutting down debate," said Lawrence Mishel, president of the Economic Policy Institute. The group has sparred with her before. A book co-written by Dr. Mishel and Richard Rothstein, Jesse's father, dedicates a section to challenging her work on charter schools.
In an email, Dr. Hoxby responds that "EPI's work is funded by unions, and the teachers' unions are openly opposed to charter schools for reasons of self-interest." EPI says it gets 29% of its funds from unions.
In July, Harvard's student newspaper, the Crimson, quoted Dr. Hoxby, whose father is African-American, as saying that "there is a lot of race and gender bias going on here."
In an email, Dr. Hoxby says that the paper misrepresented her views and that she had made no allegation of racial or gender bias. The Crimson's president, Harvard senior Lauren Schuker, says the paper stands by "a fair story [that] covered all sides." (Read the Crimson article)
Writing to the Crimson, Dr. Hoxby accused Dr. Rothstein of "ideological bias." He says allegations of bias are "absurd" and unhelpful "name-calling."
In the background of the fight is the broader attempt by some economists to adopt the scientific convention of repeating, or "replicating," studies as a way to verify them. "Unfortunately, economics lacks a strong tradition of scientific replication," Dr. Hoxby wrote in her reply to Dr. Rothstein. She added: "Some economists even seem to believe, confusedly, that replication is not worthwhile unless it upsets a previous result. Rothstein may suffer from such confusion."
Dr. Rothstein: That's "ad hominem" criticism. An economics Web log called "The Lowest Deep" (thelowestdeep.blogspot.com) sums up the squabble as a "nerdy Celebrity Death Match."
The author of the competition thesis, Prof. Friedman, hasn't weighed in on the spat, but his enthusiasm for free-market approaches in education is undimmed. "The case for vouchers is so simple," the 93-year-old Dr. Friedman said to a crowd at New York's Mandarin Oriental Hotel in June, at a gathering to mark the 50th anniversary of his idea. "In area after area, things the government does, private enterprise can do at half the cost." In an interview, he describes Dr. Hoxby as "a very intelligent gal" whose papers "impress me favorably."
At the American Economic Review, editor Robert Moffitt says the journal wouldn't publish Dr. Rothstein's paper if its editors thought it had no merit. But he adds that the decision to publish isn't an endorsement of Dr. Rothstein's critique. "It's like we're saying, 'It's a good question,' " he says.
Meantime, he adds, the Review is trying to get the professors to dial down the language. "Editors make a lot of changes to assure that both parties are meeting an academic and civil tone," he says.
| Newsletter signup | Send us your ideas