Letter from America

Your wolf is interfering with my t-value!

Angus Deaton reports on the creeping regulation affecting academic research in US universities.

My friend Dustin raises Black Angus cattle on a ranch in Montana. Yellowstone National Park, about 120 miles to the south, reintroduced long-eliminated wolves fifteen years ago, and their descendants are now widespread in South West Montana. According to Dustin, they prey on his cattle, and he regards the restoration of wolves with only slightly less detestation than he would the restoration of his land to the Crow and Shoshone who once lived there. To him, the wolves are a pest, imposed on him by coast dwelling liberals, who compound the injury by passing laws that prevent his protecting his livestock. According to a doubtlessly exaggerated account, when he has his rifle sights on a wolf that is attacking a calf, he must get live permis¬sion from all of the bureaucrats who have jurisdiction over him, his rifle in one hand, his cellphone in the other, and letting his 4x4 steer itself. He hates bureaucrats, he hates the federal government, and he hates the Darwin-believing Democrats who force him to host their not-so-cuddly pets.

Mitt Romney agrees. His website notes that “Regulations function as a hidden tax on Americans” with a burden that is “much higher than the entire burden of individual and corporate taxes combined. The end result is an economy subject to the whims of unaccountable bureaucrats pursuing their own agendas.” Among other regulations to be abolished if Romney is elected are those against emissions of carbon dioxide, those that limit the production of coal and oil, as well as the Dodd-Frank regulations of the financial sector, though these last are to be replaced by something else, not currently specified. Chief among the unaccountable bureaucrats, at least until his recent resignation, was President Obama’s head regulator, Cass Sunstein, the coauthor with Richard Thaler of Nudge, who is roundly denounced on Romney’s webpage. Sunstein’s performance was widely criticized from both sides, and he was accused of being both an under- and over-regulator, perhaps as good an accolade as could be expected.

Universities are not as heavily regulated as power plants, pharmaceutical companies, or banks. Yet regulation is becoming a significant part of the lives of academic economists in America. After the Oscar-winning documentary Inside Job pilloried the financial malfeasance of economists, the American Economic Association imposed disclosure requirements on its authors ($10,000 or more from interested parties, in cash or kind—including data—over the previous three years.) The National Institutes of Health, reeling from a series of scandals in which prominent scientists were paid hundreds of thousands of dollars by pharmaceutical companies to provide apparently disinterested scientific backing for their products, has instituted a new set of conflict of interest rules. According to an account in Nature, Sunstein successfully moderated the draft proposals, but the final rules are draconian enough. Those of us who have NIH grants must make annual declarations of all of our family’s sources of income and holdings of assets (other than mutual funds—a gift to the industry), as well as trips financed by others (e.g. for me to give a talk in London), and if “an unaccountable bureaucrat” decides that there is a possible conflict, all of these income sources, assets, and trips will be made publicly available by Princeton, either on a website, or on request within five days. One consequence of these and related regulations is that the groups that support research within universities, originally set up to help academics with logistics and funding, have been turned into unpaid policing organizations for the state. Our shepherds have been turned into wolves, and many of us have had to hire yet more bureaucrats to deal with their demands. The cost of meeting these regulations is large, not least in the fast rising ratios of bureaucrats to researchers in universities. The AEA, unlike the NIH, has no penalties for those who ignore its rules, and one might wonder whether these regulations, well-intentioned although they are, will justify their burden by doing much to rein in evildoing; as Dean Inge noted, it does the sheep little good to pass resolutions in favor of vegetarianism when the wolf remains of a different opinion.

The regulators are concerned, not only with financial ethics, but with statistical ethics, not only our values but our t-values. In order to prevent the selective reporting of the results of randomized controlled trials, all clinical trials must be pre-registered with the government, and medical journals will not publish results where this protocol has not been satisfied. The American Economic Association is under pressure to maintain (and enforce) a similar registry for economic experiments. Judging by recent trends in medicine, this is only the beginning. An article in the Journal of the American Medical Association argues that all results should be reported to the state, and that a well-run research team should be able to monitor and report every cross-tabulation or exploratory regression; otherwise, how can we judge the significance of the results that are published? Along similar lines, it is argued that we must somehow take into account not only all of the calculations that have been done, but those that people chose not to do. That such regulations might have an adverse effect on creativity is noted, but dismissed, given the absolute necessity of preserving the purity of t-values. Bias should be regulated by sticking to randomized controlled trials, and t-values regulated by banning free form data exploration.

It is hard to believe that the science is so fragile that it needs this sort of protection. Would the case for (or against) a stimulus be affected by a new revelation about Keynes’ financial portfolio? Should we stop using antibiotics until we have an account of paid trips made by Alexander Fleming? Or until we are sure that his assistants did not peer into the petri-dish in advance? One of the problems is surely the reliance on randomized controlled trials, whose results depend, not on scientific advance or on convincing demonstration of a new mechanism, but on the precise circumstances under which they were conducted; if the experiment was tainted in any way, the results are automatically suspect. If RCTs are the only way of doing science, then we need the bureaucrats to guard the sanctity of their t-values. Yet, one wonders how so much good science managed to get done in the past.

If we want to have both cattle and wolves in Montana, someone is going to have to sort out the inevitable conflicts. But are we really so overrun with wolves in economics? Is data mining such a serious threat? And while there certainly are villains, are they really undermining the integrity of advance in economics? And would we really be better off running under state supervision?

From issue no. 159, October 2012, pp.3-4.

Page Options