Media Briefings

JOURNAL RANKINGS MAY DETER ECONOMISTS FROM ADDRESSING BIG QUESTIONS

  • Published Date: August 2013

Issues such as climate change, innovation, health and good governance would all benefit from more input from economics. Yet the incentives in the profession deter economists from publishing in journals in these areas. They also deter the top journals from publishing papers on these issues – and they make it difficult for new journals to be established unless they are backed by a big professional association.

These are among the conclusions of Professor John Hudson in a study of journal rankings, published in the August 2013 issue of the Economic Journal. His analysis of the powerful impact of journal rankings also suggests that the use of certain lists pushes economists to work in such areas as finance and management.

Every eight years or so, universities up and down the land submit their work to be evaluated in what is currently known as the Research Excellence Framework (REF). For economics, despite claims to the contrary, the basis for this exercise is the quality of the journal in which a paper was published. There is a widespread assumption that one journal is perceived as 4* while another is only 2*.

A further assumption is that all economists know this ranking. Of course they don’t – and there is disagreement at the margins. But to help them in this task, there are various lists. In the UK, two of the most widely used are the Keele list and the Association of Business Schools (ABS) list. Both contain an element of subjectivity.

The main contribution of this study is to link these lists to a set of metrics, such as the number of citations that papers in a journal get, and using these metrics to produce an up-to-date list purged of any subjective bias.

The resulting list contains journals that can confidently be called 4* and others that can confidently be called 3*. But there is a grey area in between, where two journals can be very close together on the metrics, and to put one definitively in a specific category and another in a different category seems slightly arbitrary.

Hence, the study also classes journals as ‘probable 4*’ and ‘possible 4*, for example. For papers in these journals, the REF panel – and anyone else seeking to make a judgement on the work – had better read them and form a judgement.

The main interest in this study will be from academic economists, journal editors and publishing houses. But there are a number of other conclusions that carry wider relevance.

First, the metrics that influence the ABS list are different to those that influence lists produced by economists, lists such as the Keele one but also others. Economists tend to value a focus on economics. Hence, for example, a journal whose articles tend to be widely cited in journals outside economics will tend not to be as highly rated by economists as a similar journal with a more economics focus.

This is not the case in the ABS list, where an economics focus is actually penalised. Thus, the widespread use of the ABS list in economics departments is distorting incentives, pushing economists to work in areas such as finance and management, which the ABS does value – it is after all a list primarily for business schools – and downgrading those economists who work in other areas.

Economists also tend to value theory journals more than other journals. They also typically revere age – that is, very old journals tend to be rated more highly than their metrics might strictly suggest.

This latter attitude may well be a bias towards what economists have grown up with. But it means that new journals face a tough time in getting established, unless they are backed by a major organisation, such as the American Economic Association or the European Economic Association. The main exception to this would be some of the electronic journals published by Berkeley.

Professor Hudson comments:

‘The emphasis that economists put on their own discipline is not unusual and, in many ways, it is justifiable. After all, if you want to know which is the best economics journal, it is presumably the one that has most impact on other economics journals.

‘Similarly, if you are judging an economist, you will rate more highly their publications in a 4* economics journal than a 4* sociology journal.

‘But there are many issues in the world that can benefit from the toolkit and perspective of good economists, issues such as the environment and governance.

‘Yet the incentives deter the economist from publishing in journals in these areas as well as deterring top economics journals from publishing this type of paper. That is a pity.’

ENDS

Notes for editors: ‘Ranking Journals’ by John Hudson is published in the August 2013 issue of the Economic Journal.

John Hudson is in the Department of Economics at the University of Bath.

For further information: contact Romesh Vaitilingam on +44-7768-661095 (email: romesh@vaitilingam.com; Twitter: @econromesh); or John Hudson via email: j.r.hudson@bath.ac.uk