Not just the top five journals: A recipe for European economists

What the paper says

Recently, the incentive structure facing doctoral students and researchers in economics has changed significantly in many European countries as a result of the adoption of the US approach to evaluating research output. This poses a threat to the development and position of the subject of economics in Europe since evaluation for promotion is characterised by an excessive focus on publishing in the five most highly ranked journals, all but one located in the United States. However, the probability of getting an article into the top five is low and the social cost of the promotion system is consequently high. Promotion criteria other than just top-five publications should be used as a guide. About 30 years ago, two Swiss economists, Bruno Frey and Reiner Eichenberger (1993), described how incentives differed between the European and North American academic markets for economists. They argued that because US economists were expected to be more mobile, there were fewer reasons for them to study conditions specific to a particular region or country. Frey and Eichenberger (1993, p. 189) painted a very different picture for Europe: In a typical European country, the smaller market size and the higher degree of government intervention give quite a different picture. Full professors (and many others) have life-long employment, and their scientific performance has virtually no effect on salary and position, with obvious effects on the motivation for academic work. Economics professors are induced to invest their human capital in specific knowledge of local economic problems and institutions, which is helpful for a political career and getting various appointments. Graduate students in economics know that they will almost certainly stay in the same country, quite likely even in the same city, and hence have an incentive to learn about local institutions. Since this was written, the situation has changed considerably. The use of English as a lingua franca has allowed greater exchange and mobility among European researchers and across borders. Postdoctoral and tenure-track positions are filled through recruitment in the institutionalised US or European job market. Is this constructive? How should doctoral education and post-doctoral careers be organised in Europe? Until the late 1970s, doctoral candidates usually prepared and defended monographs. When compilation theses, consisting of a number of separate essays, began to take over, the constituent papers were expected to revolve around a common theme. However, over time it has become increasingly common that a dissertation consists of ‘three essays in economics’, without any connection between them. In line with the American model, doctoral students are increasingly expected to concentrate on a long ‘job market paper’ which can be presented to prospective employers.1 The idea is that it should show how skilled the PhD student is as an economist. Technical know-how, economic intuition and ability to conduct independent, innovative research should be demonstrated in this paper. Job market papers tend to be extraordinarily long, increasingly to the point where they approach the length of the old monographs.2 Everything has to be shown. But does the research idea emanate from the doctoral student or from the supervisor? Moreover, the thesis should have a single author, but this is far from always the case, especially in Europe. There are even job market papers where the student has several co-authors. But who did what, and what does the thesis say about the PhD student's savvy? Does it reveal a skilled economist, a virtuoso number cruncher, or a skilled networker? What does it signal about the creativity, perseverance, and motivation of the doctoral student? Once the thesis is finished, the doctoral student or recent graduate embarks on a tour, presenting it at as many conferences and department seminars as possible, hoping to get a postdoc position for one or two years or a tenure-track position for four to six years in order to qualify for a permanent position. Doctoral students put an enormous amount of work into acquiring the qualifications that allow them to be admitted to such positions. Departments, in turn, spend a lot of time and resources identifying which of the applicants should get an offer. Letters of recommendation are written, essays are read, candidates to be interviewed are identified, flights are planned, and seemingly endless job interviews are conducted. To succeed in getting an academic position requires that a significant part of the working time is spent on reducing the sometimes more than 100 pages of the job market paper to 20–25 pages to make it submittable for potential publication in one of the top five journals – American Economic Review, Econometrica, Journal of Political Economy, Review of Economic Studies and Quarterly Journal of Economics – thus beginning the march that may ultimately lead to a professorship at some respectable university at home or abroad. Unfortunately, most of the time this fails. In a thought-provoking article in the Journal of Economic Perspectives, Conley and Önder (2014, p. 212) state: If the objective of graduate training in top-ranked departments is to produce successful research economists, then these graduate programs are largely failing. Only a small percentage of economics PhDs manage to produce a creditable number of publications by their sixth year after graduation. Even at the top-five departments, it would be hard to argue that the bottom half of their students are successful in terms of academic research. The number of AER-equivalent papers of the median at year six is below 0.1 in all cases and is in fact zero in most. At the majority of the departments ranked in the top ten in conventional rankings … 60 percent of their students fail to meet this 0.1 AER-equivalent standard, and for the majority of the PhD graduates of the top 30 departments, 70 percent fail. Too many researchers continually revise their papers for five or six years without getting them published in any of the golden journals. Some of them will even let a manuscript remain unpublished rather than ‘pollute their CV’ with a publication in a lower-ranked journal. Unfortunately, this has proven rational from a merit point of view. When Powdthavee et al. (2018) had economists compare two publication lists with an identical number of articles in top journals, but where one also had a number of articles in lower-ranked journals, ceteris paribus, the latter was rated inferior! A recent interview survey conducted in three top-ranked Scandinavian economics departments (Hylmö et al., 2024) confirms this conclusion. The interviewees agreed that in economics, quality ratings are based solely on top-five publications, while citations are considered irrelevant. In terms of subject matter, the focus is on areas of general interest, not on specialised fields, and certainly there is no interdisciplinarity. Even more worrying, the interviewees stated that there is no ranking below the top five. The ranking is not even lexicographical – below the top five there is nothing. In practice, however, the criterion can rarely be applied and publications below the top five must be considered. The rationality of this system is dubious. The key insights from our science are not applied in evaluations of ourselves: diminishing returns, finite resources, opportunity cost, comparative advantage, and specialisation have no place. Spending five or six years on a paper hoping to achieve an unrealistic goal amounts to poor resource allocation. It would have been better to publish in some less ‘meritorious’ journal and then start a new project. The insights of economics also include the value of risk reduction through diversification – not putting all your eggs in the same basket. The struggle to get articles into top journals runs the risk of being socially inefficient and welfare-reducing, as it may discourage researchers from publishing research that may be of great value because such publications reduce their reputations within their own profession. This means that society does not get to enjoy the value of the results obtained with the resources that research funders (usually the taxpayers) have contributed to making the research possible. With the publication criterion we have just described, most new PhDs will of course be useless in the sense that the overwhelming majority fail to get into the top five. What is the probability of getting into the top journals? We provide two answers. First, Frey (2009, p. 335) suggests that: Academic economists today are caught in a ‘Publication Impossibility Theorem System’ or PITS. To further their careers, they are required to publish in A-journals, but this is impossible for the vast majority because there are few slots open in such journals. Such academic competition is held to provide the right incentives for hard work, but there may be serious negative consequences: the wrong output may be produced in an inefficient way, the wrong people may be selected, and the losers may react in a harmful way. At the same time, other meaningful activities are hampered. Björklund (2014) notes that the focus on top publications means that there is less time for domestic investigation and debate, especially since top publications often require extensive and long-term collaboration with researchers in other countries. Top American universities can possibly afford a top-five criterion. But even in the United States, there is criticism of the top-five focus (Akerlof, 2020; Deaton, 2023; Heckman & Moktan, 2020).4 In addition, the number of papers published by the five top-ranked journals has decreased over time. Today they publish around 350 papers a year. According to Deaton (2023, p. 179), the bar to entry has gradually been raised, particularly for researchers outside North America. It follows that getting published in the top five is impossible for the overwhelming majority of those doing research in Europe to qualify for a tenured position. Deaton (2023, p. 178) emphasises that of the top five, two are house journals for Harvard and Chicago (The Quarterly Journal of Economics and Journal of Political Economy, respectively). Researchers with links to these two universities may therefore have significantly greater prospects of being published in these journals. Four of the five journals have their editorial boards in the US, which contributes to the American dominance and works to the disadvantage of economists outside the United States. Thus, European researchers do not face an even playing field. The easiest way to get a top publication is to gang up with leading US economists, perhaps three or four authors on one paper; but as we have already noted, who has done what, and what does that say about the skill of the European co-author? Furthermore, it is more attractive to work with US data than with domestic data because it increases the chances of being accepted in the most highly ranked journals (Das et al., 2013). The signal sent by the system easily leads to what in our view is undesirable optimisation behaviour by European researchers. Moreover, US institutions are sufficiently large to have a ‘portfolio approach’ where recruitment is guided by what is ‘needed’ to get a diverse faculty composition. There, the top-five approach is complemented or replaced by other considerations. These universities have also established a marketplace where they are able to buy top economists from Europe as well as from each other. Few leave this club unless they do so for homesickness or for an elevated position in their own country, such as governor of the central bank. The stated preferences of economists tend to be at odds with the US top-five practice of our discipline. A worldwide survey of 10,000 researchers found that opinions regarding what research should focus on diverged widely (Andre & Falk, 2021). Most were dissatisfied with both the choice of topics and the objectives of the research. On average, respondents felt that research needs to become more policy relevant and more multidisciplinary, that researchers should be prepared to take more risks, not be afraid to challenge the conventional wisdom, and that the range of research should be widened. The problems and their relevance at the domestic level should also determine what we do. European universities have increasingly adopted the US tenure-track system. Once you become an assistant professor, your fate is in your own hands. You do not have to compete with others. You just need to show a satisfactory publication record to get promoted and tenured. As the goal of research becomes increasingly instrumental – to produce the required number of papers for the next step of promotion – there is a greater risk that the researcher will stop doing research once promoted to full professor; and when competitive recruitments at the senior level are discontinued, no new blood is brought in from outside except for new PhDs. The system encourages mechanical, often erroneous, conclusions. The top five journals are largely fashion magazines, driven by what happens to be ‘in’ at the moment. Many first-rate works have been published in what in the latter-day wisdom is perceived as the wrong place (Gans & Shepherd, 1994). Who has decided that the top five is the only good form of publishing, and why is it where and not what that governs? Writing books then becomes pointless. In the current system, Adam Smith and John Maynard Keynes would not get credit for The Wealth of Nations and The General Theory, respectively. It should be a reasonable requirement for those aspiring to a top academic position to show that they can also write longer works.5 Why is it difficult to get credit for monographs in economics and get due credit when that is the most appropriate mode for the issue at hand? Stockhammer et al. (2021, p. 224), examining the Research Excellence Framework (REF) which serves as the main research assessment for universities in the UK, conclude that “the REF has discouraged the writing of books, as opposed to specialist articles, and by making peer review the ultimate arbiter it has … acted as a curb on intellectual risk-taking and innovation”.6 Using only article publication in (highly-ranked) scientific journals as the decisive criterion for appointments most likely leads to distortions. You may get entire institutions that do nothing but try to get into the top five, usually without success. The best (the monomaniacal) becomes the enemy of the good. Researchers in general become unproductive. The system that has been established is a waste of talent and creativity, a costly pyramid scheme that eliminates the vast majority of talented individuals during their potentially most creative years.7 The obsession with the top five also leads to the avoidance of topics that are not perceived as timely or are seen as difficult to research using the recommended methods. Economic research is path dependent. The safest way to publication is to stay in the dominant groove, the current fad. Methodological pluralism which could shed additional light on a problem is not encouraged (Gräbner & Strunk, 2020). The focus is on the form of publication itself and what researchers believe are the preferences of the editors of the top-ranked journals. We have identified a number of problems with the adoption in Europe of the US system of top five publications. The obvious conclusion is that it should be reformed to create a more varied incentive structure, better adapted to conditions in Europe. The academic merit system should become more diversified and more comprehensive. This is not so difficult once you have defined what an academic researcher and teacher should do. Good guidance is provided by Ragnar Bentzel, a Swedish economist who, when serving on the selection committee for a chair in Uppsala in 1984, specified in detail the concept of ‘scientific skill’ (Lundahl, 2015). Scientific works should be written in an internationally accepted language. The subject of economics must be defined to include interaction with other social science disciplines; and Bentzel found it important that a professor – who must be responsible for research and teaching – has knowledge of a reasonably broad part of the subject. Empirical knowledge is important – about both the domestic economy and the economies of other countries. It is also important to have international contacts and to publish popular pieces. A large economics department also requires professors with diverse profiles. Different qualifications must be weighed against each other. Today, this balanced view has been replaced by one single goal: publishing in the most prestigious journals.8 In short, the merit system should not be based solely on top-five publications. Graduate students are trained to write streamlined articles using the recommended methods on issues deemed of interest to the editors and reviewers of the most prestigious journals (Ruhm, 2019). This counteracts the kind of pluralism that is necessary to address pressing domestic social issues (Novarese & Pozzali, 2010) and to produce the knowledge that can improve the functioning of national economic systems in Europe. In a more diversified PhD programme, theory and methodology would be for of important policy areas and evaluations of policy and their It is not a that such a would many students who from for graduate study in economics because they it as and p. they economics because it as a science that could become a for Moreover, economists this they will be replaced by political and economic The of research is to produce new many people today to that the main objective of research funders is to an costly where most will and the will the It is that such a & is necessary to the most professors who will therefore be the most of economics to of of this view is that senior researchers research when the potential is to publish the results in a top-five journal. there are that fail because the research idea does not to be good and should therefore be But to a just because it is rather of the funders and the ultimate at the national the own we have not the appropriate incentive structure for we would to a between and A professor who doing research should be prepared to take on more teaching and than who to We have the current in Europe adoption of the US for the recruitment of new in short, the top-five According to this the assessment of researchers is based on the publication of articles in the top five journals in economics, or even other The choice of research methods and data is thus largely by of what is seen as by the editors of these journals. In addition, four of the journals are based in the United States. who to get or into the increasingly long line of authors of one or two articles in one of these journals has more or less a the probability of getting an article accepted in one of these five journals is research in the economics departments of leading universities is doing just a different is considered in of the fact that the of article in the top five journals is while the for publishing there is conclusion is that the subject of economics should be more attractive to European This can be only by the incentive structure for education and the merit system for higher positions. We for and and for