Having qualified academic publications is one of the prominent criteria used for measuring the performance of academics in some universities.

Continued from last Saturday. First the summary of the previous week's commentary: Academic researchers submit their studies to academic journals to share their research. This is a thing explorative academics like to do. On the other hand, academics must have conducted qualified studies in order to be awarded the assistant professor, associate professor and professor titles. Moreover, having qualified academic publications is one of the prominent criteria used for measuring the performance of academics in some universities.

Unfortunately, due to the last two factors, some academics might feel like "they should have a publication no matter what the quality is". Thus, many new journals spring up. Therefore, the problem of measuring the scientific quality of the published studies arises. The short-cut "solution" of this problem is to refer to the indices that include only the journals above certain standards. Namely, the said study is recognized to be qualified if it is published in a journal involved in a respected index.

It seems reasonable up to this point; but it is not. First, there are journals of high quality that are not involved in the indices. Second, there are journals of low quality that are involved in the indices. Third, the practice of recording only the articles in indexed journals discourages the issuance of high-quality new journals as they would face difficulty in attracting qualified articles. Fourth, the number of articles in social sciences and on developing countries published in journals might decrease as it is difficult to have an article published in indexed journals. There exists another well-known problem: academics search for journals that accept articles more "easily" and send those a number of articles that only amount to a hill of beans in terms of quality.

Let me give an example about how the situation might turn. The indices also involve journals that publish studies only in one discipline such as health economics, energy economics or defense economics. As there are only a few journals on a certain discipline, the studies published in one of these necessarily cite other studies published in the journals on that particular discipline. The indices rank journals, too. One of the ranking criteria is how many times the articles in one journal are cited in articles in other journals. Therefore, the discipline-specific journals have high citation figures. My example is in two well-known journals on energy economics. A study published in one of these in the early 1970s' investigates the relation between energy consumption and growth with a simple method. This method is now widely and simply used since the relevant computer programs have advanced. In addition, the relevant data can be accessed easily. So, after this publication, studies on this subject mushroom.

It does not allow deep interpretations

The interesting part is that, the inventor of the model Nobel Prize Winner Prof. Granger states that you cannot deduce an economic policy using the method. The method only tells to what degree a variable helps to estimate the future value of another variable. Thus, it does not allow deep interpretations.

On the other hand, for instance, many studies published in the mentioned journal frequently involve the following lines: "It is identified that there exists Granger-causality between the energy consumption and economic development in country X in Y period. The said country shall not implement energy saving policies. But in country Z, no such causality exists. Thus, this country can introduce energy saving policies." The funny part is; some studies examine a number of countries in one article and then list the results: this for Chad, and for Mozambique but that for Sudan... The studies do everything the father of the method warns that they should avoid. Probably, some of these researchers do not care about the studies at all. But there are journals which publish studies composed of a simple technique and a couple of data... Then you end up with a huge number of studies on the same subject. Please do not get me wrong; there also are neat studies in the mentioned journals and on the mentioned or other subjects. However, it is true that the "index obsession" gives way to a "mushroom disease".

This commentary was published in Radikal daily on 23.04.2011