Do Journals Accept Too Many Papers? Vidya Ataly Department of Economics Cornell University Accepted for publication by the Economics Letters, 2010

Abstract This paper provides a theoretical model for analyzing the behavior of peer-reviewed journals. It …nds that, apart from natural human errors, ine¢ ciencies arise purely for reasons of inter-journal strategic behavior. Speci…cally, as a result of competition, journals tend to set their quality cut-o¤s excessively low. Keywords: peer-review system, market structure of journals, journal acceptance rate. JEL classi…cation codes: L1, L2. I am grateful to Kaushik Basu and an anonymous referee for helpful comments. 451 Uris Hall, Department of Economics, Cornell University, Ithaca, NY - 14853, USA. Phone: +1-443-995-7126. e-mail: [email protected] y

1

1

Introduction

Refereeing a paper for a journal to judge whether it is of high enough quality and satis…es the goals of the journal is di¢ cult enough in the best of times and it is sometimes even tougher because of the innovative and unfamiliar methods used in papers. Hence many papers of really good quality get rejected. George Akerlof’s seminal paper “Market for Lemons” was rejected thrice before being accepted (Gans and Shepherd, 1994). The reverse mistake of accepting papers that ought not to be set in print also, undoubtedly, occurs. There has been a rising concern about the ‡aws in the peer-review system (Blank, 1991; Hamermesh, 1994; Ellison, 2002). However, discrepancies between ideal acceptance rate and the actual acceptance rate can also occur for reasons of pure ‘market structure,’that is, stemming from strategic matters of inter-journal competition. The present paper is a contribution to this latter source of ine¢ ciency. What the paper does is to take this tiny slice of the large problem of e¢ cient peer review and analyze if there are ine¢ ciencies that stem from this. Indeed, I …nd that there are such ine¢ ciencies. In particular, I …nd that in the presence of errors in estimating the true quality of a paper, journals lower their quality cut-o¤s while competing against each other.

2

Model

Consider two journals in a …eld of publication - A and B. They have the same rank, same cost of publishing a given number of papers and make similar mistakes in reviewing an article. Suppose that each journal receives a continuum of papers for review. Assume that, quality-wise, the papers are uniformly distributed over [0; 1] : After review, journal k; k 2 fA; Bg ; observes quality qik for paper i; qik is uniformly distributed over [qi "; qi + "] where qi is the true quality of the paper i and " is the error in the review process. Assume that the errors are same for both the journals. Each journal k sets a cut-o¤ q k on the observed quality of a paper for publishing it. Thus, paper i is accepted by journal k if and only if qik q k ; otherwise rejected. If a journal rejects a “…rst-time submission,”then the paper is submitted to the other journal. I shall refer this as “second-time submission.” If a paper is rejected by both journals, it is abandoned.

2

For each journal, cost of publishing n papers is C (n) = cn:1 It is reasonable to assume that a journal’s objective is to improve its rank.2 Di¤erent analysts suggest di¤erent ways to rank a journal. Some rank journals according to the number of citations others make to the articles in the particular journal (Liebowitz and Palmer, 1984), whereas some others rank journals according to the ranks of the departments a¢ liating the authors of the articles in the particular journal (Moore, 1972). I take the …rst approach. Hence, in this paper, the objective of a journal is to maximize the total quality of the papers3 published minus the cost of publishing. For journal k; let Qk denote the expected total quality of accepted papers and nk denote the expected number of papers accepted. Therefore, nk and Qk can be written in two parts: nk = f1 q k + f2 q k ; q l ; l 2 fA; Bg ; l 6= k; Qk =

1

qk +

2

qk ; ql ;

where f1 q k denotes the expected number and 1 q k denotes the expected total quality of papers accepted by journal k from the …rst-time submissions and, likewise, f2 q k ; q l denotes the expected number and 2 q k ; q l denotes the expected total quality of papers accepted by journal k from the secondtime submissions. Let k be journal k’s pay-o¤: k

= Qk

C nk ; k 2 fA; Bg :

Therefore, each journal k’s objective is to set the cut-o¤ on observed quality of a paper q k such that its pay-o¤ is maximized. Assume that q k 2 (2"; 1 2") ; that is, journals set the cut-o¤ neither very low so that they end up accepting most of the papers, nor very high to avoid rejection of too large a number of submissions. Assume that the error in the review process is small enough: 0

"

1 5

1 Linear cost is for the technical simpli…cation. All the main results of this paper hold (qualitatively) with a convex cost function. 2 Note that this is just a horizontal competition between journals. 3 The number of times a paper is cited gives its quality.

3

and the cost of publishing is neither very high so that journals don’t want to publish at all, nor very low so that the cost does not a¤ect the pay-o¤ much: 2" c (1 3") :

2.1

First-time Submissions

Since, for each journal, the observed quality of a paper of true quality qi is uniformly distributed over [qi "; qi + "] ; the probability of acceptance of this paper by journal k 2 fA; Bg is given by

=

Pr qik 8 < :

1 2"

q k j qi 1; 0; qi + "

if if ; if

qk

qi > q k + " qi < q k " qi 2 q k "; q k + "

Let us now …nd the expected number of papers to be accepted by journal k from the …rst-time submissions: f1 q k =

Z1

Pr qik

q k j qi dqi

0

=

qk :

1

(1)

Similarly, the expected total quality (for this pool) of accepted papers is given by 1

qk

=

Z1

=

1 1 2

qi Pr qik

q k j qi dqi

0

q k2

4

"2 : 3

(2)

2.2

Second-time Submissions

The probability of acceptance of the paper of true quality qi by journal k, which has been rejected by journal l; is given by Pr qik 8 <

=

:

1 2"

q k j qi and qil < q l 0; 1; qi + "

qk

qi > q l + " or qi < q k " q k + " < qi < q l + " qi 2 q k "; min q k + "; q l + "

if if ; if

Therefore, the expected number of papers accepted by journal k from the second-time submissions is f2 q k ; q l l +" qZ

=

80 > > > <

=

q k j qi and qil < q l Pr qil < q l j qi dqi

Pr qik

0; 3 1 l q q k + 2" ; 24"2 1 k q l + 2" q k + 24" 2 q l k q q ;

> ql > > :

if if 3 ; if if

q l + 2" q k q l q k q l + 2" q l 2" q k q l q k q l 2"

(3)

Similarly, expected total quality of accepted papers in journal k for this pool of papers is qk ; ql

2

=

l +" qZ

qi Pr qik

q k j qi and qil < q l Pr qil < q l j qi dqi

0

=

8 > > > < > > > :

0; 1 2

q l2

1 24"2 q k2

ql +

q k + 2" 1 24"2 1 l2 2 q

qk

q l + 2"

if 3

q l +q k

(

2

q l + 2" k2 ; q

5

)

;

3

if q l +q k

(

2

)

; if if

ql

qk

qk

q l + 2" (4) q l 2" q k q l q k q l 2"

2.3

Cut-o¤ Quality in Nash Equilibrium

We have now all the information needed to …nd out the Nash equilibrium of this model. Recall that the pay-o¤ of journal k; k 2 fA; Bg is given by: qk ; ql =

1

qk +

2

h i c f1 q k + f2 q k ; q l

qk ; ql

where l 2 fA; Bg ; l 6= k: A Nash equilibrium here is given by a pair of quality cut-o¤s q A ; q B 2 (2"; 1 2")2 such that each journal maximizes its pay-o¤ given the other journal’s cut-o¤ and no one wants to deviate unilaterally to choose some other cut-o¤ level. Hence, q A ; q B are such that the following conditions4 hold: @ q; q B @ q; q A jq=qA = 0 and jq=qB = 0: (5) @q @q From equations (1) ; (2) ; (3) and (4) ; we get the …rst order condition as follows: @

=

8 > > > < > > > :

qk ; ql @q k qk 2 qk

1 c 24"2 1 c + 24" 2

qk c ; 2 q l q k + 2" 2q k + q l 2 q k q l + 2" 2q k + q l k 2 q c ;

if 3c " ; if 3c + " ; if if

q l + 2" q k q l q k q l + 2" (6) q l 2" q k q l q k q l 2"

In the following lemma, it is shown that the cut-o¤ on the observed quality of a paper for each journal has to be the same in a Nash equilibrium. Lemma 1 There does not exist any Nash equilibrium with q A

q B > 0:

Proof. This can be proved in two steps. First, suppose that there is a Nash equilibrium with q A q B 2": Without loss of generality, let q A q B + 2" : Then, from (5) and (6) ; we have qA = c and qB = c = qA; 4

These conditions are necessary and su¢ cient since the pay-o¤ functions are strictly concave.

6

a contradiction. Next, suppose that there is a Nash equilibrium with 2" > q A q B > 0: Without loss of generality, let q A > q B > q A 2" : Then, from (5) and (6) ; the two …rst order conditions are given by qA

1 qB 24"2

q A + 2"

2

2q A + q B

" +c 1 +

1 qB 8"2

q A + 2"

2

=0

and 2q B +

1 qB 24"2

q A + 2"

Since q A > q B > q A 2q B + = > =

qA

2q B + q A + " +c 2

q A + 2"

2

= 0:

2q B + q A + " + c 2

1 qB 8"2

1 qB 8"2 1 qB + " + 1 + 2 qB 8"

q A + 2"

2

qB + c

q A + 2"

2

qA + c

2

q A + 2"

q A + 2"

2

qA

q A + 2"

2

qA

1 qB 24"2

1 qB 8"2

2" ; we have

1 qB 24"2

1 qB 24"2 1 qB 24"2

2

q A + 2"

qB + " + 2

2

2q A + q B

" +c 1+

1 qB 8"2

q A + 2"

q A + 2"

Therefore, both …rst order conditions cannot hold together when q A > q B > q A 2" and hence there is no Nash equilibrium with 2" > q A q B > 0: The intuition behind the lemma is that if a journal sets its quality cut-o¤ higher than that of the other journal, then it can strictly increase its pro…t by reducing its cut-o¤ marginally. This happens because although the fall in cut-o¤ will cause the average quality of the papers to be published in this journal to fall slightly, however, the total quality of accepted papers increase sharply since the number of papers accepted increases. Moreover, the rise in total quality more than o¤sets the extra cost of publication. With this lemma in hand, we can now proceed to describe the Nash equilibrium in this model. Proposition 1 There exists a unique Nash equilibrium, q A ; q B = (q ; q ) where q = c + 9" : The equilibrium quality cut-o¤ increases with the error in the review process " and the cost parameter c:

7

2

2

:

Proof. From Lemma 1, we know that in a Nash equilibrium, q A = q B = q : Therefore, from (5) and (6) ; we have 3 " 3 q + + c=0 2 6 2 and there is a unique solution to the equation above: " q =c+ : 9

3

(7)

Cut-o¤ on Observed Quality while under Same “Management”

This section tries to work out something akin to the social planner’s problem. However, since I have totally ignored the bene…ts the authors get from publishing a paper and the larger public’s need for “knowledge,”I cannot get a complete social welfare function. Instead of maximizing the social welfare, let the planner choose the quality cut-o¤s for each journal to maximize the sum of their pay-o¤s. Alternatively, this can be thought of as the pay-o¤ maximization exercise of the management when both the journals are under the same management. Note that this is not monopoly or collusion, they are still two separate journals. If a journal rejects a paper, then the author submits it to the other journal. Depending on the observed quality revealed after review and the quality cut-o¤, the second journal accepts or rejects the paper. If it accepts, then the pay-o¤ of the journal changes— increases or decreases depending on the cost of publishing the paper and its quality. Hence, rejection to a paper by a journal has an externality on the pay-o¤ of the other journal. While choosing their pay-o¤ maximizing cut-o¤s on the observed quality of a paper, journals do not consider this externality on the other journal, however, the “management” does. The objective of the “management” is to maximize the total pay-o¤ of the two journals keeping this externality in mind. max (q A ;q B )2(2";1

=

2")

qA; qB

2

qA; qB +

max

(q A ;q B )2(2";1 2")2

8

qB ; qA

The …rst order conditions5 for this maximization problem are given by @

qA; qB @ = 0 and @q A

qA; qB = 0: @q B

(8)

Analogous to Lemma 1 in the previous section, we have the following lemma for the management’s problem. Lemma 2 The “optimum” choice of cut-o¤ s on the observed qualities for both the journals have to be the same, q A = q B : Proof. Similar to the proof of Lemma 1. Proposition 2 There exists a unique quality cut-o¤ for both the journals that maximizes the total pay-o¤ and it is higher than the Nash Equilibrium quality cut-o¤ . The gap between them decreases as the error in the review process decreases. Proof. By Lemma 2, we know that q A = q B = qb; at optimum: Therefore, from (6) and (8) ; we have qb +

" +c=0 3

and there is a unique solution to the equation above: " qb = c + : 3

From equation (7) ; comparing qb and q ; …nd that qb

q =

2" >0 9

and as " ! 0; qb ! q : So, journals, left to themselves, accept too many papers.

4

Conclusion

Using a simple duopoly model, I have shown that how the error in the review process leads journals to choose a lower cut-o¤ on observed quality than the one optimal for them collectively. 5

Second order conditions hold since the pay-o¤ function is strictly concave.

9

Admittedly, the analysis in this paper is restrictive. I assumed that the error term " is same for both the journals across all papers. However, different journals might have di¤erent errors because their referee pools are di¤erent. Moreover, errors might vary author-wise. It has been found that there is a signi…cant di¤erence between the referee reports when the author’s identity is unknown and when it is known (Blank, 1991). This suggests different " for each individual paper. There are some more serious assumptions. I considered only two journals and that too of same rank which is far from reality. I did not consider the bene…ts that an author gets from publishing a paper. The bene…ts of the larger public from the knowledge di¤usion by publishing a paper are not considered as well. The peer review system has its problems, but it keeps a lot of “bad”quality papers out. The journal review process has a large impact on the growth of the economics literature and it has a very signi…cant impact on the growth of knowledge, the design of policy and the careers of the researchers. Hence, it is very important to raise awareness of journal editors and referees. The error in the system might make them compromise on the quality threshold which in turn may increase the number of low quality paper submissions raising the pressure on the referees and thus increasing the error even more. This calls for a reform of the peer review system. This paper provides no blueprint for reform but a basic analytical structure that can be a …rst step towards such a blueprint.

10

References [1] Blank, R. M., 1991, The e¤ects of double-blind versus single-blind reviewing: experimental evidence from the American Economic Review, The American Economic Review 81(5), 1041-1067. [2] Ellison, G., 2002, The slowdown of the economics publishing process, Journal of Political Economy 110(5), 947-993. [3] Gans, J. S. and G. B. Shepherd, 1994, How are the mighty fallen: rejected classic articles by leading economists, The Journal of Economic Perspectives 8(1), 165-179. [4] Hamermesh, D. S., 1994, Facts and myths about refereeing, The Journal of Economic Perspectives 8(1), 153-163. [5] Liebowitz, S. J. and J. P. Palmer, 1984, Assessing the relative impacts of economics journals, Journal of Economic Literature 22(1), 77-88. [6] Moore, W. J., 1972, The relative quality of economics journals: a suggested rating system, Economic Inquiry 10(2), 156-169.

11

Do Journals Accept Too Many Papers?!

The reverse mis$ take of accepting papers that ..... However, since I have totally ignored the benefits the authors get from publishing a paper and the larger ...

105KB Sizes 1 Downloads 220 Views

Recommend Documents

Too Many Tamales.pdf
Page 1 of 5. Page 1 of 5. Page 2 of 5. Page 2 of 5. Page 3 of 5. Page 3 of 5. Too Many Tamales.pdf. Too Many Tamales.pdf. Open. Extract. Open with. Sign In.

We do not know exactly how many
Dec 25, 2012 - So one went to Central Asia, another went to the Middle East and North African region, another went to South America, another stayed right ...

Accept SDK - GitHub
1.2.2 Activating Developer Mode . ..... Microphone access must be allowed by the user the first time the application is run in order to use the audio jack for ...

PAPERS IN SCIENTIFIC JOURNALS J.A. Díaz, M.J. Jiménez. 2017 ...
2016. Energy Procedia. 91C, pp. 1024-1032. DOI: 10.1016/j.egypro.2016.06.271. S. Soutullo ... data analysis from dynamic experimental campaigns part 1: physical aspects”. Energy ... “Evaluating rehabilitation of the social housing envelope:.

Watch Too Many Cooks (1931) Full Movie Online Free ...
Try one of the apps below to open or edit this item. Watch Too Many Cooks (1931) Full Movie Online Free .Mp4_______________.pdf. Watch Too Many Cooks ...

Read Online Too Many Cats (Step into Reading)
... while every armchair broadcaster with a voice recorder app is eager to get in ... World Wide Web listed his top worries for the future of his creation earlier this ...

Too Many Meetings: Communication in Organizations
Two different timelines of events are considered in order to model the firm's lack of commitment. In the first .... The ratio of the two communication precisions is just the square root of the ratio of the cost of ..... In the case when the firm is n

Watch Too Many Millions (1934) Full Movie Online Free ...
Watch Too Many Millions (1934) Full Movie Online Free .Mp4____________.pdf. Watch Too Many Millions (1934) Full Movie Online Free .Mp4____________.

TURN DOWN FOR WHAT ACROSS 1. "Too Many ... -
Pot holder? 2. Village Voice co-founder Norman. 3. Mother and Shaker. 4. Chess player's Elo number, e.g.: Abbr. 5. Gridiron greats. 6. Wedding. 7. Clayey mud.

Accept SDK - GitHub
Accept SDK for iOS is a mPOS solution for Apple devices that enables electronic ... Accept SDK can only run on an armv7 compatible iOS device (iPhone 3GS or.

Preparation of Papers for AIAA Journals
Jul 14, 2011 - [1], require relative positioning with moderate accuracy (about 50 m, 95%). ...... illustration, this paper considers only straight-line flight. ... error, since bearing errors have a pronounced effect on relative positioning accuracy 

Small Changes in Thyroxine Dosage Do Not ... - Oxford Journals
May 2, 2006 - domain of the GHQ-28, which was significantly worse for the ..... Prince RL, Stewart G, Stuckey B, Will RK, Retallack RW, Price RI, Ward L.

Please do not accept unless the cage below is filled ... -
Please tick the appropriate. Purpose. Course Description. Course Code e-Test. Practical Test. UCCA. FIT 101. English for Communication and IT. FIT 201. Examination fee. Fundamentals in Mathematics. FIT 301. Registration Fee. Certificate fee. Other Fe

Social Preferences Under Risk: Why do Opportunity ... - SSRN papers
Page 1 ... under risk that considers decision makers for whom generous behav- ior is driven largely ... Our model considers a setup with a decision maker (DM).

Application for Approval to Accept Outside Employment
I understand and agree to abide by the provisions of the City of Mesquite Outside Employment Policy. I understand that my outside employment must not impair ...

papers
1. INTRODUCTION. I met Robert F. Wagner shortly after I began my career in medical imaging. ... technology of photographic processes. Immediately after .... For disk detection, the best detectability was obtained at very small values of α.17. 3.

papers
I met Robert F. Wagner shortly after I began my career in medical imaging. We bumped into each ... technology of photographic processes. Immediately after the ...

A Few Bad Votes Too Many? Towards Robust Ranking ...
Apr 22, 2008 - ingly, a small fraction of malicious users are trying to “game the system” ... republish, to post on servers or to redistribute to lists, requires prior specific permission ... recent work [3] showed that incorporating user vote in

Part 1: syncope and too-many-solutions Part 2: mutual ...
One of the generalizations that McCarthy addresses is that the syncope constraints can never force stress assignment. In other words, dispreference for vowels in weak positions is never resolved by making the positions strong (this would correspond t

One many, many readings1
Since many shows an obvious analogy to gradable adjectives – it can be put in the compara- tive form ..... by persons comparable to her, e.g., 3-year-old girls.