To ArXiv or not to ArXiv: A Study Quantifying Pros and Cons of Posting Preprints Online
Abstract: Double-blind conferences have engaged in debates over whether to allow authors to post their papers online on arXiv or elsewhere during the review process. Independently, some authors of research papers face the dilemma of whether to put their papers on arXiv due to its pros and cons. We conduct a study to substantiate this debate and dilemma via quantitative measurements. Specifically, we conducted surveys of reviewers in two top-tier double-blind computer science conferences—ICML 2021 (5361 submissions and 4699 reviewers) and EC 2021 (498 submissions and 190 reviewers). Our two main findings are as follows. First, more than a third of the reviewers self-report searching online for a paper they are assigned to review. Second, outside the review process, we find that preprints from better-ranked affiliations see a weakly higher visibility, with a correlation of 0.06 in ICML and 0.05 in EC. In particular, papers associated with the top-10-ranked affiliations had a visibility of approximately 11% in ICML and 22% in EC, whereas the remaining papers had a visibilityof 7% and 18% respectively.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: Changes are highlighted in the manuscript in blue.
1. We add key motivations and key takeaways of our research in the introduction section, to clarify the framing of the paper.
2. We clarify the research question in Q2, and clearly state the causal part of the question.
3. We provide the graphical causal model assumed in our setting and provide an explanation for it in Section 3.3.1.
4. We relate our analysis approach to the assumed causal model in sections 3.3.2 and 3.3.3.
5. We relate our results to the causal model assumed in our setting in Section 4.
6. In Section 4, we describe general data-based trends in authors' posting behavior and the visibility gained by preprints.
7. In Section 5, we make updates to provide clear actionable takeaways for authors
Assigned Action Editor: ~Laurent_Charlin1
Submission Number: 551
Loading