Articles freely available online are more highly cited. For greater impact and faster scientific progress, authors and publishers should aim to make research easy to access.
The volume of scientific literature typically far exceeds the ability of scientists to identify and utilize all relevant information in their research. Improvements to the accessibility of scientific literature, allowing scientists to locate more relevant research within a given time, have the potential to dramatically improve communication and progress in science. With the web, scientists now have very convenient access to an increasing amount of literature that previously required trips to the library, inter-library loan delays, or substantial effort in locating the source. Evidence shows that usage increases when access is more convenient , and maximizing the usage of the scientific record benefits all of society.
Although availability varies greatly by discipline, over a million research articles are freely available on the web. Some journals and conferences provide free access online, others allow authors to post articles on the web, and others allow authors to purchase the right to post their articles on the web.
In this article we investigate the impact of free online availability by analyzing citation rates. We do not discuss methods of creating free online availability, such as time-delayed release or publication/membership/conference charges. Online availability of an article may not be expected to greatly improve access and impact by itself. For example, efficient means of locating articles via web search engines or specialized search services is required, and a substantial percentage of the literature needs to be indexed by these search services before it is worthwhile for many scientists to use them. Computer science is a forerunner in web availability -- a substantial percentage of the literature is online and available through search engines such as Google (google.com), or specialized services such as ResearchIndex  (researchindex.org). Even so, the greatest impact of the online availability of computer science literature is likely yet to come, because comprehensive search services and more powerful search methods have only become available recently.
We analyzed 119,924 conference articles in computer science and related disciplines, obtained from DBLP (dblp.uni-trier.de). In computer science, conference articles are typically formal publications and are often more prestigious than journal articles, with acceptance rates at some conferences below 10%. Citation counts and online availability were estimated using ResearchIndex. The analysis excludes self-citations, where a citation is considered to be a self-citation if one or more of the citing and cited authors match.
Figure 1 shows the probability that an article is freely available online as a function of the number of citations to the article, and the year of publication of the article. The results are dramatic. There is a clear correlation between the number of times an article is cited, and the probability that the article is online. More highly cited articles, and more recent articles, are significantly more likely to be online.
Figure 1: Analysis of 119,924 conference articles in computer science and related disciplines. More highly cited articles, and more recent articles, are substantially more likely to be freely available on the web. The actual percentage of articles available online is greater due to limitations in the extraction of article information from online documents, and limitations in locating articles on the web. Only points with greater than 100 articles are computed.
The mean number of citations to offline articles is 2.74, and the mean number of citations to online articles is 7.03, or 2.6 times greater than the number for offline articles. These numbers mask variations over time -- in particular, older articles have more citations on average, and older articles are less likely to be online. When considering articles within each year, and averaging across all years from 1990 to 2000, we find that online articles are cited 4.5 times more often than offline articles.
We also analyzed differences within each publication venue, where multiple years for the same conference are considered as separate venues. We computed the percentage increase in the average number of citations to online articles compared to offline articles. When offline articles were more highly cited, we used the negative of the percentage increase for offline articles. For example, if the average number of citations for offline articles is 2, and the average for online articles is 4, the percentage increase would be 100%. For the opposite situation, the percentage increase would be -100%. Figure 2 shows the results. Averaging the percentage increase across 1,494 venues containing at least five offline and five online articles results in an average of 336% more citations to online articles compared to offline articles published in the same venue [the first, second (median), and third quartiles of the distribution are 58%, 158%, and 361%].
Figure 2: Analysis of citation rates within publication venues. The graph shows the distribution of the percentage increase for the average number of citations to online articles compared to offline articles. The analysis covers 1,494 publication venues containing at least 5 online and 5 offline articles. For 90% of venues, online articles are more highly cited on average. On average there are 336% more citations to online articles compared to offline articles published in the same venue [the first, second (median), and third quartiles of the distribution are 58%, 158%, and 361%].
The preceding data does not allow us to make conclusions as to the cause of the correlation between high citation rates and online availability. Online articles may be more highly cited because they are easier to access and thus more visible and more likely to be read, or because higher quality articles are more likely to be made available online. Intuitively, it seems likely that the easier availability and improved visibility of online articles plays a significant role. If we assume that articles published in the same venue are of similar quality, then the analysis by venue suggests that online articles are more highly cited because of their easier availability. This assumption is likely to be more valid for top-tier conferences with very high acceptance standards. Restricting the above analysis to the top publication venues by average citation rate results in a similarly dramatic increase in citation rates for online articles. For example, when restricting to the top 20 venues, the average increase in the citation rate for online articles is 286% [the first, second (median), and third quartiles of the distribution are 66%, 284%, and 471%].
Free online availability facilitates access in multiple ways, including online archives, direct connections between scientists or research groups, hassle-free links from email, discussion groups, and other services, indexing by web search engines, and the creation of third-party search services. Free online availability of scientific literature offers substantial benefits to science and society. To maximize impact, minimize redundancy, and speed scientific progress, author and publishers should aim to make research easy to access.
Thanks to Gary Flake, Andrew Odlyzko, and David Pennock for useful comments and suggestions.
1. Steve Lawrence, C. Lee Giles, Kurt Bollacker, "Digital Libraries and Autonomous Citation Indexing," IEEE Computer, 32(6):67-71, 1999. ResearchIndex
2. A. Odlyzko, "The Rapid Evolution of Scholarly Communication," Learned Publishing, to appear, 2001. ResearchIndex
1. Caplan MS. Neonatal necrotizing enterocolitis [Introduction] Semin Perinatol. 2008;32(2):69.[PubMed]
2. Neu J, Mshvildadze M, Mai V. A roadmap for understanding and preventing necrotizing enterocolitis. Curr Gastroenterol Rep. 2008;10(5):450–457.[PubMed]
3. Luig M, Lui K NSW ACT NICUS Group. Epidemiology of necrotizing enterocolitis—part II: risks and susceptibility of premature infants during the surfactant era: a regional study. J Paediatr Child Health. 2005;41(4):174–179.[PubMed]
4. Gagliardi L, Bellu R, Cardilli V, De Curtis M Network Neonatale Lombardo. Necrotising enterocolitis in very low birth weight infants in Italy: incidence and non-nutritional risk factors. J Pediatr Gastroenterol Nutr. 2008;47(2):206–210.[PubMed]
5. Patole SK. Prevention and treatment of necrotizing enterocolitis in preterm neonates. Early Hum Dev. 2007;83:635–642.[PubMed]
6. Christensen RD, Gordon PV, Besner GE. Can we cut the incidence of necrotizing enterocolitis in half—today? Fetal Pediatr Pathol. 2010;29:185–198.[PubMed]
7. Hentschel J, de Veer I, Gastmeier P, Ruden H, Obladen M. Neonatal nosocomial infection surveillance: incidences by site and a cluster of necrotizing enterocolitis. Infection. 1999;27(4/5):234–238.[PubMed]
8. Boccia D, Stolfi I, Lana S, Moro ML. Nosocomial necrotizing enterocolitis outbreaks: epidemiology and control measures. Eur J Pediatr. 2001;160:385–391.[PubMed]
9. Bisquera JA, Cooper TR, Berseth CL. Impact of necrotizing enterocolitis on length of stay and hospital charges in very low birth weight infants. Pediatrics. 2002;109(3):423–428.[PubMed]
10. McElhinney DB, Hedrick HL, Bush DM, et al. Necrotizing enterocolitis in neonates with congenital heart disease: risk factors and outcomes. Pediatrics. 2000;106(5):1080–1087.[PubMed]
11. Lambert DK, Christensen RD, Henry E, et al. Necrotizing enterocolitis in term neonates: data from a multi-hospital health-care system. J Perinatol. 2007;27:437–443.[PubMed]
12. Maayan-Metzger A, Itzchak A, Mazkereth R, Kuint J. Necrotizing enterocolitis in full-term infants: case-control study and review of the literature. J Perinatol. 2004;24(8):494–499.[PubMed]
13. Manogura AC, Turan O, Kush ML, et al. Predictors of necrotizing enterocolitis in preterm growth-restricted neonates. Am J Obstetr Gynecol. 2008;198(6):638.e1–638.e5.[PubMed]
14. Stout G, Lambert DK, Baer VL, et al. Necrotizing enterocolitis during the first week of life: a multicentered case-control and cohort comparison study. J Perinatol. 2008;28(8):556–560.[PubMed]
15. Ghidini A, Espada RA, Spong CY. Does exposure to magnesium sulfate in utero decrease the risk of necrotizing enterocolitis in premature infants? Acta Obstet Gynecol Scand. 2001;80(2):126–129.[PubMed]
16. Hand IL, Noble L, McVeigh TJ, Kim M, Yoon JJ. The effects of intrauterine cocaine exposure on the respiratory status of the very low birth weight infant. J Perinatol. 2001;21(6):372–375.[PubMed]
17. Bashiri A, Zmora E, Sheiner E, Hershkovitz R, Shoham-Vardi I, Mazor M. Maternal hypertensive disorders are an independent risk factor for the development of necrotizing enterocolitis in very low birth weight infants. Fetal Diagn Ther. 2003;18(6):404–407.[PubMed]
18. Ogunyemi D, Murillo M, Jackson U, Hunter N, Alperson B. The relationship between placental histopathology findings and perinatal outcome in preterm infants. J Mat-Fetal Neonatal Med. 2003;13(2):102–109.[PubMed]
19. Ruangtrakool R, Laohapensang M, Sathornkich C, Talalak P. Necrotizing enterocolitis: a comparison between full-term and pre-term neonates. J Med Assoc Thail. 2001;84(3):323–331.[PubMed]
20. Desfrere L, de Oliveira I, Goffinet F, et al. Increased incidence of necrotizing enterocolitis in premature infants born to HIV-positive mothers. AIDS. 2005;19(14):1487–1493.[PubMed]
21. Gregory KE. Clinical predictors of necrotizing enterocolitis in premature infants. Nurs Res. 2008;57(4):260–270.[PubMed]
22. Kamoji VM, Dorling JS, Manktelow B, Draper ES, Field DJ. Antenatal umbilical Doppler abnormalities: an independent risk factor for early onset neonatal necrotizing enterocolitis in premature infants. Acta Paediatr. 2008;97(3):327–331.[PubMed]
23. Dollberg S, Lusky A, Reichman B. Patent ductus arteriosus, indomethacin and necrotizing enterocolitis in very low birth weight infants: a population-based study. J Pediatr Gastroenterol Nutr. 2005;40(2):184–188.[PubMed]
24. Bertino E, Giuliani F, Prandi G, Coscia A, Martano C, Fabris C. Necrotizing enterocolitis: risk factor analysis and role of gastric residuals in very low birth weight infants. J Pediatr Gastroenterol Nutr. 2009;48(4):437–442.[PubMed]
25. Necrotizing Enterocolitis Guideline Team, Cincinnati Children’s Hospital Medical Center. Pediatric Evidence-Based Care Guidelines. Vol. 28. Cincinnati, OH: Cincinnati Children’s Hospital Medical Center Guideline; 2010. Evidence-Based Care Guideline for Necrotizing Enterocolitis Among Very Low Birth Weight Infants; pp. 1–10.
26. Ohlsson A, Walia R, Shah S. Ibuprofen for the treatment of patent ductus arteriosus in preterm and/or low birthweight infants. Cochrane Database Syst Rev. 2010;(4):CD003481.[PubMed]
27. Patole S, McGlone L, Muller R. Virtual elimination of necrotizing enterocolitis for 5 years—reasons? Med Hypothesis. 2003;61(5/6):617–622.[PubMed]
28. Carter BM, Holditch-Davis D. Risk factors for necrotizing enterocolitis in preterm infants: how race, gender, and health status contribute. Adv Neonat Care. 2008;8(5):285–290.[PMC free article][PubMed]
29. Claud EC, Walker WA. Bacterial colonization, probiotics, and necrotizing enterocolitis. J Clin Gastroenterol. 2008;42(suppl 2):S46–S52.[PubMed]
30. Mehall JR, Kite CA, Saltzman DA, Wallett T, Jackson RJ, Smith SD. Prospective study of the incidence and complications of bacterial contamination of enteral feeding in neonates. J Pediatr Surg. 2002;37(8):1177–1182.[PubMed]
31. Gantz M, Roy J, Guillet R. Analyzing retrospective data with time-varying exposure: a cautionary tale of H2 blockers in ELBW neonates. Am J Perinatol. 2008;25:93–100.[PubMed]
32. National Institute of Child Health and Human Development. [Accessed June 22, 2011];Common reflux treatment linked to life threatening bowel infection in premature infants. http//: www.nih.gov/news/pr/feb2006/nichd-08.htm. Published February 8, 2006.
33. Christensen RD, Lambert DK, Henry E, et al. Is “transfusion-associated necrotizing enterocolitis” an authentic pathogenic entity? Transfusion. 2010;50(5):1106–1112.[PubMed]
34. Josephson CD, Wesolowski A, Bao G, et al. Do red cell transfusions increase the risk of necrotizing ente rocolitis in premature infants? J Pediatr. 2010;157(6):972, 978.e1–3.[PMC free article][PubMed]
35. Ruangtrakool R, Laohapensang M, Sathornkich C, Talalak P. Necrotizing enterocolitis: a comparison between full-term and pre-term neonates. J Med Assoc Thailand. 2001;84(3):323–331.[PubMed]
36. Figueras-Aloy J, Rodriguez-Miguelez JM, Iriondo-Sanz M, Salvia-Roiges MD, Botet-Mussons F, Carbonell-Estrany X. Intravenous immunoglobulin and necrotizing enterocolitis in newborns with hemolytic disease. Pediatrics. 2010;125(1):139–144.[PubMed]
37. Ververidis M, Kiely EM, Spitz L, Drake DP, Eaton S, Pierro A. The clinical significance of thrombocytopenia in neonates with necrotizing enterocolitis. J Pediatr Surg. 2001;36(5):799–803.[PubMed]
38. Tepas JJ, Sharma R, Hudak ML, Garrison RD, Pieper P. Coming full circle: an evidence-based definition of the timing and type of surgical management of very low-birth-weight (<1000 g) infants with signs of acute intestinal perforation. J Ped Surg. 2006;41:418–422.[PubMed]
39. Chauhan M, Henderson G, McGuire W. Enteral feeding for very low birth weight infants: reducing the risk of necrotising enterocolitis. Arch Dis Child Fetal Neonatal Ed. 2008;93(2):F162–F166.[PubMed]
40. Schurr P, Perkins EM. The relationship between feeding and necrotizing enterocolitis in very low birth weight infants. Neonatal Network J Neonatal Nurs. 2008;27(6):397–407.[PubMed]
41. Lucas A, Cole TJ. Breast milk and neonatal necrotising enterocolitis. Lancet. 1990;336(8730):1519–1523.[PubMed]
42. Berseth CL, Bisquera JA, Paje VU. Prolonging small feeding volumes early in life decreases the incidence of necrotizing enterocolitis in very low birth weight infants. Pediatrics. 2003;111(3):529–534.[PubMed]
43. Morgan J, Young L, McGuire W. Slow advancement of enteral feed volumes to prevent necrotising enterocolitis in very low birth weight infants. Cochrane Database Syst Rev. 2011;(3):CD001241.[PubMed]
44. Krishnamurthy S, Gupta P, Debnath S, Gomber S. Slow versus rapid enteral feeding advancement in preterm infants 1000–1499 g: a randomized controlled trial. Acta Paediatr. 2010;99(1):42–46.[PubMed]
45. Hughes C, Dowling R. Speed of onset of adaptive mucosal hypoplasia and hypofunction in the intestine of parenterally fed rats. Clin Sci. 1980;59:317–327.[PubMed]
46. Stoll BJ, Hansen N, Fanaroff A, et al. Late-onset sepsis in very low birth weight neonates: the experience of the NICHD Neonatal Research Network. Pediatrics. 2008;110:285–291.[PubMed]
47. Flidel-Ramon O, Friedman S, Lev E, Juster-Reicher A, Amitay M, Shinwell E. Early enteral feeding and nosocomial sepsis in very low birthweight infants. Arch Dis Child, Fetal Neonatal Ed. 2004;89:F289–F292.[PMC free article][PubMed]