The replication crisis and the need to change the policy of scientific publication

Document Type : علمی - پژوهشی

Author

Department of philosophy, science and research branch, Islamic Azad university, Tehran, Iran

Abstract

Over the past decade, we have increasingly realized that when we try to replicate the data which are published in influential scientific journals, the initial results are not obtained. In other words, the stated findings of the articles which are published in the experimental sciences, including medicine and psychology, are generally invalid. This phenomenon has led to a crisis in the experimental sciences, known in 2012 as the "replication crisis." In the present article, first, the crisis is introduced and a brief account of the opinions of the scientists about the severity of the crisis is produced. Then, five possible causes for its occurrence are introduced. The claim of the article is that the most important cause for the crisis of reproducibility of scientific results of academic articles should be sought in the wrong policies of the academic institute or funding bodies of putting pressure on scientists to publish articles (publish or perish). In the final section, some solutions, corresponding to the five causes which are identified, to alleviate the crisis are presented . Two of these solutions, namely, expanding the domain of the “open science” and secondly, reducing the role of articles in assessing the competence of researchers and academics are introduced as the main strategies to alleviate the crisis.

Keywords

References

  1.  

     

    1. Aitkenhead, Decca (2013), “Peter Higgs: I wouldn't be productive enough for today's academic system”, The Guardian, 6 December.
    2. Ariely, D. (2008), Predictably Irrational: The Hidden Forces That Shape Our Decisions, Harper Collins.
    3. Bar-Hillel, M., (1980), “The base-rate fallacy in probability judgments”, Acta Psychologica, 44(3), pp. 211-233.
    4. Beard, C. (2011), “Cognitive bias modification for anxiety: current evidence and future directions”, Expert Review of Neurotherapeutics, 11(2), 299-311.
    5. Benjamin, Daniel J.; et al. (2018), “Redefine statistical significance”, Nature Human Behaviour. 2 (1): 6–10.
    6. Boyd, Nora Mills and James Bogen, “Theory and Observation in Science”, The Stanford Encyclopedia of Philosophy (Fall 2021 Edition), Edward N. Zalta (ed.), forthcoming URL = <https://plato.stanford.edu/archives/fall2021/entries/science-theory-observation/>
    7. Collaboration, Open Science (2012), “An Open, Large-Scale, Collaborative Effort to Estimate the Reproducibility of Psychological Science”, Perspectives on Psychological Science, 7 (6): 657–660.
    8. Collins, H., (1985), Changing Order: Replication and Induction in Scientific Practice, London: Sage Publications.

    6.     Bird, alexander (2020), “understanding the replication crisis as a base rate fallacy”, the british journal for the philosophy of science.

    10. D. J. Witherspoon et-al (2007), “Genetic Similarities Within and Between Human Populations”, Genetics 176(1): 351–359.

    11. Denworth, Lydia (2019), “A Significant Problem: Standard scientific methods are under fire. Will anything change? ”, Scientific American, Vol. 321, No. 4, pp. 62–67.

    12. Evidence-Based Medicine Working Group (1992), “Evidence-based medicine. A new approach to teaching the practice of medicine”, JAMA, 268 (17): 2420–25.

    13. Fang FC, Casadevall A. (2015), “Competitive science: is competition ruining science? Infect Immun”, 83: 1229 –1233.

    14. Feyerabend, P.K., (1959), “An Attempt at a Realistic Interpretation of Expeience”, in P.K. Feyerabend, Realism, Rationalism, and Scientific Method (Philosophical Papers I), Cambridge: Cambridge University Press, 1985, pp. 17–36.

    15. Geoffrey S. Ginsburg, Huntington Ph.D Willard (2009), Essentials of Genomic and Personalized Medicine, Academic Press.

    16. Gigerenzer, G. (2006), “Bounded and Rational”, Contemporary Debates in Cognitive Science, R. J. Stainton, Blackwell Publishing: 115–133.

    17. Hanson, N.R., (1958), Patterns of Discovery, Cambridge, Cambridge University Press.

    18. Ioannidis, John P. A. (August 1, 2005), “Why Most Published Research Findings Are False”, PLOS Medicine. 2 (8): e124.

    19. Jimenez, A, Mesoudi A., (2019), “Prestige-biased social learning: current evidence and outstanding questions”, Palgrave Communications, 5(1).

    20. Johnson, V. E., Payne, R. D., Wang, T., Asher, A. and Mandal, S. (2017), “On the Reproducibility of Psychological Science”, Journal of the American Statistical Association, 112, pp. 1–10.

    21. Kahneman, D. and Tversky, A., (1972), “Subjective probability: A judgment of representativeness”, Cognitive psychology, 3(3), pp.430-454.

    22. Kerr, N. L. (1998), “HARKing: Hypothesizing after the results are known”, Personality and Social Psychology Review, 2 (3): 196–217.

    23. Kuhn, T.S. (1962), The Structure of Scientific Revolutions, Chicago: University of Chicago Press, reprinted 1996.

    24. Lakens, Daniel; et al. (March 2018), “Justify your alpha”, Nature Human Behaviour, 2 (3): 168–171.

    25. Lee, C.J.; Sugimoto, C.R.; Zhang, G.; Cronin, B. (2013), “Bias in peer review”, Journal of the Association for Information Science and Technology, 64: 2–17

    26. Lorenz, Edward N. (March 1963), “Deterministic Nonperiodic Flow”, Journal of the Atmospheric Sciences. 20 (2): 130–141.

    27. Martin, G. N.; Clarke, Richard M. (2017), "Are Psychology Journals Anti-replication? A Snapshot of Editorial Practices", Frontiers in Psychology, 8: 523.

    28. Mirowski, P. (2011), Science-Mart: Privatizing American Science, Harvard University Press.

    29. Münsterberg, H. (1899), “Psychology and history”, Psychological Review, 6, 1-31.

    30. Nature Video (28 May 2016), “Is There a Reproducibility Crisis in Science?”, Scientific American, https://www.scientificamerican.com/video/is-there-a-reproducibility-crisis-in-science/.

    31. Neill, U. S. (2008), “Publish or perish, but at what cost?”, Journal of Clinical Investigation, 118 (7): 2368.

    32. Neill, U. S. (2008), "Publish or perish, but at what cost?", Journal of Clinical Investigation. 118 (7): 2368.

    33. Orrell, David (2012), Truth or Beauty: Science and the Quest for Order, New Haven: Yale University Press.

    34. Orrell, David; Smith, Leonard; Barkmeijer, Jan; Palmer, Tim (2001), “Model error in weather forecasting”, Nonlinear Processes in Geophysics, 9 (6): 357–371.

    35. Osmo Kivinen, Juha Hedman, Kalle Artukka, (2017), “Scientific publishing and global university rankings. How well are top publishing universities recognized?”, Scientometrics, Vol.112, pp. 679-695.

    36. Oswald, Margit E.; Grosjean, Stefan (2004), “Confirmation bias”, in Pohl, Rüdiger F. (ed.), Cognitive illusions: A handbook on fallacies and biases in thinking, judgement and memory, Hove, UK: Psychology Press, pp. 79–96.

    37. “Peer reviewers unmasked: largest global survey reveals trends”, (7 Sep 2017), Nature News available at https://www.nature.com/articles/d41586-018-06602-y.

    38. Pickering, A. (1986, January), Against correspondence: A constructivist view of experiment and the real. In PSA: Proceedings of the Biennial Meeting of the Philosophy of Science Association (Vol. 1986, No. 2, pp. 196-206). Philosophy of Science Association.

    39. Price, Derek J. de Solla (1963), Little science, big science, New York: Columbia University Press.

    40. Rouse, Joseph (1998), “Heideggerian philosophy of science”, Rutledge Encyclopedia of Philosophy, Taylor and Francis.

    41. Sherif, Muzafer; Taub, Daniel; Hovland, Carl I. (1958), “Assimilation and contrast effects of anchoring stimuli on judgments”, Journal of Experimental Psychology, 55 (2): 150–155.

    42. Song, F.; Parekh, S.; Hooper, L.; Loke, Y. K.; Ryder, J.; Sutton, A. J.; Hing, C.; Kwok, C. S.; Pang, C.; Harvey, I. (2010), “Dissemination and publication of research findings: An updated review of related biases”, Health Technology Assessment (Winchester, England). 14 (8): iii, iix–xi, iix–193.

    43. Stagge, James H.; Rosenberg, David E.; Abdallah, Adel M.; Akbar, Hadia; Attallah, Nour A.; James, Ryan (2019-02-26), “Assessing data availability and research reproducibility in hydrology and water resources”, Scientific Data, 6: 190030.

    44. Stanley, T. D.; Carter, Evan C.; Doucouliagos, Hristos (2018), “What meta-analyses reveal about the replicability of psychological research”, Psychological Bulletin, 144 (12): 1325–1346.

    45. Stigler, Stephen M. (1986), The History of Statistics: The Measurement of Uncertainty Before 1900, Harvard University Press. 

    46. Wallot, Sebastian; Kelty-Stephen, Damian G. (2018), “Interaction-Dominant Causation in Mind and Brain, and Its Implication for Questions of Generalization and Replication”, Minds and Machines, 28 (2): 353–374.

    47. Wason, Peter C. (1960), “On the failure to eliminate hypotheses in a conceptual task”, Quarterly Journal of Experimental Psychology, 12 (3): 129–140.

    48. Weir, Kristen (2011), “A reproducibility crisis?”, American Psychological Association.

     

     

Files

Share

How to cite

Statistics