Survivorship Bias in Data Analysis and Interpretation and Strategies for Reducing It

Extended Abstract
 
Introduction and Objectives: Survivorship bias (sūgīrī-yi bāzmāndigī) is an important cognitive bias that affects the interpretation and analysis of data and the decision-making process, causing a person to reach decisions that may not be correct or fail due to ignoring some important information. Recognizing and understanding this bias can help a person perform better and more qualitatively in the decision-making process and, as an outcome, achieve better results.
Survivorship bias means deviation in data interpretation due to ignoring information that was not considered in data analysis and review due to its absence in the data. This bias is due to focusing on data that has been successful and ignoring data that has been unsuccessful.
Despite the importance of survivorship bias in data analysis and interpretation and conclusions, unfortunately, this type of bias has not been considered in research methods under the title of “research biases” (sūgīrīhā-yi pizhūhishī) and also as one of the cognitive biases associated with researchers that affect their research findings. Most analysis, interpretation, and conclusions have been carried out by considering only successful samples without considering failed samples. Of course, this warning was given in Ioannidis’s exciting article in 2019 titled “Why Most Published Research Findings Are Incorrect,” but it has not been heeded. Therefore, explaining the aforementioned bias and its undesirable consequences, as well as providing solutions to prevent or reduce it, are among the goals of this research.
Method: In this study, a descriptive-analytical method (ravish-i tawṣīfī-taḥlīlī) is used to investigate and explain. In order to collect information, the library method was used, and articles from various scientific databases and existing books were reviewed and studied.
In this research, first, a detailed examination of survivorship bias and its role in data analysis and interpretation deviation was conducted, including the types of cognitive bias categories and the nature of survivorship bias’s placement in these categories. Then, the findings were explained and a summary was provided.
This study, due to the neglect of survivorship bias in research methods, comprehensively addresses this bias and provides new insights in this area. With this approach, the present study can be a reliable source for researchers to protect their research findings from survivorship bias, which adds to the validity of their findings.
Results: Survivorship bias distorts research findings and leads to unreliable results. This type of bias is problematic because it leaves out important research data and threatens external validity (ravāyī-yi bīrūnī). In fact, the external validity of the findings is compromised by the inability to accurately represent society. Furthermore, this bias usually leads to overly optimistic and successful conclusions. Survivorship bias can lead researchers to draw incorrect conclusions because the observed data is incomplete.
Discussion and Conclusion: Cognitive bias (sūgīrī-yi shinākhtī) means a deviation in the process of recognizing and interpreting information that occurs due to the assumptions and beliefs that an individual has. Survivorship bias, as one of the important cognitive biases, occurs when a person only considers the surviving observations without paying attention to the data points that were “not left out” in the event. Survivorship bias causes only living or successful samples to be examined, which leads to errors in judgment or conclusions. Survivorship bias means focusing on cases that have passed a selection stage and ignoring others. This bias addresses our tendency to gain useful information from successes and ignore similar failures.
Survivorship bias acts like a mental trap (talah-ʾi ẕihnī), leading us to make false perceptions and wrong decisions. By focusing on successes and ignoring failures, we paint an incomplete and exaggerated picture of reality. Therefore, one of the main concerns of this research is how to reduce or prevent survivorship bias due to its undesirable consequences.
To reduce or prevent survivorship bias, before anything else, one must recognize and become aware of these cognitive biases. After recognition, various strategies can be used to reduce or prevent survivorship bias, such as:

Identifying failed data (shināsāyī-yi dādihā-yi nākām)
Analyzing complete data (taḥlīl-i dādihā-yi kāmil)
Consulting with competent individuals (mushāvirih bā afrād-i dhī al-ṣalāḥ)
Studying data randomly (muṭāliʿih-ʾi dādihā bih ṣūrat-i taṣādufī)
Accepting failure as a tool for learning (pazīriftan-i shikast bih ʿunvān-i abzārī barā-yi yādgīrī)
Challenging assumptions (bih chālish kishīdan-i farżiyāt)
Examining market data (barrasī-yi dādihā-yi bāzār)
Properly designing a research protocol (ṭarāḥī-yi ṣaḥīḥ-i prūtukul-i taḥqīqātī)
Employing random or stratified sampling (bih kār gīrī-yi namūnihgīrī-yi taṣādufī yā ṭabaqihʾī)
Using mathematical models and methods such as neural network models (istifādih az mudilhā va ravishhā-yi riyāḍī hamchūn mudilhā-yi shabakahā-yi ʿaṣabī)

Acknowledgement: I would like to thank the professors at the University of Tehran for providing constructive feedback.
Conflict of Interest: The authors declare that there is no conflict of interest related to this research.