Pride and Prejudice in Academia

Pride comes before a fall. This phrase is a good summary of the reproducibility crisis in psychological research (or credibility revolution as some authors prefer to call it, e.g., Vazire, 2018), although this problem affects many other disciplines. Studying at the University of Amsterdam, and especially in the Research Master’s Psychology, we are continuously confronted with analyses and recommendations tackling this rather concerning issue. The course “Good Research Practices” (Sarafoglou et al., 2020) introduces the main events, problems, voices, and proposed solutions that were raised in the context of the crisis. Alongside the academic curriculum, projects such as the Student Initiative for Open Science (SIOS) similarly fight for open science and reproducibility. Equipped with a critical eye and lots of tools to navigate this crisis, we are then sent out into the research world to do better science, to be the generation that does it right. How has it come to this? 

The science community has rules and conventions which govern the interactions within the system, just like any other congregation of living beings. On a more individual level, the people who conduct science add their own motivations and aspirations to the equation. In the following, I argue how pride and prejudice in the system have shaped the science community and simultaneously failed its main undertaking: advancing knowledge.

Pride is entrenched in every university as an institution and as a symbol – ranging from proud parents when their child graduates to rankings which seek to determine where one can receive the best education. No wonder rankings are a source of pride, especially for universities that rank highly. They should motivate institutions to continuously improve the quality of research and education. Instead, we observe a self-perpetuating system of prestigious names which acquire grants to conduct research and attract excellent students who themselves turn into big names.

Speaking of big names: next to the obsession with university rankings, the science community has also created measures and indices for, well, anything one could be proud of. Journals can brag about their impact factor (again attracting the big names) while individual researchers might have a quick look at their own number of citations or their h-index, a count for citations in relation to the number of publications, every now and then (to check if they are already considered a big name). Such indices are supposed to inform the community about the importance of a journal or the productivity and success of a scholar. They are related to the allocation of resources such as grants and jobs at high-ranking universities (Bornmann & Daniel, 2007), and having a big name might even influence how peer reviewers evaluate a manuscript – showing how status and bias intertwine (Huber et al., 2022). Unfortunately, this great source of pride is not without prejudice: Indices are prone to biases and distortions. For example, researchers in less popular disciplines and niches simply cannot be cited by as many people as researchers in disciplines with many scholars, consequently influencing the h-index (Alonso et al., 2009). 

Similarly, researchers-to-be are likely to get their share of pride through academic validation, again linking the individual student to the institutional level. In order to be one of the few to study at the big-name universities, students have to show excellent performance on another index – grades. But an emphasis on grades can nudge students towards a so-called performance goal orientation, where they study to obtain good grades, instead of adopting a learning goal orientation, where they would value their learning curve more than the resulting grade (Elliott & Dweck, 1988). This is especially harmful because it creates an unsatisfying and frustrating learning environment and is also less successful in the long-term because it negatively influences student’s motivation and beliefs about their own abilities (Lu et al., 2022; Wolters et al., 1996).

Striving for recognition itself is not a problem, obviously, yet it shapes how the science community conducts education and research. It creates a competitive setting in which fast science with flashy results is rewarded while more substantial indicators of quality are ignored. In the fight for big names, citations, grants, and simply a steady job, (future) researchers cannot escape this machinery of publish or perish. So they publish. 

While the pride-generating and -perpetuating aspects of science affect probably everyone in the system, prejudice further shapes the setting for specific subgroups. To list only two of many examples, racism and sexism are a reality in the ostensibly merit-focused science community (Calaza et al., 2021). These prejudices often work in subtle but impactful ways, such as resulting in the underrepresentation of women in science (Ceci et al., 2014; Serio, 2016). A recent debate in the psychological publishing world has once again directed the community’s attention towards racism and the lack of diversity in editing and publishing, but also concerning research questions and samples (Roberts, 2022). 

Apart from systemic racism and sexism, prejudice in the broader sense also finds its way into the work of researchers: Biases impact how we formulate hypotheses, test them, and interpret the results. In his book, Chambers (2017) identifies seven “deadly” sins of psychology which describe research practices in psychology that have created and still reinforce the status quo. For example, the author elaborates on the confirmation bias, wherein researchers tend to focus on confirming their hypotheses instead of trying to falsify them. In combination with the publication bias that favours statistically significant and highly interesting findings (whatever that means) over null findings; this results in a cherry-picked landscape of research findings serving as our “neutral” basis of knowledge, which we use to make decisions in all areas of life. 

In addition to this, biases can influence our work in ways we cannot control or be conscious of. Although a researcher might be convinced of their correct behaviour in all areas of hypotheses formulating, data collection and analysis, some researcher degrees of freedom can still lead to a misrepresentation of the true findings. Vague hypotheses, changing or adding a hypothesis after the results are known (HARKing), and selectively searching for bugs in code that revealed unwanted results are just a few examples of how biases might unwantedly creep in when conducting research. Less unconscious on the other hand is fraudulent behaviour such as faking data, however, examples of such serious academic fraud are fortunately few. More common is a questionable research practice called p-hacking: Researchers “torture” the data until the p-value falls below the conventional .05 for statistical significance, for example, by excluding outliers that would render the result non-significant.

Having brushed over some observations, studies, and aspects of both pride and prejudice in academia, I want to conclude on a more positive note: There is a lot of room for improvement, and it is happening. Especially in the past ten years, the awareness of the problems has increased tremendously and countless plans for action have been proposed and many also realised. For example, the Centre for Open Science has created the Open Science Framework, facilitating preregistration and data sharing. Both practices are essential in the fight for reproducible, transparent research and help researchers to combat their own biases, clearly distinguish exploratory from confirmatory analyses, and slowly change the status quo. 

One step further is the format of registered reports, where proposals are peer reviewed and obtain in-principle acceptance for publication before actually conducting the research. Vetting the theory and methods can prevent wasting precious resources on a flawed or underpowered research design, and as long as the authors adhere to their proposed methods, their article will be published regardless of the outcome. This is one way of ensuring that the research landscape is not cherry-picked (entirely) by the journals but actually reflects the spectrum of research and knowledge.

In addition to this, the so-called TOP (transparency and openness promotion) guidelines help authors judge the endorsement of such practices by journals, and in return, journals can offer badges to reward open science practices. Replication projects seeking to separate the wheat from the chaff and new ideas of conducting research receive more attention, e.g., many-lab studies or adversarial collaborations. Especially the latter illustrates how pride and prejudice can be overcome in a concrete setting: Researchers of opposing views coordinate a study together, aimed at resolving the scientific dispute. It forces them to understand each other’s viewpoint, agree and commit to a specific research design and the results (e.g., Kahneman and Hertwig about conjunction effects in Mellers et al., 2001). And, of course, education and student initiatives concern themselves more and more with making open science practices the norm. 

Disentangling pride and prejudice from science and academia is undoubtedly a highly complex undertaking, although increased awareness, new ideas and approaches, and collective effort are excellent prerequisites for the scientific community to revolutionise the way science is done. <<

Student Initiative for Open Science

This article is part of an ongoing collaborative project with the Student Initiative for Open Science (SIOS). The Amsterdam-based initiative is focused on educating undergraduate- and graduate-level students about good research practices.

In 2023, SIOS will be organising numerous colloquium events, including movie nights, lectures, and pub quizzes. Keep an eye on the website! SIOS is also looking for new members – if the article spoke to you and you would like to be part of the process of implementing the open science framework in the academic world, join today!