Yaïr Pinto’s (Brain and Cognition) question
Dear Alexandra,
Psychology, and many other fields, have been plagued by the ‘replication’ crisis. In short, many prominent findings do not replicate, which suggests that the reported effects were flukes, or worse. Do you think that the current approach to tackle this problem is sufficient? Or do you think that we should adopt certain measures more rigorously to prevent this crisis from re-occurring? I am asking specifically, because as far as I understand we have had replication crises before, and generally memory of these kind of events seems to be short lived.
Yaïr
Alexandra Sarafoglou’s (Psychological Methods) answer
Dear Yaïr,
You were asking whether we believe that the current efforts in the psychological community are sufficient to increase the reproducibility in our field in the long run. We believe that this is the case.
This becomes clear, if we take a look at the various initiatives the scientific community has launched in the past years to increase the openness and transparency in the field. For instance, to incentivize data sharing researchers can sign the peer reviewers openness (PRO) initiative. When signing the initiative, researchers agree to exclusively review articles of authors that either share their data and materials in a public repository, or provide reasons why this was not possible. Over a thousand journals have implemented certain transparency standards by becoming signatories of the Transparency and Openness Promotion guidelines (TOP guidelines). Many journals certify articles with open science badges if authors preregistered their studies, or shared their data or materials. Lastly, study preregistration is gaining popularity quickly. When preregistering their studies, researchers commit to an analysis plan before they have collected any data. Researchers can choose to preregister their study either independently or integrate preregistration with the peer-review process (i.e., in the form of a Registered Report which was adopted by over 150 journals).
In the Netherlands, several labs focus especially on reproducibility research (e.g., the Meta-Research Center at Tilburg University). The Netherlands Organisation for Scientific Research (NWO) implemented an open access policy. And recently many Dutch universities have started Open Science Communities which aim to promote open scientific practices across all faculties and departments. For instance, our Open Science Community at the UvA offers advice on open science related questions for students and researchers from all faculties, and occasionally organizes events which aim to exchange experiences and knowledge (see https://openscience-amsterdam.com). All these initiatives do not only draw attention to the reproducibility crisis and the problems we are currently facing, they are offering concrete incentives for the adoption of open scientific practices as well.
Most crucially, however, is that compared to previous crises, all parties within the psychological community are doing their part to implement the reforms. That is, researchers, but also reviewers, journal editors, universities, and grant holders. For these reasons, we believe that the current efforts within the psychological community will increase the reproducibility in our field in the long run.
Alexandra
Alexandra’s question is for Arjen Noordhof (Clinical Psychology)
Dear Arjen,
I often heard from friends who work as psychotherapists, that outside of university clinics there is an increasing disconnection between clinical researchers and clinical practitioners. Specifically, clinical researchers are often interested in developing and evaluating standardized therapies that target isolated psychological disorders. Often, however, psychotherapists do not follow the recommended guidelines. Instead, they adapt their therapeutic methods to specific characteristics of their patients, because, for instance, the patients require treatment for multiple mental health problems. I was wondering whether you agree with the assessment that there is a disconnection between clinical research and clinical practice and how clinical researchers can improve the exchange of information.
Alexandra
Yaïr Pinto’s (Brain and Cognition) question
Dear Alexandra,
Psychology, and many other fields, have been plagued by the ‘replication’ crisis. In short, many prominent findings do not replicate, which suggests that the reported effects were flukes, or worse. Do you think that the current approach to tackle this problem is sufficient? Or do you think that we should adopt certain measures more rigorously to prevent this crisis from re-occurring? I am asking specifically, because as far as I understand we have had replication crises before, and generally memory of these kind of events seems to be short lived.
Yaïr
Alexandra Sarafoglou’s (Psychological Methods) answer
Dear Yaïr,
You were asking whether we believe that the current efforts in the psychological community are sufficient to increase the reproducibility in our field in the long run. We believe that this is the case.
This becomes clear, if we take a look at the various initiatives the scientific community has launched in the past years to increase the openness and transparency in the field. For instance, to incentivize data sharing researchers can sign the peer reviewers openness (PRO) initiative. When signing the initiative, researchers agree to exclusively review articles of authors that either share their data and materials in a public repository, or provide reasons why this was not possible. Over a thousand journals have implemented certain transparency standards by becoming signatories of the Transparency and Openness Promotion guidelines (TOP guidelines). Many journals certify articles with open science badges if authors preregistered their studies, or shared their data or materials. Lastly, study preregistration is gaining popularity quickly. When preregistering their studies, researchers commit to an analysis plan before they have collected any data. Researchers can choose to preregister their study either independently or integrate preregistration with the peer-review process (i.e., in the form of a Registered Report which was adopted by over 150 journals).
In the Netherlands, several labs focus especially on reproducibility research (e.g., the Meta-Research Center at Tilburg University). The Netherlands Organisation for Scientific Research (NWO) implemented an open access policy. And recently many Dutch universities have started Open Science Communities which aim to promote open scientific practices across all faculties and departments. For instance, our Open Science Community at the UvA offers advice on open science related questions for students and researchers from all faculties, and occasionally organizes events which aim to exchange experiences and knowledge (see https://openscience-amsterdam.com). All these initiatives do not only draw attention to the reproducibility crisis and the problems we are currently facing, they are offering concrete incentives for the adoption of open scientific practices as well.
Most crucially, however, is that compared to previous crises, all parties within the psychological community are doing their part to implement the reforms. That is, researchers, but also reviewers, journal editors, universities, and grant holders. For these reasons, we believe that the current efforts within the psychological community will increase the reproducibility in our field in the long run.
Alexandra
Alexandra’s question is for Arjen Noordhof (Clinical Psychology)
Dear Arjen,
I often heard from friends who work as psychotherapists, that outside of university clinics there is an increasing disconnection between clinical researchers and clinical practitioners. Specifically, clinical researchers are often interested in developing and evaluating standardized therapies that target isolated psychological disorders. Often, however, psychotherapists do not follow the recommended guidelines. Instead, they adapt their therapeutic methods to specific characteristics of their patients, because, for instance, the patients require treatment for multiple mental health problems. I was wondering whether you agree with the assessment that there is a disconnection between clinical research and clinical practice and how clinical researchers can improve the exchange of information.
Alexandra
