The use of deliberate reflection to reduce confirmation bias among orthopedic surgery residents

Received on: Nov, 17th, 2021. Approved on: Jan, 18th, 2022. Published on: Mar. 07th, 2022. Abstract Introduction: cognitive biases might affect decision-making processes such as clinical reasoning and confirmation bias is among the most important ones. The use of strategies that stimulate deliberate reflection during the diagnostic process seems to reduce availability bias, but its effect in reducing confirmation bias needs to be evaluated. Aims: to examine whether deliberate reflection reduces confirmation bias and increases the diagnostic accuracy of orthopedic residents solving written clinical cases. Methods: experimental study comparing the diagnostic accuracy of orthopedic residents in the resolution of eight written clinical cases containing a referral diagnosis. Half of the written cases had a wrong referral diagnosis. One group of residents used deliberate reflection (RG), which stimulates comparison and contrast of clinical hypotheses in a systematic manner, and a control group (CG), was asked to provide differential diagnoses with no further instruction. The study included 55 third-year orthopedic residents, 27 allocated to the RG and 28 to the CG. Results: residents on the RG had higher diagnostic scores than the CG for clinical cases with a correct referral diagnosis (62.0±20.1 vs. 49.1±21.0 respectively; p = 0.021). For clinical cases with incorrect referral diagnosis, diagnostic accuracy was similar between residents on the RG and those on the CG (39.8±24.3 vs. 44.6±26.7 respectively; p = 0.662). We observed an overall confirmation bias in 26.3% of initial diagnoses (non-analytic phase) and 19.5% of final diagnoses (analytic phase) when solving clinical cases with incorrect referral diagnosis. Residents from RG showed a reduction in confirmation of incorrect referral diagnosis when comparing the initial diagnosis given in the non-analytic phase with the one provided as the final diagnosis (25.9±17.7 vs. 17.6±18.1, respectively; Cohen d: 0.46; p = 0.003). In the CG, the reduction in the confirmation of incorrect diagnosis was not statistically significant. Conclusions: confirmation bias was present when residents solved written clinical cases with incorrect referral diagnoses, and deliberate reflection reduced such bias. Despite the reduction in confirmation bias, diagnostic accuracy of residents from the RG was similar to those from the CG when solving the set of clinical cases with a wrong referral diagnosis.


Introduction
Clinical reasoning is a critical skill for practicing physicians and is related to their ability to generate diagnoses and make decisions. Evidence shows that there are two different forms of reasoning, one being non-analytical, based on pattern recognition (type 1), and the other being analytical dependent on the application of rules (type 2) (1).
Non-analytical reasoning is fast and intuitive and used for generating initial hypotheses and solving routine clinical cases. Analytical reasoning is more conscious, slow, and logical, and therefore more time consuming and more demanding on working memory for its execution. The latter is used in hypothesis checking and seems fundamental in resolving complex and atypical clinical cases, more susceptible to "diagnostic pitfalls" (2). The non-analytical and analytical reasoning systems seems to act simultaneously, with the non-analytical system generating the initial diagnostic hypotheses and the analytical system monitoring them, and even being able to correct erroneous diagnostic hypotheses that have been (3)(4). Cognitive errors appear to be the leading cause of diagnostic errors and are often the result of cognitive biases (8)(9)(10). In a study analyzing 100 diagnostic error cases, cognitive errors were observed in 74% of cases (11). Among such biases is the confirmation bias, which can be defined as a clear tendency to maintain the initially established clinical hypothesis, disregarding existing contrary evidence (13). In other words, confirmation bias is present when clinical reasoning is based only on the initial clinical hypothesis and, in a selective Antônio Barbosa Chaves • et al. The use of deliberate reflection to reduce confirmation bias among orthopedic surgery residents

3/9
and biased manner, the information initially provided is accepted, disregarding clinical data that contradicts it (8,13). It also involves a tendency to selectively search for confirmatory evidence that supports the initial diagnosis, rather than searching for evidence that might refute (14)(15).
Several instructional methods have been proposed for reducing cognitive biases in clinical reasoning (16). One such method involves using an instrument that promotes deliberate reflection, guiding the learner to analyze the initial clinical diagnosis searching for evidence that confirms or contradicts it, and contrast it with alternative (4).
The use of deliberate reflection, i.e., stimulating analytical reasoning in a structured manner, has led to improved diagnostic accuracy among internal medicine residents in studies that analyzed cognitive error related to availability bias (17,18).

Study design
Controlled experimental study with third-year orthopedic residents conducted in a single session where they were asked to solve eight written clinical cases of upper limb injuries. The clinical cases contained referral diagnoses to induce confirmation bias. The residents were not informed that in four cases, this referral diagnosis was plausible but incorrect. Participants were randomly assigned to one of two experimental groups (deliberate reflection vs differential diagnosis) and solved one clinical case at a time. Once the diagnoses chosen were defined, the cases were prepared by one of the study authors (ABC) and validated by two other hand surgery specialists and two orthopaedic residents.

Intervention
The eight written clinical cases were presented in a sequential and controlled manner. Participants received written directions on how to solve the cases, which differed according to the experimental group to which they were allocated.
In the first step of clinical cases analysis, participants in both groups were instructed to do a quick reading of the case (1 minute) and provide an initial diagnostic hypothesis (non-analytical step). The directions for the second step, which lasted 5 minutes for each case, differed by experimental group. Residents in the reflection group (RG) were instructed to reanalyse the case using an instrument to promote deliberate reflection as used in previous studies (17,(19)(20). Briefly, deliberate reflection consisted of describing the clinical findings that favour or contradict the initial diagnostic hypothesis, as well as expected findings in the case description if this hypothesis held true that were, however, absent. Then the residents were asked to list two other differential diagnoses and repeat the above procedure for each of the hypotheses. Finally, they were asked to contrast the diagnoses and select which one would be the most likely for the case. Residents in the control group (CG) were asked to reread the case and list two differential diagnoses before choosing a final diagnosis with no further instructions. Figure 1 shows the flowchart of the study.
The lack of overall differences in diagnostic accuracy between the experimental groups could also be explained by the fact that both groups had enough time to reflect upon the cases. The difference between groups in our study was the type of reflection, with one group reflecting in a deliberate and structured way, being encouraged to contrast differential diagnoses systematically, and the other group reflecting in a less structured Antônio Barbosa Chaves • et al. The use of deliberate reflection to reduce confirmation bias among orthopedic surgery residents 7/9 way since they were asked only to provide a list of differential diagnoses. It may also be that this structuring of reflection is not as crucial for more advanced learners as it is for undergraduate medical students where the effect of deliberate reflection has mostly been shown (23,25).
Another explanation for the lack of an overall effect of deliberate reflection may be related to the fact that the learner's knowledge is more important than the way they reason (26)(27)(28). Because our study was randomized, we would not expect overall differences in knowledge between the two groups, and neither intervention provided content that could increase residents' knowledge.
Strategies aimed at increasing specific knowledge through multiple examples seem to be critical in helping medical doctors to differentiate between diseases that appear similar and are promising strategies to "immunize" them against biases in reasoning, such as availability bias (18).
Unfortunately, definitive solutions for cognitive errors do not exist. Further studies are needed to assess the effect of combined educational strategies and tools that encourage students and residents to become aware of these biases, to consider alternative diagnoses, to improve metacognition, to calibrate their diagnosis from immediate feedback and to strengthen their knowledge of discriminating features (14,16,26).

Conclusions
In clinical cases in which an incorrect referral diagnosis was provided, the group of orthopedic residents who solved them using deliberate reflection significantly reduced confirmation bias.
However, no improvement in overall diagnostic accuracy was observed using deliberate reflection compared to a control group, guided only to provide differential diagnoses.

Funding
This study did not receive financial support from external sources.

Conflicts of interest disclosure
The authors declare no competing interests relevant to the content of this study.

Authors' contributions
All the authors declare to have made substantial contributions to the conception, or design, or acquisition, or analysis, or interpretation of data; and drafting the work or revising it critically for important intellectual content; and to approve the version to be published.

Availability of data and responsibility for the results
All the authors declare to have had full access to the available data and they assume full responsibility for the integrity of these results.