Many doctors consistently overestimate the likelihood of certain common diseases and conditions, both before and after testing is performed, according to a new study. The misunderstanding of probability may lead to overtesting and overtreatment, the authors say.
“Correct ordering and interpretation of tests are increasingly important given the increase in the number and complexity of tests,” wrote study authors in the US led by Dr Daniel Morgan, of the University of Maryland School of Medicine, in JAMA Internal Medicine. “Medical decisions, like other human decisions, may not be rational and are prone to errors associated with poor knowledge of the base rate of disease or other errors associated with probability.”
The researchers conducted a survey-based study including a total of 553 physicians, nurses, and physician assistants across eight US states. The practitioners were presented with four clinical scenarios and were asked to estimate the probability of four diseases (pneumonia, breast cancer, cardiac ischaemia, and urinary tract infection) based on the patient characteristics and risk factors. They were then asked to estimate the probability again given either a positive or negative test result for those conditions.
The pretest probabilities for disease were overestimated in all four scenarios. For example, in one scenario a 45-year-old woman comes in for an annual visit, and has no specific risk factors or symptoms for breast cancer. The evidence-based probability for breast cancer in this scenario is 0.2-0.3%, but the survey respondents had a median estimate of 5%.
If the woman then undergoes mammography and a positive result is returned, the respondents estimated the probability of breast cancer at 50%; the evidence suggests it is actually 3-9%. If the test result is negative, the evidence says the risk of breast cancer is below 0.05%, but the respondents still estimated the likelihood at 5%.
The same pattern was observed for scenarios involving pneumonia, cardiac ischaemia, and urinary tract infection, though there was some variation in how the positive or negative test results impacted estimates of probability.
“Many practitioners reported that they would treat patients for disease for which likelihood had been overestimated,” the authors wrote. For example, 78.2% said they would treat cardiac ischaemia when a positive test would place the patient at 11% or less chance of disease.
They noted that for the purposes of the survey test results were simplified to positive or negative, while in actual practice many tests offer a range of descriptions.
“This significant overestimation of disease likely limits the ability of practitioners to engage in precise and evidence-based medical practice or shared decision-making,” the authors concluded.
In an invited commentary on the study, Dr Arjun Manrai, of Boston Children’s Hospital and Harvard Medical School, wrote that the study does simplify what are always much more complicated situations, where other factors such as physical exam findings and patient demographics may play a role in the survey respondents’ estimates.
“What tacit assumptions are being folded into the respondents’ answers that go beyond a case prompt of a few lines?” he asked. “Are nontraditional risk factors or comorbidities the same, or are different demographic groups being considered? To what extent are the responses manifesting a psychological desire not to miss anything that could cause harm?” It is also true that many respondents were residents or academic physicians in clinical settings with higher disease prevalence than that seen in the literature.
“The findings… also point to new targets for medical education and research avenues for how probabilistic information might be better integrated into care,” Dr Manrai wrote.