Neal Dawson
Cleveland, Ohio

Our research this past year has used methods that we believe are valuable in performing nomothetic comparisons in real life medical care settings.

We have been evaluating the accuracy of physician judgments in actual clinical settings and have compared the accuracy of easily identified groups of physicians, e.g., physicians from different medical specialties. This has led to the need to address the issue of physicians seeing similar but not identical patients in clinical settings, e.g., patients who have the same diagnosis but may have differing levels of severity of illness or comorbidity. This is caused by selection bias. In prior studies of factors that influence patient survival, we have used the propensity score methodology to adjust for selection bias that arises when patients are selected (e.g., for a given treatment) or are otherwise nonrandomly distributed within a clinical care setting based on patient characteristics (e.g., severity of illness). A propensity score for a dichotomous outcome is created using logistic regression. The dependent variable is the outcome of interest (e.g., survival) and the independent variables are the patient based characteristics that are associated with selection (e.g., for treatment) or that are associated with the outcome of interest (e.g., mortality). We have adapted this methodology to create groups of patientswho have sufficiently similar baseline characteristics to allow a fair comparison of judgmental accuracy.

We were interested in comparing the accuracy of survival estimates of two types of physicians who care for seriously ill cancer patients: generalists and oncologists. Using patient characteristics, we created a propensity model that predicted the likelihood of a patient seeing an oncologist (vs. a generalist). The adjustment for selection bias created two groups of essentially identical patients with respect to baseline characteristics that are important to mortality risk. We then compared the accuracy of generalists and oncologists judgments of survival of patients under their care.

Across all judgments, we found that oncologists were generally too optimistic in their predictions and that generalists were too pessimistic but were more accurate than the oncologists. Accuracy varied by patient age such that oncologists were more accurate for younger patients (under 40), generalists were more accurate for middle aged and for older patients. Accuracy also varied by the number of estimates made (<10 vs. >=10). This relationship is currently being evaluated and will be presented at the Brunswik Society meeting in Dallas along with details of the propensity score methodology.

Contact Neal Dawson