Article #11
Newsletter list 


Previous Article 

Next Article 


Nomothetic judgment analysis

Mandeep Dhami
London, United Kingdom

Judgment Analysis (JA) within the framework of Social Judgment Theory is commonly (traditionally) conducted at the idiographic level, using linear regression modelling techniques. Over the past year, in an attempt to apply JA to expert decision making in the legal domain, I stumbled upon many practical problems concerning collection and analysis of judgment data.

The first study investigated British magistrates' bail decision making on hypothetical cases, using an orthogonal cue set. The high case to cue ratio necessary for regression modelling meant that I had to either reduce the number of cues studied or increase the number of cases. Magistrates were unwilling to participate in either a time consuming task, or an unrealistic task. The second study investigated magistrates' bail decision making in the courtroom, and used a fully representative design. Here, there were high inter-cue correlations between some cues; and in the courtroom, data was missing on some legally important cues, which created problems for conducting regression modeling. The cues with a lot of missing data had to be omitted, and it was difficult to ascertain which cues were used.

After brief consideration of abandoning JA and/or legal decision making, I decided instead to test alternative ways of conducting JA.

Firstly, rather than conducting JA at the idiographic level, I developed and tested JA at the nomothetic level. Individual magistrates made judgments on a smaller case to cue ratio, namely 3:1. The judgments of the whole group were then analyzed using regression modeling techniques, and the model of the group was cross-validated at the idiographic level on a set of holdout cases. The cross-validation procedure revealed that the model of the group adequately represented the policies of individual magistrates, taking into consideration each magistrate's level of consistency; and it also identified a small sample of magistrates for whom idiographic level analysis may be necessary. The ability of the nomothetic level analysis to adequately represent individual magistrates' policies is not surprising as previous studies using JA at the idiographic level have found that clusters of judges with similar policies emerge.

Secondly, rather than using regression modeling techniques to model judgment, I adapted and tested a simple heuristic referred to as Take The Best (TTB) which is one of a family of algorithms developed within the framework of Probabilistic Mental Models, by Gigerenzer & Goldstein (1996). Regular readers of the Brunswik Newsletter may recall that Ulrich Hoffrage mentioned such models in the 1996 issue, and attendants at the 1997 Brunswik Meeting in Philadelphia may recall hearing Gerd Gigerenzer speak on this topic. The TTB I adapted and tested uses judgment data in a frequency format and can cope with inter-cue correlations and cues with missing data. The TTB model cross-validated well on a set of holdout cases. The representation of human judgment provided by the TTB model is one of limited information search and one reason decision making, and so suggests that magistrates are non-additive, non-linear and non-compensatory. Given the limited demand on information processing capacity and complexity, the TTB model provides a psychologically more plausible account of human judgment.

So, over the past year, I've used JA, but not as we know it. In the year to come, I aim to continue using simple heuristics in modeling human judgment.

Anyone interested in copies of the above studies may contact me, and I would welcome any comments.

Contact Mandeep Dhami

Previous Article                Next Article

Newsletter list