Item analysis

Within psychometrics, Item analysis refers to statistical methods used for selecting items for inclusion in a psychological test. The concept goes back at least to Guildford (1936). The process of item analysis varies depending on the psychometric model adopted; for example classical test theory or the Rasch model will call for different procedures. In all cases, however, the purpose of item analysis is to produce a relatively short list of items (that is, questions to be included in an interview or questionnaire) that constitute a pure but comprehensive test of one or a few psychological constructs.

The following brief description of the procedures used in item analysis is drawn from Kline (1986). In order to carry out the analysis, a large pool of candidate items all of which show some degree of face validity, are given to a large sample of participants who are representative of the population for whom the test is used. Ideally there should be at least 10 times as many candidate items as the desired final length of the test, and several times more people in the sample than there are items in the pool. A variety of statistical procedures are applied to the responses to the candidate items, in order to eliminate unsatisfactory items. For example, under classical test theory, items will be eliminated if the answers to them:

• show little variation within the sample
• are strongly correlated with one or more other items
• are weakly correlated with the totality of the remaining items, reflected in an increase in Cronbach's alpha if the item is eliminated from the test.

In practical test construction, item analysis is an iterative process, and it cannot be entirely automated; the psychometrician's judgement is required to determine whether the emerging set of items to be retained will constitute a satisfactory test of the target construct. The three criteria given above will not always agree, and a balance has to be struck between them in deciding whether or not to include an item.

References

• Guilford, J. P. (1936). Psychometric methods. New York: McGraw-Hill.
• Kline, P. (1986). A handbook of test construction. London: Methuen