A Comparison of Item-Selection Methods for Adaptive Tests with Content

Linden van der, Wim J. (2005) A Comparison of Item-Selection Methods for Adaptive Tests with Content. Journal of Educational Measurement, 42 (3). pp. 283-302. ISSN 0022-0655

PDF
Restricted to UT campus only: Request a copy
153Kb

Abstract:	In test assembly, a fundamental difference exists between algorithms that select a test sequentially or simultaneously. Sequential assembly allows us to optimize an objective function at the examinee's ability estimate, such as the test information function in computerized adaptive testing. But it leads to the non-trivial problem of how to realize a set of content constraints on the test—a problem more naturally solved by a simultaneous item-selection method. Three main item-selection methods in adaptive testing offer solutions to this dilemma. The spiraling method moves item selection across categories of items in the pool proportionally to the numbers needed from them. Item selection by the weighted-deviations method (WDM) and the shadow test approach (STA) is based on projections of the future consequences of selecting an item. These two methods differ in that the former calculates a projection of a weighted sum of the attributes of the eventual test and the latter a projection of the test itself. The pros and cons of these methods are analyzed. An empirical comparison between the WDM and STA was conducted for an adaptive version of the Law School Admission Test (LSAT), which showed equally good item-exposure rates but violations of some of the constraints and larger bias and inaccuracy of the ability estimator for the WDM.
Item Type:	Article
Copyright:	© 2005 Wiley InterScience
Faculty:	Behavioural Sciences (BS)
Research Group:	Research Methodology, Measurement and Data Analysis (OMD)
Link to this item:	http://purl.utwente.nl/publications/58408
Official URL:	http://dx.doi.org/10.1111/j.1745-3984.2005.00015.x
Export this item as:	BibTeX EndNote HTML Citation Reference Manager

Show download statistics for this publication

Repository Staff Only: item control page

Metis ID: 225578