Adaptive testing simulations with CATSim
Designed specifically to help you convert from traditional linear tests to adaptive tests
Computerized adaptive testing (CAT) is a sophisticated approach for delivering tests, surveys, and other measurements. It is based on complex computer algorithms that adapt the test to each examinee while also controlling for practical issues such as content distribution, item exposure, and test length. CATs have been shown to reduce test length by up to 90% without a loss of precision. However, to achieve such a reduction and make use of the advantages of CAT, it is absolutely essential that the CAT developer perform research studies to simulate the performance of CATs (see Thompson & Weiss, 2011).
CATSim implements three types of adaptive testing simulations: post-hoc (real-data) simulations, hybrid simulations, and monte-carlo simulations. These should be used in various stages of developing a CAT by documenting validity evidence for CAT options such as starting theta, theta estimation method, item selection method, item exposure control, and termination criterion.
CATSim implements simulations for all three dichotomous item response theory (IRT) models and five polytomous IRT models. CATSim includes all of the CAT options in FastTest, our secure platform for online testing, so that the results of using CATSim can easily be implemented in your testing program.
CATSim is designed to present options that directly represent specifications that must be made for CAT delivery. CAT requires the specification of 5 components:
CATSim records the adaptive testing simulations with item-by-item detail, but also provides a summary report describing the dependent variables necessary to adequately evaluate a CAT, such as average test length, accuracy, and item exposure.