Page 370 - Invited Paper Session (IPS) - Volume 2

IPS279 Rense Lange
computation of the item, steps, and grader parameters consistently took less
than 0.001 seconds. The updating of the frequency matrices was quite efficient
as a complete run with 10,000 students and 250 parameter updates required
less than 0.5 seconds on a standard 2014 MacBook Pro laptop computer.

4. Discussion and Conclusion
We found that the present approach can estimate MFRS parameters
without noticeable delay for plausible numbers of students and raters. For
instance, current OBJECTIVE runs with up to 2 million students still require just
0.001 seconds to update all item, steps, and rater parameters. An alternative
approach was used to recover the model parameters as the principal
eigenvector of log(R) matrices [8, 9]. However, this did not improve
performance or precision and the results are not further reported.
This paper did not address the estimation of the person parameters, nor
did it discuss the computation of the diagnostics and quality control statistics
[1, 2, 3]. Also, parallel processing as referred to above has not yet been
implemented (the use of Python’s “multiprocessing” libraries is considered for
this). This work is progressing and I intend to report further results on future
occasions.

References:
1. Van der Linden, W.J. & Hambleton, R.K. (1997). Handbook of modern
item response theory. New York: Springer.
2. Lange, R. & Lange, X. (2012). Quality control in crowdsourcing: An
Objective Measurement Approach to identifying and correcting rater
effects in the social evaluation of products and services. AAAI Spring
Symposium Series, North America, Stanford, CA, 2012.
3. Linacre, J. M. (1989). Many-facet Rasch measurement. Chicago: MESA
Press.
4. Linacre, J. M. (2018). Facets® Rasch measurement computer program.
Beaverton, OR: Winsteps.com.
5. Andrich, D., Sheridan, B. E., & Luo, G. (2004). RUMM2020: Rasch
unidimensional measurement models [Computer software]. Perth,
Western Australia: RUMM Laboratory.
6. Arce-Ferrer, A. & Lange, R. (2012). Assessing incidence and consequence
of rater effects on open-ended scoring. Paper presented at the 8-th
Conference of the International Test Commission (ITC). Amsterdam, The
Netherlands, July 3-5, 2012.
7. Rasch, G. (1960/1980). Probabilistic models for some intelligence and
attainment tests. (1960, Copenhagen, Danish Institute for Educational
Research), 1980, Chicago: U. of Chicago Press.
8. Garner, M. & Engelhard, G.E. (2009). Using paired comparison matrices

357 | I S I W S C 2 0 1 9