Model fit and parameter uncertainty in amortized variational inference based item response theory.

Abstract

Amortized variational inference (AVI) has become a popular, computationally efficient alternative to marginal maximum likelihood estimation (MML) for estimating high-dimensional item response theory models on large datasets. However, currently the AVI framework lacks methods to assess model fit and determine parameter uncertainty. Therefore, it would be desirable to have the well established, classical goodness-of-fit based modelling assessment tools from item response theory available to assess models fit with AVI. However, AVI optimizes a lower-bound to the marginal likelihood, and the asymptotic properties that are required to use these tools are not guaranteed. In this paper, we use AVI estimates in a likelihood-based Fisher scoring procedure, ensuring asymptotic guarantees associated with MML. Large-sample theory for one-step estimators shows that when the initial estimator is sufficiently accurate (within an O p ( N ) neighbourhood of the true parameter), a single Fisher scoring update is sufficient to obtain a consistent, and asymptotically efficient estimator. We investigate empirically whether one scoring step suffices to restore the validity of classical goodness-of-fit measures and standard error estimation. Using simulation studies and an application to large-scale Narcissism Personality Inventory data, we show that the proposed AVI-initialized one-step Fisher scoring approach yields fit statistics and uncertainty estimates that are on par with MML while retaining the computational advantage of AVI.