Unknown heterogeneity, the EC-EM algorithm, and Large T Approximation

Abstract

We study a panel structure with n subjects/entities being observed over T periods. We consider a class of models for each subject's data generating process, and allow for unknown heterogeneity. In other words, we do not know how many types we have, what the types are, and which subjects belong to each type. We propose a large T approximation to the posterior mode on the unknowns through the Estimation/Classification (EC) algorithm of El-Gamal and Grether (1995) which is linear in n, T, and the unknown number of types. If our class of models (likelihood functions) allows for a consistent asymptotically normal estimator under the assumption of homogeneity (number of types = 1), then the estimators obtained by our EC algorithm inherit those asymptotic properties as T ↑ ∞ and then as n t ↑ ∞ (with a block-diagonal covariance matrix facilitating hypothesis-testing). We then propose a large T approximation to the EM algorithm to obtain posteriors on the subject classifications and diagnostics for the goodness of the large T approximation in the• EC stage. If the large T approximation does not seem to be appropriate, then we suggest the use of the more computationally costly EM algorithm, or the - even more costly - full Bayesian updating. We illustrate the procedure with two applications to experimental data on probability assessments within a class of Pro bit and a class of Tobit models.