Economical Experiments: Bayesian Efficient Experimental Design
We propose and implement a Bayesian optimal design procedure. Our procedure takes as its primitives a class of models, a class of experimental designs, and priors on the nuisance parameters of those models. We select the experimental design that maximizes the information (in the sense of Kullback-Liebler) from the experiment. We sequentially sample with the given design and models until all but one of the models has viable posterior odds. A model which has low posterior odds in a small collection of models will have an even lower posterior odds when compared to a larger class, and hence we can dismiss it. The procedure can be used sequentially by introducing new models and comparing them to the models that survived earlier rounds of experiments. The emphasis is not on running as many experiments as possible, but rather on choosing experimental designs to distinguish between models in the shortest possible time period. The first stage of optimal design is illustrated with a simple experimental game with one-sided incomplete information.
We acknowledge the financial support from NSF grant #SES-9223701 to the California Institute of Technology. We also acknowledge the research assistance of Eugene Grayver who wrote the software for the experiments. This paper was presented at the Stony Brook workshop of Experimental Game Theory, summer 1993. Published as El-Gamal, Mahmoud A., and Thomas R. Palfrey. "Economical experiments: Bayesian efficient experimental design." International Journal of Game Theory 25, no. 4 (1996): 495-517.
Submitted - sswp884.pdf