A Caltech Library Service

Cost-Bounded Active Classification Using Partially Observable Markov Decision Processes

Wu, Bo and Ahmadi, Mohamadreza and Bharadwaj, Suda and Topcu, Ufuk (2019) Cost-Bounded Active Classification Using Partially Observable Markov Decision Processes. In: 2019 American Control Conference (ACC). IEEE , Piscataway, NJ, pp. 1216-1223. ISBN 978-1-5386-7926-5.

[img] PDF - Submitted Version
See Usage Policy.


Use this Persistent URL to link to this item:


Active classification, i.e., the sequential decision making process aimed at data acquisition for classification purposes, arises naturally in many applications, including medical diagnosis, intrusion detection, and object tracking. In this work, we study the problem of actively classifying dynamical systems with a finite set of Markov decision process (MDP) models. We are interested in finding strategies that actively interact with the dynamical system, and observe its reactions so that the true model is determined efficiently with high confidence. To this end, we present a decision-theoretic framework based on partially observable Markov decision processes (POMDPs). The proposed framework relies on assigning a classification belief (a probability distribution) to each candidate MDP model. Given an initial belief, some misclassification probabilities, a cost bound, and a finite time horizon, we design POMDP strategies leading to classification decisions. We present two different approaches to find such strategies. The first approach computes the optimal strategy “exactly” using value iteration. To overcome the computational complexity of finding exact solutions, the second approach is based on adaptive sampling to approximate the optimal probability of reaching a classification decision. We illustrate the proposed methodology using two examples from medical diagnosis and intruder detection.

Item Type:Book Section
Related URLs:
URLURL TypeDescription Paper
Ahmadi, Mohamadreza0000-0003-1447-3012
Additional Information:© 2019 AACC. This work was supported by AFOSR FA9550-19-1-0005, DARPA D19AP00004, NSF 1646522 and NSF 1652113.
Funding AgencyGrant Number
Air Force Office of Scientific Research (AFOSR)FA9550-19-1-0005
Defense Advanced Research Projects Agency (DARPA)D19AP00004
Record Number:CaltechAUTHORS:20190905-154452427
Persistent URL:
Official Citation:B. Wu, M. Ahmadi, S. Bharadwaj and U. Topcu, "Cost-Bounded Active Classification Using Partially Observable Markov Decision Processes," 2019 American Control Conference (ACC), Philadelphia, PA, USA, 2019, pp. 1216-1223. URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:98463
Deposited By: Tony Diaz
Deposited On:05 Sep 2019 22:51
Last Modified:02 Jun 2023 00:42

Repository Staff Only: item control page