Videos

Modeling with Rules

Presenter
March 29, 2012
Keywords:
  • Model theory
MSC:
  • 03Cxx
Abstract
I am working on the design of predictive models that are both accurate and interpretable by a human. These models are built from association rules such as "dyspepsia & epigastric pain -> heartburn." Association rules are commonly used in the data mining community for database exploration, but have not been heavily employed in machine learning or statistics for prediction problems. I will present three algorithms for "decision lists," where classification is based on a list of rules: 1) A very simple Bayesian rule-based algorithm, which is to order rules based on the "adjusted confidence." In this case, users can understand the whole algorithm as well as the reason for the prediction. (This algorithm has a foundation in statistical learning theory, though I will not focus on this during the talk.) 2) A Bayesian hierarchical model for sequentially predicting conditions of medical patients, using association rules. This is essentially a recommender system for medical conditions. The model allows us to borrow strength from similar patients when there are not enough data available about a given patient. This is a hierarchical version of the adjusted confidence algorithm from the first topic. 3) A mixed-inter optimization (MIO) approach for learning decision lists. This algorithm has high accuracy and interpretability - both owing to the use of MIO. In our experiments, this algorithm has interpretability on par with decision trees, and has accuracy on par with boosted decision trees and SVM's with Gaussian kernels. The algorithm has regularization both on the sizes of rules, and also on the total number of rules in the list, leading to small lists. This is joint work with David Madigan, Tyler McCormick, Allison Chang, and Dimitris Bertsimas. Publications/Drafts of these works are here: topic 1: http://web.mit.edu/rudin/www/RudinLeKoMaSSRN11.pdf topic 2: http://web.mit.edu/rudin/www/McCormickRuMa11OR38511.pdf topic 3: http://web.mit.edu/rudin/www/BertsimasChRuOR38611.pdf