Practical PAC Learning

Dale Schuurmans, Russell Greiner

Abstract: We present new strategies for "probably approximately correct" (par) learning that use fewer training examples than previous approaches. The idea is to observe training examples one-at-a-time and decide "on-line" when to return a hypothesis, rather than collect a large fixed-size training sample. This yields sequential learning procedures that par-learn by observing a small random number of examples. We provide theoretical bounds on the expected training sample size of our procedure -- but establish its efficiency primarily by a scries of experiments which show sequential learning actually uses many times fewer training examples in practice. These results demonstrate that pac-learning can be far more efficiently achieved in practice than previously thought.