Learning in Embedded Systems

Learning to perform complex action strategies is an important problem in the fields of artificial intelligence, robotics, and machine learning. Filled with interesting new experimental results, Learning in Embedded Systems explores algorithms that learn efficiently from trial-and error experience with an external world. It is the first detailed exploration of the problem of learning action strategies in the context of designing embedded systems that adapt their behavior to a complex, changing environment; such systems include mobile robots, factory process controllers, and long-term software databases. Kaelbling investigates a rapidly expanding branch of machine learning known as reinforcement learning, including the important problems of controlled exploration of the environment, learning in highly complex environments, and learning from delayed reward. She reviews past work in this area and presents a number of significant new results. These include the intervalestimation algorithm for exploration, the use of biases to make learning more efficient in complex environments, a generate-and-test algorithm that combines symbolic and statistical processing into a flexible learning method, and some of the first reinforcement-learning experiments with a real robot.