Functional Models for Regression Tree Leaves

Luís Torgo
1997


Abstract

This paper presents a study about functional models for regression tree leaves. We evaluate experimentally several alternatives to the averages commonly used in regression trees. We have implemented an inductive learning system (HTL) that learns regression trees and is able to use several alternative models in the tree leaves. We study the effect on accuracy of using the different models and evaluate their computational cost. The experiments carried out on 11 datasets revealed that it is possible to significantly outperform the naive averages of regression trees. Among the four alternative models that we evaluated, kernel regressors were usually the best in terms of accuracy. Our study also indicates that by integrating regression trees with other regression approaches we are able to overcome the limitations of individual methods both in terms of accuracy as well as in computational efficiency.