The symbolic machine learning (ML) field has defined some problems as the fundamental goals of its research. Among them are classification problems, clustering, discovery, etc. The problem of prediction is clearly not among them. There are few works in the ML field on this subject. Among the few exceptions are some papers on predicting discrete sequences of symbols [4, 10]. However, these works do not address the particular case of time series prediction, so they have a quite different scope from the work presented here.
The field of time series prediction is considered a key problem in several scientific areas and has been receiving increasing attention from several academic and non-academic institutions. Several fields related to machine learning like Neural Nets and Genetic Algorithms have already been working on this subject (see for instance [11, 13 and 18]). It is our believe that the symbolic machine learning area could add something to this scientific topic.
The main goal of this paper is to try to adapt an existing propositional learning system to the problem of time series prediction. We used the regression tree learner M5 [14, 15] in our experiments because it is able to perform numeric classification. In time series we want to predict future values of a numeric variable. This was one of the reasons for choosing the M5 system. M5 builds a decision tree with linear models in the leaves acting like a kind of piecewise linear regression model.
The main motivation for trying to use symbolic systems on these tasks is that they make it easy to include background knowledge to help the learning task. We argue that this inclusion could improve performance in prediction problems. Nevertheless, at the present stage of our work we still do not explore this issue.
The main difficulty of this adaptation is that prediction problems are different from classification problems that are the target problems addressed by the majority of ML systems. On prediction problems we have issues like time dependencies among data (autocovariance and autocorrelation of a stochastic process [7]) which are not directly mapped into the propositional learning framework. Our work is currently concentrated on this adaptation. We outline a technique based on feature selection that enables the use of ML systems on prediction. Feature selection is needed because if we use few features to describe the data we might be loosing important information on the temporal behavior of the data. On the contrary if we learn with too many features we may overfit the noise. The results of our experiments indicate that the choice of attributes is relevant to the obtained performance. This conclusion is in accordance to other work [3, 6] on testing the sensibility of tree learners to the used set of attributes.