One basic assumption for predicting future values of a time series is that any value of the goal variable depends in some way on the previous values. The usual goal of classical linear time series models is to find a mathematical model that fits the observed past values. The resulting model is usually some kind of relation [17] of the next value to the previous ones.
We need to enable ML systems to learn theories that relate class values to previous values of the class. This somehow runs out of the scope of usual ML problems where we learn relations of class labels to attribute values.
There is an easy solution to this problem. We can pre-process the time series data creating attributes that contain previous class values. With this simple coding scheme we can learn relations among successive values of the goal variable.
The number of previous values of the goal variable (lagged values [7]) that we include as attributes is an important question as it will determine the type of models learned by the systems. If we include too many previous values we may introduce irrelevant information that can act as a kind of noise to the learning algorithm.
In the experiments described in this paper we have tried several values for the lag.
The introduction of attributes need not be restricted to information on previous class values. We can also include other relations between previous values of the goal variable. For instance if we want to capture information on trends we can add attributes calculated as differences of previous class values. This could be taken further by adding attributes with the value of differences of differences and so on.
Of course many other possible relations can be considered. In our experiments we limit ourselves to previous class values, differences, differences of differences and weighted averages of previous values. This does not mean that we think these relations will be sufficient to deal with any time series problem. They were just a first trial on the issue of feature selection. The next step is doing a kind of automatic testing of the relevance of these new attributes to guide the decision about which ones should be used in the learning process. This issue is discussed in more detail on section 2.2.2.
As said before we have tried attributes expressing different types of relations among previous class values. Using a basic a set of arithmetic functions we can define a kind of feature space. It is our goal to develop an algorithm that searches this space looking for the ideal feature set that guarantees the best performance in the given data. This is motivated by our experiments that showed the relation between the used features and the predicting accuracy. They also showed that the best set of attributes is domain dependent.
Recently, very interesting works [3, 6, 12] on the subject of deciding which attributes are relevant for a given learning task where presented. These techniques could be applied in our problem to help the decision of which attributes to use for a given time series prediction problem.