<< , >> , up , Title , Contents

3. Forgetting

The notion of flexible concepts suggests that some kind of forgetting capability should be included in an incremental learning system which deals with this type of concepts in order to put away those aspects of flexible concepts that have become obsolete. In this respect, the mechanisms of forgetting within learning systems have been studied in [Kubat,1989]. Also it has been found that forgetting the irrelevant pieces of knowledge can improve the accuracy of knowledge bases modelling static concepts. This was pointed out in [Markovitch&Scott,1988] and some mechanisms for forgetting were suggested in [Markovitch&Scott,1989].

In our opinion, there are two explanations for the effectiveness of forgetting: (1) noise in training examples and (2) improper selection of training examples. The first point is normally solved using pruning techniques. These include pre-pruning and post-pruning (for example [Cestnik et al., 1987] apply them to trees), respectively if the pruning is done during learning or after it. These methods are based on statistical tests of significance of the hypotheses (rules). Those tests indicate portions of the learned theory that are untrustable and should not be considered.

In (2) improper selection of training examples can lead to the learning of useless rules. For an illustration of that, consider the set of examples from fig.1a. The examples are classified into two classes, C1 and C2. If we choose the examples marked by * as being the training examples, a typical learning algorithm (no matter whether being incremental or not) will produce rules similar to those in fig.1b.

If we apply these rules on the the testing set consisting of all five examples of fig. 1a, we find out that only three of these examples (1, 2 and 3) are correctly classified. Now, if we forget the rule 'u /\ s => C1', we realize that the number of examples correctly classified by the new set of (two) rules increased to four with only example 2 being classified incorrectly.

Fig. 1 - Illustration of the meaning of forgetting

Now, an interesting problem arises: what part of the knowledge should be forgotten and under what circumstances? In the following sections, we present an approach based on the notion of knowledge integration, together with some experimental results. It is our belief that the performance gain achieved by this approach is obtained mainly by the fact that it solves the issue of reasonable forgetting of some useless rules, which makes it an alternative, or perhaps complement, to various pruning mechanisms.

<< , >> , up , Title , Contents