This research line consisted on trying to develop a propositional learning system that learns regression rules from examples. A regression rule is a IF-THEN rule that has as conditional part a set of conditions on the input features and as conclusion a regression model. One of the key goals of the project was to use more complex regression models than the ones usually seen on current ML regression systems. The resulting system (R2) uses a lattice of target regression models ranging from simpler y = "constant" to more complex ones like linear regression functions on the input features. The system was developed in such a way that this latice is easily enlarged without the need of further modifications on other system components. R2 uses a kind of covering algorithm that starts to find an uncovered region of the input space and finds the model (from its model latice) that best fits this region. After this initial phase the system tries to restrict the region (i.e. the applicability domain of the model) so that the fit improves. With this restriction the coverage of the rule gets worse. We have thus a typical case of consistency/coverage trade-off. R2 uses a heuristic weighted average of fit (consistency) and coverage to know when to stop this domain specialization process.
This project was somehow unfinished due to a strategic change of research topic. This change was motivated by the slowness of rule-based systems when compared to other approaches like tree-based propositional systems. As one of my current goals is to deal with very large datasets I've started to look at regression trees which lead to the HTL project. Neverthelesss, I still think there is space for some further improvements on R2, like changing the implementation language from the slow Prolog or improving the latice of target models to enhance the descriptive power of the target theories.