Get Complete Project Material File(s) Now! »

## Machine Learning (ML)

The problem examined in this thesis, the parametrization of a BR program in light of a statistical objective, can be seen as a ML problem. The main purpose of using ML in our work is to provide an existing framework to examine our problem, as well as use the known results pertaining to the learnability of function classes to prove that it cannot be solved in the general case. Research linking Production Rules and Machine Learning has recently focused on transforming the product of ML techniques, such as decision trees or associa- tion rules, into executable production rules [124, 102, 11]. Past efforts have also used Production Systems as learning systems themselves, among other works on learning by doing [9, 141]. The ML problem of modifying an existing computer program or complex function to account for new data is sometimes called theory revision [51]. The theory revision problem restricted to modifying parameters of the existing theory is similar to our problem. An approach to the non-statistical theory revision applied to Business Rules appears in [31], working with CLIPS-R. Answers to the theory revision problem with access to labeled data points (with a known label for each input) appear in [4, 64, 65]. We work on extending work made on this problem to statistical goal learning problems in Production Rules systems, in particular BR systems. In this section, we first formally express our problem and contrast it to existing ML formulations. We then present some existing results we will use in Chapter 3 pertaining to the learnability of function classes.

### Statistical goal learning

One of the challenges of BR systems in the current industry is maintaining the flexibility that has lead to its adoption as the automatic decision management tool of choice by many organizations. While BRMS make it simple to maintain and modify BR programs, they do not offer a way to predict the effect of any modification, nor do they supply a framework for Machine Learning (ML) of BR programs in view of a goal. Business Users must rely on their knowledge of the business process and the goal to implement any modifications to the BRs on their own. In a digitalized world where ML and “big data” are becoming more than buzzwords, this can undermine the agility of BRMS, or at least make it costly and inefficient. The goals of such modifications can be of a statistical nature, e.g. adjusting the average decision over a given set of inputs. For example, a bank might require that no more than a certain percentage, e.g. 30%, of all requested loans be examined manually, due to human resource concerns. As the number of manually examined loan requests should always be as high as possible, the goal is in fact to have the BR program which determines such things assign 30% of all loan requests to manual examination. Such a requirement could be fulfilled by automatically learning the “right values” of some adjustable parameters in the corresponding BR program. If the output of the BR program is a binary variable using 1 for manual evaluation and 0 for automatic evaluation of a loan request, the learning problem would be to find the values of the parameters which satisfy the statistical goal: “average of all outputs is 0.3”. This could arise as a consequence of exceptional circumstances, e.g. a new legislation, or of natural trend evolution, e.g. the client base changing so that too many or too few loan requests are evaluated automatically.

#### Mathematical Programming (MP)

The approach we use to learn the parameters of BR programs is to treat the learning problem in Eq. 2.2 as an optimization problem. A common approach to optimization, which we adopt in this thesis, is called Mathematical Programming (MP) [142]. MP is the practice of using Mathematical Programs (or MP problems) to model and solve problems. Among those Mathematical Programs, we are make particular use of the Mixed-Integer Linear Programming (MILP) problems, which can be solved particularly well by existing optimization solvers. In this section, we first introduce MP problems as well as some well-known classes of MP problems. We then mention the associated state of the art solvers and explain our choice of using CPLEX to test our algorithm (in Ch. 6).

**Table of contents :**

Glossary

**1 Introduction **

1.1 Motivation

1.2 The contributions

1.3 The approach

**2 Definitions and state of the art **

2.1 Business Rules (BRs)

2.1.1 BRs in industry and research

2.1.2 A formal definition of BRs

2.2 Machine Learning (ML)

2.2.1 Statistical goal learning

2.2.2 Computational Learning theory

2.3 Mathematical Programming (MP)

2.3.1 MP formalism

2.3.2 MP solvers

**3 BR programs **

3.1 Turing-Completeness of BR programs

3.1.1 Turing-Completeness

3.1.2 Basic Turing-completeness Proof

3.2 Constructive proof and resulting Operational Semantics

3.2.1 BR form of WHILE programs

3.2.2 WHILE form of BR programs

3.2.3 A Structural Operational Semantics for BR programs .

3.3 Learnability of the statistical goal learning problem

3.3.1 PAC-Learnability

3.3.2 Complete unlearnability

**4 MP based algorithm **

4.1 General Iteration Bounded BR programs

4.1.1 IBBR programs in two BR languages

4.1.2 Complexity of IBBR programs

4.1.3 Declarative form of an IBBR programs

4.2 A learning algorithm: MP

4.3 Linear Iteration Bounded BR programs

4.3.1 Linear BR programs

4.3.2 Example: Decision Trees

4.4 An algorithm for LIBBR programs

4.4.1 A learning algorithm: MIP

4.4.2 A learning algorithm: MILP

4.5 Theoretical complexity

**5 General statistical goal learning **

5.1 Learning quantized distributions

5.2 Upper bound on a specific output value’s probability

5.3 Almost uniform distribution

**6 Experimental work **

6.1 Parsing ODM

6.2 Experimental data on learning LIBBR

6.2.1 Experimental setup

6.2.2 Validating the learning algorithm

6.2.3 Testing performance

6.2.4 Testing other statistical goal learning problems

**7 Discussion **

7.1 A formal study of BRs

7.2 New learning algorithms

7.3 Industrial applications

7.4 Conclusion

**A Complete proof of some theorems **

A.1 Complete proof of Theorem 4.8

A.2 Complete proof of Theorem 4.9

A.3 Complete proof of Theorem 4.10 .