Project topics and materials
Get Complete Project Material File(s) Now! »
Identification of the key manufacturing parameters impacting the prediction accuracy of SVM model for quality assessment
In this second chapter, the issue of attribute noise (or measurement uncertainties), as well as its impact on the robustness of SVM prediction accuracy, is discussed. Two main tasks are carried out accordingly. At first, the robustness of SVM accuracy regarding measurement uncertainties is assessed. The robustness assessment consists of quantifying the decrease in the accuracy of the SVM prediction due to perturbing the data with artificial measurement uncertainties. Afterwards, three approaches are proposed to identify which parameters’ uncertainties contribute the most to the SVM accuracy decrease. These approaches allow assigning to each parameter a quantitative coefficient that represents the magnitude of the impact of the uncertainties of each parameter on the robustness of SVM. On the one hand, the first two approaches rely on Monte Carlo simulations for the quantification of these coefficients. On the other hand, simple statistical tools are used in the third approach to estimate the impact of the parameters’ uncertainties on the SVM robustness. The proposed approaches would eventually make it possible to identify the uncertainties of the parameters that mostly affect the SVM. Such parameters are referred to as key measurement uncertainties. Identifying the key measurement uncertainties would provide a better understanding of how the SVM is affected by measurement uncertainties, as it would provide a strong basis for improving the robustness of SVM. In the following, the proposed approaches are described, then the results of applying them to four datasets are discussed and compared.
Introduction
It has been shown that measurement uncertainties have a negative impact on the predictive performance of classification methods, particularly SVM models. This impact can be manifested as a decrease in the ability of the classifier to generalize or an increase in the complexity of the model created (Wickramasinghe, 2017). As shown in Figure 2.1, the issue addressed in the second chapter is to quantify the effect of these measurement uncertainties on the predictive performance of SVM and thus on the generalization of SVM.
In order to quantify the overall impact of measurement uncertainties on the accuracy of the SVM, it is necessary to understand what is happening when predicting a data point that is subject to measurement uncertainties. As shown in Figure 2.2, measurement uncertainties are equivalent to translations that make one point move in all directions. Such movements may result in a point crossing the decision boundary and may therefore be misclassified. These movements become much more complicated when dealing with non-linear decision boundaries, as shown in Figure 2.3, and especially in high-dimensional spaces. For this reason, the impact of measurement uncertainties on each datapoint should be considered during the quantification task.
Accordingly, in this study, the probability that a datapoint that is subject to measurement uncertainties may cross the decision boundary is computed. An approach based on Monte-Carlo simulation is therefore proposed in order to quantify the impact of measurement uncertainties on the prediction performances of SVM.
Assessment of the SVM robustness by Monte-Carlo simulation
In this section, we proposed an approach based on a Monte-Carlo simulation in order to assess and quantify the impact of measurement uncertainties on SVM ‘s robustness.
The term Monte-Carlo refers to a family of algorithmic methods which use random processes to calculate an approximate numeric value. The main objective of Monte-Carlo simulation is to conduct a large number of experiments on random samples and then to draw conclusions about the model outputs (Metropolis and Ulam, 1949).
Monte Carlo simulations are considered as effective mathematical analysis methods used to solve complex engineering problems due to their ability to manage a large number of random variables, different types of distribution, and highly nonlinear engineering models (Mohammadi et al., 2015). Some of the particular applications of Monte-Carlo methods include computing integrals in dimensions greater than 1 (e.g. for computing areas and volumes). They are also widely used in particle physics, where probabilistic simulations are used to estimate a signal’s form, or detector sensitivity.
In this work, Monte-Carlo simulations are used to calculate the decrease in the accuracy of the SVM due to the perturbation of data with uncertainties. To do so, each dataset should be split into a training set (2/3 of the dataset) and a test set (1/3 of the dataset). That is necessary to tune the SVM hyperparameters by using the GA presented in Algorithm 1. Afterwards, random artificial uncertainties are generated to perturb the initial test set and thus define the noisy test sets. In this study, the impact of gaussian measurement uncertainties is studied, as they are considered to be one of the most commonly encountered uncertainties.
To generate artificial gaussian measurement uncertainties, a gaussian distribution (0, ) is associated to each input parameter, where is the standard deviation of the ℎ manufacturing parameter, and denotes the noise level. Once the gaussian distributions are established, the inverse transform sampling method is used to generate the different measurement uncertainties (Özdemir and Çavuş 2016). Afterwards, an initial experiment is carried out to assess the robustness of SVM to gaussian measurement uncertainties. This evaluation of SVM robustness is depicted in Algorithm 2.
Algorithm 2 assessment of SVM robustness by Monte Carlo simulation – noising all manufacturing parameters with gaussian uncertainties.
Inputs: Dataset M, N-sample of artificial gaussian measurement uncertainties sets MU
Output: SVM accuracy decrease due to the uncertainties of all manufacturing parameters: ∆
01: Split M into training set Ml, and test set Mt
2: Use Algorithm 1 to optimize the SVM model: Model_SVM
3: Predict Mt using Model_SVM: Acc
4: Initialize the SVM accuracy decrease: ∆ ← 0
5: for ∈ {1, … , } do
06: Pick the ℎ set of MU: ( )
7: Perturb M with( ): ( ) ⃪ + ( )
8: Split ( ) into training set ( ), and test set ( )
9: Predict ( ) using Model_SVM:( )
10: Update ∆ : ∆ ⃪ ∗ (( −( )) + ( − ) ∗ ∆ )
11: end for
12: return ∆
Algorithm 2 was applied on the four datasets, previously described in chapter1 and that using 1000 noisy test sets.
The noise levels were set at 2.5% for the three first datasets, and at 15% for the last dataset. These noise levels were estimated by taking into account the environment impact, and the precision and the accuracy of the sensors needed within the studied manufacturing/process industries. Table 2.1 gives the different results of the application of Algorithm 2.
As shown in Table 2.1, noising the test set with measurement uncertainties resulted in a decrease in the accuracy of the SVM prediction, i.e., the robustness of the generalization of SVM is affected by uncertainties. The results do not allow identifying which parameter uncertainties have the greatest impact on the SVM accuracy as explained in Figure 2.4 and Figure 2.5. In Figure 2.4, it is shown that the uncertainties of the first parameter have no effect on the classification of the datapoint; on the other hand, the uncertainties associated with the second parameter may result in a misclassification of this datapoint. Accordingly, the following research question was formulated: “Which parameter uncertainties have a significant impact on the accuracy of the SVM prediction?” To address this scientific question, three approaches for the evaluation of the impact of the uncertainties of each manufacturing parameter, and the identification of the key measurement uncertainties are proposed.
Identification of the key measurement uncertainties by Monte Carlo simulation
The first approach for the identification of the key measurement uncertainties is similar to the quantification performed in part II.2, where Monte-Carlo simulations are used to assess the robustness of SVM to gaussian measurement uncertainties. However, in this approach, the parameters are noised one by one instead of all at once. Thus, as many Monte-Carlo simulations as the number of parameters are required. This would allow quantifying the decrease in SVM accuracy due to the perturbation of one parameter with gaussian measurement uncertainties. These simulations allow ranking the manufacturing parameters according to their impacts on SVM, which allows identifying the key measurement uncertainties in return. The global description of the approach could be illustrated by Figure 2.6.
Table of contents :
Introduction
1 Chapter I Machine learning applications for quality assessment in manufacturing industry : Support Vector Machine (SVM)
1 I.1 Machine learning in manufacturing
1.1 Introduction
1.2 Classification methods for quality assessment in manufacturing
1.2.1 Decision trees (DT)
1.2.2 Support vector machine (SVM)
1.2.3 Multilayer perceptron (MLP)
I.2 Case studies
2.1 Product quality assessment by SVM – application on a manufacturing process (Roll_0/1 data)
2.2 Quality assessment using SVM
2.3 Identification of defects causes by C4.5
2.4 Optical process monitoring for Laser Powder Bed Fusion (L-PBF)
3 Support vector machines under uncertainties
3.1 Data noise: definition, sources, and impacts
3.2 SVM under uncertainties
Chapter II Identification of the key manufacturing parameters impacting the prediction accuracy of SVM model for quality assessment
1 Introduction
2 Assessment of the SVM robustness by Monte-Carlo simulation
3 Identification of the key measurement uncertainties by Monte Carlo simulations
4 Assessment of the SVM sensitivity by Sobol analysis
5 Estimation of key measurement uncertainties by correlation research
6 Discussion and conclusion
Chapter III Towards Robust SVM Model
1 Introduction
2 Selection of robust SVM models
2.1 Bi-objective optimization for the selection of robust SVM models
2.2 Bi-level optimization for the selection of robust SVM models
3 Feature weighting for the improvement of SVM robustness
3.1 Approach based on Sobol sensitivity indices
3.2 Improvement of SVM robustness: feature weighting and SVM hyperparameters
4 Discussion and conclusions
Chapter IV: General conclusion and perspectives
1 Conclusion
1.1 Assessment and improvement the quality of manufacturing systems based on I.1machine learning algorithms and classification approaches .
1.2 Quantification of the impact of measurement uncertainties on the performances of I.1the SVM classification method .
1.3 Improvement of the robustness of SVM classification
References
Appendices
