Get Complete Project Material File(s) Now! »

## Fact Inference with AMIE

In Section 2.6 we showed the performance of AMIE in terms of runtime. However, we did not analyze the quality of the output rules. This section evaluates AMIE in terms of the produced rules, more specifically, in how correct are the conclusions drawn by those rules. In Section 2.1 we stated that, one of the goals of the AMIE mining model is to learn rules that make correct predictions, in and beyond the KB. The task of predicting facts outside the KB is often referred in the literature as link prediction [14, 45, 65, 81, 93, 112, 125] because it aims at finding semantic links (relations) between entities in the KB. This is a difficult endeavor : It amounts to guessing the places of residence for people, their birth place, or even their death place. Naturally, we may not assume a high precision in the prediction of the future. We may only expect educated guesses.

To evaluate the precision of these guesses for the studied rule mining systems, we proceeded as follows : We ran the system on the YAGO2 dataset and ranked the obtained rules by confidence. For each rule, we evaluated whether the predictions that go beyond YAGO2 were true. We did this by either checking if the prediction appears in a newer version of the KB (YAGO2s), or by manually checking them in Wikipedia. If we could find the predicted fact in neither, we evaluated it as false. Since the number of predictions can be large, we conducted the evaluation of the precision on a random sample of the predictions. We applied this experimental setup to assess the precision of the

predictions for AMIE using the standard confidence, AMIE using the PCA confidence (Section 2.7.1) and ALEPH using the positives-only learning function (Section 2.7.2).

### Standard vs. PCA Confidence

Our first goal is to see whether the PCA confidence or the standard confidence perform better for the task of fact prediction. For this purpose, we ran AMIE on YAGO2, and sorted the resulting rules first by descending PCA confidence, and then by descending standard confidence. We looked at the top ranked rules in each case, and evaluated the precision of the predictions. The two bottom curves of Figure 2.1 plot the aggregated predictions versus the aggregated precision for the standard and the PCA confidence.

The n-th dot from the left represents the total number of unique predictions and the total precision of these predictions (ratio of correct predictions), aggregated over the first n rules. As we see, ranking the rules by standard confidence is a very conservative approach : It identifies rules with reasonable precision, but these do not produce many predictions. Going down in the list of ranked rules, the rules produce more predictions – but at lower precision. The top 30 rules produce 113K predictions at an aggregated precision of 34%. In contrast, if we rank the rules by PCA confidence, we quickly get large numbers of predictions. The top 10 rules already produce 135K predictions – at

a precision of 45%. The top 30 rules produce 3 times more predictions than the top 30 rules by standard confidence – at comparable precision. This is because the PCA confidence is less conservative than the standard confidence. We thus conclude that the PCA confidence is better suited for making predictions than the standard confidence. In addition, we investigate how the confidence metrics correlate with the actual precision of the rule, e.g., in case we want to use them as estimators of the real precision.

Table 2.14 summarizes our findings. We rank the mined rules by their precision and report the average absolute error of the standard and PCA confidence weighted by the number of predictions produced by the rules. We can observe that, on average, the PCA confidence estimates the precision of the rules better than the standard confidence. Thus, reasoning approaches could use the PCA confidence as a weight for the rule.

#### Predicting Semantics for Wikilinks

Our approach to semantify wikilinks relies on the intuition that (a) wikilinks often convey a semantic connection between entities, (b) some of them are already semantified in KBs, (c) the types of the entities in the wikilink define the signature of its implicit relation, and (d) the already semantified wikilinks can help us semantify the others. The already semantified wikilinks constitute our training set. From this training set, we mine a set of semantic patterns in the form of logical rules.

To justify our intuition, we look at the types of the endpoints of semantified wikilinks in DBpedia. We restrict our analysis to the classes Person, Place and Organization. Table 4.1 shows the most common relations holding between pairs of those entities for which there exists at least one wikilink.

For example, we observe that when a person links to a place, in 56% of the cases, the person was born in that place. Similarly, when an organization links to a place, in 19% of the cases, this corresponds to its location. We also observe that in our dataset 81% of the links for these classes are not semantified. Rule mining techniques can help us learn the patterns suggested by Table 4.1 and semantify more links.

**Table of contents :**

**1 Introduction **

1.1 Motivation

1.2 Contribution

1.3 Preliminaries

1.3.1 RDF Knowledge Bases

1.3.2 Closed vs. Open World Assumption

1.3.3 Functions

1.3.4 Rules

1.3.5 Language bias

**2 Rule Mining **

2.1 Introduction

2.2 Related Work

2.2.1 Association Rule Mining

2.2.2 Inductive Logic Programming

2.2.3 Expert Rule Mining

2.2.4 Generating Schemas

2.2.5 Relational Learning

2.2.6 Learning Rules From Hybrid Sources

2.2.7 Further Applications of Rule Mining

2.3 Mining Model

2.3.1 Language bias

2.3.2 Measures of Significance

2.3.3 Measures of Correctness

2.4 The Partial Completeness Assumption

2.4.1 The PCA in real data

2.5 AMIE

2.5.1 Algorithm

2.5.2 Mining Operators

2.5.3 Confidence calculation

2.5.4 Query Implementation Details

2.6 Performance Experiments

2.6.1 Experimental Setup

2.6.2 AMIE vs. WARMR

2.6.3 AMIE vs. ALEPH

2.6.4 AMIE with Different Settings

2.7 Fact Inference with AMIE

2.7.1 Standard vs. PCA Confidence

2.7.2 AMIE vs. ALEPH

2.8 Conclusion

**3 Speeding Up Rule Mining **

3.1 Speeding Up Rule Refinement

3.2 Speeding up Confidence Evaluation

3.2.1 Motivation

3.2.2 Confidence Approximation

3.2.3 Computing the Approximation

3.2.4 Confidence Upper Bounds

3.3 Experiments

3.3.1 Experimental setup

3.4 AMIE+ Demo

3.4.1 Interface

3.4.2 Implementation

3.5 Conclusion

**4 Wikilinks Semantification **

4.1 Motivation

4.2 Related Work

4.3 Predicting Semantics for Wikilinks

4.3.1 Mining Semantification Rules

4.3.2 Predicting Relations for Wikilinks

4.4 Wikilinks for Rule Mining

4.5 Conclusion

**5 Schema Alignment **

5.1 Motivation

5.2 Related work

5.3 Method

5.4 Experiments

5.4.1 Setup

5.4.2 Simple Mappings

5.4.3 More complex patterns

5.5 Conclusion

**6 Canonicalization of open KBs **

6.1 Introduction

6.2 Related Work

6.2.1 Open Information Extraction

6.2.2 Linking and clustering entities

6.2.3 Clustering relations

6.3 Canonicalizing noun phrases

6.3.1 Mentions

6.3.2 Clustering

6.3.3 Similarity Functions

6.3.4 Combined Feature

6.3.5 Canonicalization

6.4 Canonicalizing verbal phrases

6.4.1 A semi-canonicalized KB

6.4.2 Rule Mining

6.4.3 Phrase Clustering

6.4.4 Canonicalization

6.5 Experiments

6.5.1 Entity clustering

6.5.2 Relation clustering

6.6 Conclusions

**7 Predicting Completeness in Knowledge Bases **

7.1 Introduction

7.2 Related Work

7.3 Preliminaries

7.4 Completeness Oracles

7.4.1 Simple Oracles

7.4.2 Parametrized Oracles

7.5 Learning Completeness

7.5.1 Combining Oracles

7.5.2 Enhancing AMIE

7.5.3 AMIE as Completeness Oracle

7.6 Experiments

7.6.1 Setup

7.6.2 Basic Completeness Oracles

7.6.3 Learned Completeness Oracles

7.7 Application

7.8 Conclusion

**8 Vision on Numerical Rule Mining **

8.1 Introduction

8.2 Numerical Rules

8.2.1 Numerical Constraints

8.2.2 Descriptive and Predictive rules

8.2.3 Functional notation

8.2.4 Evaluation Metrics

8.3 Existing Work

8.4 Conclusion

**9 Conclusion **

9.1 Summary

9.2 Outlook

**Appendices**