Online Incremental Learning Based on Crowdsourcing For Indonesian Ontology Relation Extraction

Abstract

Ontology is one form of structured representation of knowledge. Ontology is widely used and developed in information retrieval because of its ability to represent knowledge in a form that machines and humans can understand. With the increasing scale and complexity of ontology, there are more significant challenges in identifying extra-logical errors. Ontological development methods mostly use machine learning, which is at risk of missed extra-logical errors. To handle it, crowdsourcing is used, i.e. dividing a large job into several small jobs and hiring the masses to complete it. Data processing is usually done offline to take advantage of crowdsourcing, and batches are converted into online and incremental. Online incremental learning directly arranges an iterative model after a change is made by ensuring that the knowledge that has been obtained before is maintained. This study built an interactive medium to present the initial relationship between concept pairs. Crowdsourcing participants were asked to validate the relationship repeatedly until a specified accuracy value was reached. This study found that the crowdsourcing process was able to improve the model used in the relationship extraction process, from F1-Score 87.2% to 89.8%. Improvements using crowdsourcing achieve the same result as improvements by experts. Thus, crowdsourcing can correct extra-logical errors appropriately as an expert. In addition, it was also found that offline incremental learning using Random Forest resulted in higher model accuracy than incremental online learning using Mondrian Forest. The accuracy of the Random Forest model has a final accuracy of 90.6%, while the accuracy of the Mondrian Forest model is 89.7%. From these results, it was concluded that incremental online learning cannot provide better results than offline incremental learning to improve the meronymy relationship extraction process.