Review Comment:
The article describes Owl Monkey (OM), an application that supports naive ontology users to build views on an ontology. The application has been built for an industrial use cases in Hydro Quebec. The application is also positioned to be particularly useful in the context of an IoT application. Although the proposed application addresses an important problem, namely the support of domain experts who are naive ontology engineers, in creating queries on an ontology model, the paper in its current form is lacking in detail, scientific rigour and lacks an evaluation. In the following I detail on these issues:
- Lack of formal definitions: The paper lacks completely any formal definitions of terms and operations that are performed by the application on the TBox and ABox model. The authors mention that through "lazy class loading" (unclear what that means), the same set of generic queries can be used for ad hoc addition of a class to the solution set. What is the solution set and how can classes be added? From this statement it appears as if the tool allows for the creation of classes, instances and queries, but again, it is unclear what the user actually generates? To give some examples on how definitions are missing, the tool apparently distinguishes "subject classes" and "non-subject classes", but it is unclear what that means, formally. Then the authors mention that the "forms are built dynamically based on the visible columns of the grid and the column order" and "the nature of the required fields is determined by the ranges of the properties represented by these columns." What are the visible columns and how are they generated? It continues with a lack of definitions for "term patterns", "semantic grid", "view class", "column groups" etc. Consequently, it is totally unclear if the tool actually allows to build queries, classes and properties or even to create individuals, i.e. there is no clear distinction between the TBox and ABox in the query building, and what variables are created in these queries. The latter is also a consequence of not defining in the paper, what the user actually creates through the GUI. The authors claim that the proposed application is a Visual Query System (VQS), but what is created by the tool, SPARQL queries or maybe even SPARQL templates (SHACL or SheX)? Restrictions on properties seem to also only take domain range restrictions into account, but not single range existential restrictions (guarded class restrictions). Another example of a lack of detail is the statement that "it was necessary to develop contrivances to avoid redoing the inference whenever data was modified. This has allowed the use of forward chaining for all use cases encountered to date."
- Lack of positioning of the application in regards to state-of-the-art. Considerable space in the Literature Review is given to the introduction of standard concepts such as SPARQL, RDF, OWL, triple stores etc., but state of the art in the three areas the authors identified as methods directly related to the proposed application, visual query systems, form representations and facet representations is not discussed. A large body of work is available in the areas of form-based ontology editing (e.g. https://scholar.google.com.au/scholar?hl=en&as_sdt=0%2C5&q=form-based+rd...) and visual query systems (e.g. https://scholar.google.com.au/scholar?hl=en&as_sdt=0%2C5&q=visual+query+...). Only later in the paper, some tools that the authors consider related work are mentioned, [21, 22, 24, 25], but not critically discussed.
- Lack of Evaluation or Use Case Evidence: The paper has been submitted as a Full Paper and therefore the lack of an evaluation is by itself an exclusion criteria for publication. The paper does mention that the tool was built for Hydro-Québec, but it fails to report on results of its usage. It appears as if the tool has not been fully rolled out yet, and no evidence can be provided at this stage, that the tool indeed eases the burden of building ontology models. This evidence of how the application would benefit naive users in comparison to the state of the art ontology editors such as WebProtege is crucial for a publication of this type. A submission as an Application Report, that has less strict requirements on an evaluation, would allow the authors to detail on the impact of the described application in a use case, rather than a comprehensive evaluation.
In light of the aforementioned issues, and the fact that the journal operates under a strict two-strike-rule, I recommend a reject and encourage the authors to resubmit the work as an application report once they can report on some evidence from the use case on how the application has helped engineers in Hydro-Québec to build ontology views.
|