Review Comment:
The paper presents a system for question-answering over RDF KBs. It uses controlled natural language and a form of KB-driven auto-completion to avoid ambiguities in query interpretation and assist users in formulating, KB-compliant queries. To this purpose, user inputs are matched to RDF entities of the underlying graph and query formulation is seen as navigation over the latter, using an automaton for controlling the transition over states that realise patterns of queries.
The problem addressed is very interesting and is relevant to the SI scope. The approach is well-motivated and explained; moreover, the evaluation experiments are promising, as the presented system performs better than relevant state-of-the-art systems against which comparisons have been carried out. The related work is quite extensive; nonetheless, in the referenced surveys on NL approaches to QA in the introduction section, more recent publications could be added (e.g. “C. Unger, A. Freitas, and P. Cimiano. An introduction to Question Answering over Linked Data. Reasoning Web Summer School, pages 100-140, 2014.”); moreover, as the use of CNL and the KB-driven auto-completion are very closely to the idea of feedback and clarification dialogues presented in “D. Damljanovic, M. Agatonovic, H. Cunningham, K. Bontcheva. Improving habitability of natural language interfaces for querying ontologies with feedback and clarification dialogues. J. Web Sem. 19: 1-21 (2013)”.
However, there are some concerns with respect to considering the manuscript for publication in its current form. The first is that significant part of its contents can be found in two recent publications by the authors that haven’t been cited in the current submission: “Answering Controlled Natural Language Questions on RDF Knowledge Bases, EDBT 2016: 608-611” and “Answering End-User Questions, Queries and Searches on Wikipedia and its History, IEEE Data Eng. Bull. 39(3): 85-96 (2016)”. Moreover, as there is no advancement in the afforded question-answering expressivity capabilities (the same extensions are reported as future work in all), the contribution of the current manuscript is considerably weakened. Second, although in the evaluation experiments, datasets of the well-established QALD benchmarks are used, it is not clear why for the comparison with SWIPE only the QALD3 MusicBrainz dataset was considered and not a DBpedia one, either from QALD6 or even QALD4 for which a cited work reports that SWIPE is the most effective visual query system. Last, although the performance over the QALD datasets backs up the claim that the presented approach doesn’t appear to reduce the expressive power (since with few exceptions of unsupported features, the initial question can be rewritten in a CANaLI equivalent form), there is no end-user evaluation on how intuitive and practical it is to use the system, nor a comprehensive listing of the types of questions/constructions that are supported (e.g. queries that can be expressed using what-questions (“what is the age ...”) but not how-questions (“how old is...”). Also, a link to the online demo should be provided explicitly; now it can only be found by following the reference on the results of experimental evaluation for all considered benchmarks.
There are a few typos and sentences that would benefit from a proof reading; examples include: “The results obtained on each question the benchmarks are reported in [10]., “The results of that comparison Figure 10”, etc.
|
