RehabRobo-Query: Answering Natural Language Queries about Rehabilitation Robotics Ontology on the Cloud

The revised version of the paper has certainly made quite a few improvements over the first manuscript. Still some of my concerns have not been treated / answered in a satisfactory way, that’s why I am suggesting a minor revision.
Comment on domain-dependent vs. domain-independent:
The authors have answered that parsing RehabRobo-CNL is domain dependent. Maybe it is easier to treat it as a domain dependent task, but still I believe that many parts of the parsing are domain-independent, as the authors state in the manuscript (probably without realizing it).
For example:
- In page 7, left column, 4th paragraph they state: “The functions presented in italic font in Table 1 refine these queries by embedding relevant information from REHABROBO-ONTO.” This means that the functions are independent of the specific ontology, so if you change the ontology they will return information of relevant type from the new ontology.
- In the next paragraph: “For instance, only the verb “reference” can appear after the type Publications.” This is a direct relationship to rdfs:range property of an OWL object property. E.g. if the “hasReference” object property has as range class “References”, therefore the above restriction could be extracted automatically from the ontology, regardless of the REHABROBO ontology; this is true for any object property and any range class.
- Other places in the manuscript where it is evident that similar pieces of information can be automatically extracted from the ontology are:
o “Note that these matchings essentially come from the structure of REHABROBO-ONTO, e.g., concept names and relation names.”
o “Note that these matchings are due to properties of concepts in REHABROBO-ONTO.”
o “Further, the values for the nouns are extracted from REHABROBO-ONTO to allow suitable entries from the users.”
So, my comment is “Couldn’t you define (and describe) a generic procedure for extracting such information from any ontology and embed it into RehabRobo-CNL parsing procedure”?
Comment on reusing reference (BIBO/FABIO) and FOAF ontologies:
I accept the rationale of the authors. However, I would like to see the paragraph that justifies their modelling choice inside the manuscript, because all papers have also a didactic purpose. Since almost all ontology engineering methodologies suggest to try re-use existing ontologies, their justification on the opposite might be useful for other researchers to read.
Concerning the translation between CNL-to-SPARQL in either one or two steps, I understand the explanation given by the authors. However, having two different procedures for performing the same task raises the question whether these two procedure are equivalent, meaning that they always produce exactly the same results given the same input. This should be proved.
Concerning section 5.3, sure the new Table 8 improves things. However, I still do not understand why the authors do not provide an algorithm similar to the previous subsections. They do not justify this in their answer.
I also have some other (new) comments:
Page 2, right column, 2nd paragraph: “This article is concerned with reasoning about rehabilitation robotics over REHABROBO-ONTO.” Later on, in Section 3, 1st paragraph: “Reasoning over REHABROBO-ONTO can be done by means of answering questions posed by the user in a natural language.” I believe that the authors misuse the word “reasoning”. This word has a well-defined meaning in the world of ontologies and is different from “querying”. Therefore, I suggest to revise these phrases.
Section 3.1: The title “Sorts of queries supported by REHABROBO-CNL” would be better as “Types of queries supported by REHABROBO-CNL” or “Query types supported by REHABROBO-CNL”.
Section 3.1, 2nd paragraph: “We have identified different sorts of complex queries useful from the perspectives of rehabilitation robotics and physical medicine.” My question is how have these been identified? Through a survey on experts? Through informal conversations with experts? Through authors’ own experience? The source(s) of inspiration for these queries should be mentioned.
Section 7 (new): Very nice addition!
Section 7.2, 2nd paragraph: “Each of participant have been given…” should become “Each of the participants has been given…” or “Each participant has been given…”
Section 8, Related Work: The described related work is properly compared to authors’ own work.

The paper describes a method for expressing and answering natural language queries about rehabilitation robots. The method consists of a controlled natural query language (RehabRobo-Query), a translation of queries expressed in RehabRobo-Query into SPARQL queries making use of the terms of an ontology for rehabilitation robotics (REHABROBO-ONTO), and the use of a DL reasoner (Pellet) for answering the SPARQL queries. The paper also presents a query interface, which facilitates the formation of the natural language queries, and two experiments evaluating the translation algorithms and the language itself.

Compared to the original submission, the revised paper describes in more detail the domain of rehabilitation robotics. It also presents an analysis and and an experimental evaluation of the translation algorithms, and a user evaluation of the language using domain experts. Finally, it includes a more comprehensive presentation and comparison with related work.

Although the quality of the paper has been improved, the paper is in my opinion not yet ready for publication. Specifically:

- The paper still lacks realistic examples from the domain of rehabilitation robotics. You present several examples of queries but you only briefly present one use case where one of these queries could be useful (robots for patients with rotator cuff lesions). It would be useful to give some more similar examples or a use-case scenario, which would better motivate the need for the proposed query language. Or a better explanation of the cases where the types of queries that you present in 3.1 would be useful.

- You need to give more details about the evaluations you performed:
* How did you choose the set of queries you used for evaluating the performance of the translation algorithms? Which criteria did you use for their selection?
* What instructions did you give the participants of the user-based experiments? Were they asked to freely formulate their own queries or did you ask them to write queries with which they could retrieve certain information from the knowledge base?
* The format of the questions you present in Table 1 (e.g. “rate the usefulness,…”) does not match the format of the answers described in 7.2 (“strongly agree”, “strongly disagree”, etc.).

- With the survey questions that you used in the user-based evaluation, you were only able to evaluate the usefulness of the query language. It’s not clear though whether the participants found the language and the interface intuitive and usable, which is equally (if not more) important. Were the participants able to form the queries they wanted to make? did their queries return the expected results? did they give any feedback on how the language/interface can be improved?

- Although the aim of the paper is not to present the ontology itself, it would be useful to give some more information about it, such as: some examples of subclasses of the main classes that you present, some examples of instances of the main classes (e.g. of Assessment) which would better clarify their use, the features of OWL that you used.

- There are still some language errors/typos throughout the paper that need to be fixed. Below are some examples - but there are probably more, so you need to proof-read the paper again or ask a native English speaker to proof read it for you:
* Section 1, line 2: remove the duplicate “of”
* Section 1, par.3: “Robot-assisted rehabilitation devices can be applied to patients” -> clarify what you mean by “Robot-assisted rehabilitation devices”; and how can these be applied to patients? another verb would be more appropriate here, e.g. “can be used to support patients”
* Section 1, par.4: “it become a challenge” -> “it has become a challenge”
* Section 1, par.5: “these critical information” -> “this critical information”
* Section 1, par.6: “is likely be critical” -> “is likely a critical”
* Section 7.1, par.1: “as listed” -> “listed”
* Section 7.1, par.2: “after high population” -> “after increasing the size of”
* Section 7.2, par.2: “Each of participant have been given” -> “each participant has been given”
* Section 7.2, par.3: “the factor of occupation” -> “occupation as a factor”
* Section 8, par.4: “a more variety” -> “a greater variety”
* Section 8. par.5: “on the top of Sesame” -> “on top of Sesame”
* Section 8, par.7: “other varieties of queries” -> “other types of queries”
* Section 9. par.2: “right rehabilitation robots” -> “appropriate rehabilitation robots”
* Section 9, par.2: “aids exchange … and thus to improve” -> “aids the exchange … and therefore improves”

- You also need to check your references. For example, refs [1] and [2] are missing the author names and in ref [12] the names of the authors are not clear.

Overall, the paper has some original contributions the most important of which is the evaluation of the proposed query language, but it still has some (mostly presentation) flaws. After a major revision addressing the points above, the paper could be considered for publication.

This manuscript was submitted as 'full paper' and should be reviewed along the usual dimensions for research contributions which include (1) originality, (2) significance of the results, and (3) quality of writing.

My previous comments about originality, significance of the results and quality of writing have been successfully addressed. The final document is well structured and easily followed. I just refer some minor recommendations, mainly lexical and syntactic:
*Neurological injuries, such as stroke, are the leading cause of of permanent-> delete the second “of”
*welfare of the patients-> welfare of patients
*while negatively impacting national economies->while negatively impact national economies
*In particular, the use of robotic devices in repetitive and physically involved rehabilitation eliminate the physical burden of movement therapy for the therapists, enable safe and versatile training with increased intensity, increase the reliability, accuracy, and effectiveness of traditional -> it's better to use numbering for separating the different items because it's difficult to follow the line of issues discussed
* Robot assisted rehabilitation -> Robot-assisted rehabilitation (remain consistent throughout the document using “-“)
* but commonly in unstructured forms -> stored (?) commonly in unstructured forms
* such as text in relevant publications -> not clear what you mean here
*is likely be critical step-> is likely to be a critical step
* REHABROBO-QUERY also utilizes Amazon Web Services for cloud computing->not clear what you mean? please define which cloud computing services you mean
*as the ontology REHABROBOONTO-> as the one provided by the REHABROBOONTO
*After we summarize -> When/Once we summarize
*algorithms and rules, and by providing evaluations -> algorithms and rules, evaluations
* by first identifying the purpose, and then the basic -> split it in a different sentence e.g. First, we
* REHABROBO-ONTO is developed using -> I would prefer to see present perfect in this sentence (but also at the whole document if possible)
*Here are some examples:
– What are the robots that target shoulder movements and that have at least 210 RoM for the flexion/extension movements of the shoulder? -> Which are the robots that target shoulder movements and have at least 210 RoM for the flexion/extension movements of the shoulder?
I would change in questions all “what” with “which” and erase the second "that"
* with (some intervention time or with all targeted disorders)? -> no need for brackets
* freedom >=2 -> it's better to write it as "greater than" instead of using the symbol (please do the same to the whole document and keep the symbol in brackets)
* These are queries to extract information about publications about rehabilitation robots, owners/institutes/laboratories of the robots -> These are queries to extract information about publications, owners, institutes and laboratories of the robots
* What are the robots with no targeted disorder or (with active degree of freedom>1 and with control modes=’{active,assistive}’ and with no disorder level)? -> not so clear
*Now we have an overall understanding... let us -> Having an overall understanding ... let us now
* something goes wrong with the row of tables inside the whole document
* User-Interface of REHABROBO-QUERY for Query Answering: Intelligent and Interactive -> Intelligent and Interactive User-Interface of REHABROBO-QUERY for Query Answering
* while also including links to detailed -> while also includes links to detailed
*nodes in the ontology, -> nodes of the ontology,
*transformation is linear time in the size of the query -> not so clear
*thus note that it is independent -> thus it is independent
*as further resources are necessitated by a larger ontology size. -> I would suggest to rephrase it, e.g. the ontology is further populated and increases in size
*fits the needs of the people -> fits different needs of people
* Along these lines, we have conducted -> I would suggest to change the "along these lines"
*introduced an interactive intelligent user-interface ->introduced an interactive and intelligent user-interface