Modular Ontology Modeling

Manuscript: Modular Ontology Modeling Contribution

This manuscript presents a method, supported by instrumental software tool, for modular OWL ontologies development which reads as substantially inspired by the approaches known from Conceptual Modeling. I am happy to welcome this approach which is indeed more effective and efficient than more traditional mainstream practices in Ontology Engineering.
Unfortunately, the manuscript is vague in stating what are the contributions, compared to the background work by the same authors. I could only note, that the manuscript mentions MOMo, CoModIDE, and a summary of past or current experiences, as presented in te paper. However:
-There is nothing really new in MOMo (as presented) in terms of being an ontology engineering methodology. many teams, including my team, used very similar approaches in their ontology engineering practices before, perhaps not including all the ingredients recommended by MOMo.
-CoModIDE and its usability evaluation might have been a nice contribution, but (i) the number of subjects is quite small to be reliably regarded as representative; and (ii) this study has already been reported in [16]
-Experiental presentation reads like a (patchwork) summary compiled from the previously published papers by the same authors.
One may argue that, cumulatively, these three parts combine in a solid contribution. However the authors do not state that in the manuscript.

Extension and Refinement

The authors claim that this manuscript “... significantly extends [16] and summarizes several other workshop and conference papers: [17], [18], and [19].” Extension needs to be presented in a more explicit way, however. I checked [16] in order to find out what was the
extension. Unfortunately, I did not find anything substantial in this manuscript (Sect 4), except Sect
4.5 CoModIDE 2.0, that extends the results already published in [16], in my humble opinion. Evaluating usability and acceptance of CoModIDE 2.0 would have been a logical follow-up. The manuscript does this not, however.

Review of the Related Work

I am basically happy with the provided review of the related work as all the aspects have been sufficiently covered. I am not fully satisfied with the analysis of the gaps in the related work and the way the contributions by the authors bridge these gaps. There are several relevant mentions in the review, but those are scattered throughout the section. A summary table or figure presenting the advantages of MOMo + CoModIDE would have been really beneficial.

Novelty

Frankly, I am not happy with the novelty as presented. Please refer to my comment on the contribution.

Quality and Rigor

Overall, the quality of the presented research results is acceptable. I also liked the degree of statistical rigor in the presentation of CoModIDE usability evaluation. I was not happy, however, about the quality and rigor in the presentation of MOMo and Experiental survey. The discussion of

MOMo is too much prescriptive and declarative, in my interpretation. It does neither really argue for this or that methodological choice, nor explain its utility, compared to the other possible choices. The presentation of practical experiences does not look like a harmonized and convincing story aimed at proving a hypothesis. It is rather a compilation of what has been previously published.

Soundness of Evaluation

Overall, this facet of the presented work is OK. However, as noted above, this is not a new result.

Clarity, Structure, and Language

Overall, the paper is clearly written and sufficiently immustrated. The structure follows the proper pattern. There several flaws, however, regarding this aspect. The introduction is too long, so it makes the manuscript unbalanced. Figures are of insufficient quality or too small – hence, hardly readable. Some references seem to be wrong.

Anticipated Value and Interest

This is the hardest aspect to elaborate, in fact. To make it clear, I supposed that all the flaws mentioned in the review above, are removed / reworked / solved in a way. Then, I would be happy to state that this paper will definitely have substantial impact on the discipline. In its current state, I would rather refrain from stating this. The same is valid for the anticipated interest (and potential citations). I would not cite this paper if published in its current shape.

Focused Comments

A number of the focuused comments, questions, and suggestions are given as Acrobat comments and colour mark-up in the attachment to this report.

Suggestion on Acceptance

Potentially, this is a good submission which might be accepted after a major revision if there is a room, in my opinion. The flaws mentioned in the review are surmountable. Quite some work needs to be done for it, however.

Supplementary material can be found here: https://drive.google.com/file/d/1IF9eXGFJ-CVaadYr2ESZ-AqiP87UUOe3/view?u...

This manuscript presents the Modular Ontology Modeling methodology (MOMo) and CoModIDE, an IDE that supports it. It summarizes and significantly extends earlier publications by the authors. Section 1 informally establishes requirements for MOMo by discussing four issues that prevent ontology reuse, and it introduces the basics of modular ontologies and ontology design patterns. Section 2 reviews works that are related w.r.t. dimensions that cover the research space well: traditional ontology engineering methods, ontology design patterns (ODPs), eXtreme Design being a methodology based on ODPs, other pattern-based methods, and graphical conceptual modelling approaches. Section 3 introduces MOMo in detail. Section 4 presents CoModIDE's tool support and a comprehensive evaluation of its usability. The short Section 5 presents a few companion resources to CoModIDE. Section 6 informally reports on experience gained from applying MOMo in the engineering of various ontologies. Section 7 concludes and provides an outlook to future work.

Overall, the manuscript provides evidence that MOMo and CoModIDE do their jobs well. There are just several minor issues that need fixing but will be relatively straightforward to fix: unjustified claims, imprecise statements, as well as minor spelling and grammar issues. A discussion of the most serious issues follows, by descending severity. For further issues, please find attached an annotated PDF.

* You say that the object(s) of a functional property "can have at most one URI". In OWL these could actually be multiple URIs, but all of them would be assumed to be owl:sameAs, i.e., different names for the same thing.
* "[graphical] representations are sometimes ambiguous" – I would hope that every individual graphical notation that third parties have specified for OWL is _not_ ambiguous. Similarly, it should be explained more precisely in what way "unlike other OWL generators, SDOnt does not give a strictly disambiguous diagram".
* "Avoid partial overlap of module groupings (grey boxes) in the diagram, even if modules are in fact overlapping." – please explain how overlaps should be modeled instead. By duplicating class nodes?
* Related to the previous point: you recommend that in some cases diagrams should have a class depicted more than once. In what diagram language is this common practice or even permitted? (Please name examples!)
* For "established practice" and "proven to be useful" in Section 3.6.8, do you have references that provide evidence?
* In "[y]our experience, it is best to focus on understandability of the diagram [17]. The following guidelines […]" – do you have (maybe in [17]?) scientific evidence for these guidelines?
* Regarding CV3, why is familiarity with Manchester Syntax relevant for CoModIDE?
* "the provenance module depicted in Figure 9 was derived from the pattern depicted in Figure 4". Why and how? With what objectives and use case in mind?
* You found that "in many cases, 10–12 sufficiently different competency questions will be enough". How does this compare to what others have found out? (Cite literature!)
* You might want to refer to some more practical examples: regarding structural tautologies, you might want to refer to the similar semantics of schema.org's domainIncludes and rangeIncludes. For intentionally not specifying domains, Dublin Core is a well known example.
* I wonder to what extent you can really promise that CoModIDE's "graphical rendering [is] consistent across […] Protégé versions".
* "We are […] utilizing the relatively short list of 17 such axioms" – it's not clear whether this refers to 17 concrete axioms in your ontologies, or to 17 general axiom _patterns_.

Annotated PDF: https://drive.google.com/file/d/1MXfTsC3FqudF2RgZqaPKxS5VKPXXr6gO/view?u...

The paper “Modular Ontology Modeling” presents the Modular Ontology Modeling (MOMo) methodology on the one hand (as an abstract process or workflow) and the Comprehensive Modular Ontology IDE (CoModIDE) on the other. CoModIDE is a tool/implementation – realised as a Protégé plugin – that plays a crucial role when fully implementing the MOMo workflow in a concrete ontology development project. The MOMo workflow comprises the following ten steps: 1. Describe use cases and gather possible data sources; 2. Gather competency questions; 3. Identify key notions for the domain to be modelled; 4. Identify existing ontology design patterns to be used; 5. Create schema diagrams for modules; 6. Set up documentation and determine axioms for each module; 7. Create ontology schema diagram from the module schema diagrams; 8. Add axioms spanning more than one module; 9. Reflect on entity naming and all axioms; 10. Create OWL file(s).

The article consists of the following sections: 1. Introduction (three pages), 2. Related Work (four pages), 3. The Modular Ontology Modeling Methodology (seven pages), 4. CoModIDE (seven pages), 5. Additional Infrastructures and Resources (one column), 6. Experiences (three pages), 7. Conclusion (one page). The two core parts of the paper are Section 3, which describes the different steps of the MOMo methodology in sometimes abstract, sometimes concrete terms, and Section 4, which describes the CoModIDE editor plugin. Section 4 also includes an exhaustive evaluation of the developed tool, which is described on a total of almost five pages.

The paper starts out with the aspect of reusing ontologies, which is, according to the authors, a difficult task in itself and only done rarely. This choice is interesting since this aspect of reusing of ontologies is becoming less and less important the more the paper progresses. It’s also not really clear how MOMo addresses this issue – probably through the use of Ontology Design Patterns but that’s not really MOMo-specific. Furthermore, this reviewer would’ve liked to see some empirical evidence with regard to the claim that ontologies are rarely reused.

On page 2 (lines 5-8) the authors claim: “using a fine-grained ontology for a use case that requires only coarse granularity data is unwieldy due to (possibly massively) increased size of ontology and data graph.” This is certainly true, but what about the simple solution of deleting the detailed bits and pieces from the existing ontology that are not needed in the new application scenario?

Also on page 2 (right column, lines 29-35), the different steps that can be performed to reuse an ontology deserve some more explanation because these different ways of approaching an existing ontology and making it work for one’s own use case can be very valuable information, especially for less advanced ontologists. At the same time, I was wondering when “keeping track of the provenance” (line 39) may become important in a certain reuse scenario or is it relevant for all types of reuse?

“Modules” are an essential concept in MOMo. On page 3 (left column, line 20) an example is mentioned: “cooking recipe”. Is this a good/representative example for such a module? Shouldn’t “ingredient” be the right module or perhaps “instruction”, maybe “prerequisite” – out of which “cooking recipe” is being constructed? Also, “cooking recipe” is conceptually very close to the well-known text type of the same name. This reviewer finds this example unnecessarily confusing. In addition, I’d like to suggest to provide some more examples, especially examples that contrast good modules with bad ontologies.

Also (line 33-34), the statement “modules, in this sense, indicate a departure from a more traditional perspective on ontologies, where they are often viewed as enhanced taxonomies.” This should be explained in more detail.

Overall, while Section 1 presents various good and certainly correct observations, this part is quite long and also vague. Concrete examples need to be presented earlier, I also suggest to significantly condense the section.

Section 3 is one of the two core parts of the paper. It’s probably due to the abstract nature of the paper’s overall topic that, again, quite a few abstract and vague terms and phrases are used (“strong, versatile modular ontology”, “it is advisable that …”, “establish a good modular ontology” etc.). These are quite subjective statements and should, from this reviewer’s point of view, be used only scarcely. Alternatively, one could attempt to provide solid definitions of these terms, which is, of course, challenging.

With reference to page 10 (left column, lines 33-45) I was wondering the following: if modules, as the paragraph implies, are completely arbitrary, why make use of the concept of “modules” at all?

On page 11, the ten steps are presented. My suggestion would be to present these steps in the form of a table or figure and to also include relevant key information, for example, the respective output of the step, any prerequisites, the medium involved etc. Such a compressed overview would’ve made the paper clearer and it could help to remove some of the rather fine-grained textual descriptions.

Section 3.6.3 (“Identify key notions for the domain to be modelled”) is, at least to this reviewer, one of the key sections of the paper. This section would benefit from a more thorough description.

Section 4 is the second core part of the paper, it presents CoModIDE. Here, an obvious question with regard to the statement “Per MOMo, the formalization of the developed solution into an OWL ontology is carried out after-the-fact, by a designated ontologist with extensive knowledge of both the language and applicable tooling.” (page 15, right column, lines 3-6): if an expert is needed at the end anyway, is all the effort of following the MOMo process really worth it? Can this trade-off (quick and dirty ontology done by an expert vs. thoroughly developed MOMo ontology) be measured somehow?

Also on page 15 (line 38), this reviewer was surprised to see that the built-in ODP repository is not the very first item in that list.

With regard to the evaluation (Section 4.2), there are several questions: concerning H1, who are the actual intended users of the tool – experts? Also, I think the relationship of this specific evaluation with the overall methodology needs to be stressed in a more thorough way (which of the ten steps does the evaluation refer to?). One of my most crucial questions (Section 4.2.3): the participants of the study were given the two schema diagrams shown in Figures 11 and 12, correct? These two schema diagrams explicitly mention several design patterns: Provenance, Event, Trajectory, ExplicitTyping. Magnifying Figure 10 (the CoModIDE screenshot) quite a bit, these design patterns are actually listed in the tool itself while in standard Protégé they are not listed. Having said that, I don’t think it’s a big surprise that the participants could solve the task faster and more efficiently with CoModIDE than with standard Protégé. In one tool the needed building blocks are presented in a menu, ready to be used, in the other tool they aren’t. I don’t think this aspect is addressed anywhere in the evaluation but it obviously should be.

With regard to Section 6, this reviewer is not really sure how helpful these “experiences” really are. If they are mentioned at all in the paper, then maybe also include additional examples, code, schemas, diagrams – something tangible.

While reading the paper this reviewer constantly thought that the topic of the article is, by all means, interesting and relevant for the journal but maybe a journal article is not the right text type to bring these different and also abstract notions as well as collections of best practices across. It’s absolutely obvious that the authors have decades of experience designing ontologies and that many additional examples could have been mentioned but the available space of a journal article is, at the end of the day, limited. My feeling is that a (significantly) extended version of the paper with many more examples and practical examples could also make a good textbook or perhaps an interactive online resource or university course. In fact, some of the more practical topics of the paper are so crucial that it may even make sense to explore them for a potential OWL2 standardisation track, should this ever happen.

Typos I noticed (there are more):

-Page 4, left column, line 17 (“is”)
-Page 6, right column, line 11: Please include, ideally for all citations, the full author name instead of only the reference in square brackets to make the text flow a bit more naturally.
-Page 16, left column, line 11 (“will need be”)
-Page 16, left column, line 18 (“an a graphical”)
-Page 17, left column, line 17 (“each the steps”)
-Page 22, left column, line 23 (“a general patterns”)
-Page 22, right column, line 33-36 (there’s something wrong here)