WebProtégé: A Collaborative Ontology Editor and Knowledge Acquisition Tool for the Web
This is a revised manuscript following an accept pending minor revisions, which has now been accepted for publication. The reviews of the original submission are below.
Solicited review by Gianluca Correndo:
The paper presents WebProtege, a distributed ontology editor that allows communities to create and discuss ontologies using a complex web enabled interface. The tools is supported by an ontology for describing ontology evolution (i.e. the Changes and Annotation Ontology, or ChAO).
The tool is quite mature and already used in a number of scenarios in biomedicine where its features have been evaluated in deep within the biomedical domain. The use case reported is the development of the ICD-11 which is extensevely described in three further publications referenced in the paper. It would be beneficial to stress better the contribution of this paper from the past WebProtege literature.
The authors mentioned in the related work only two tools for collaboratively editing ontologies: knoodl, and ontowiki. Other tools like neologism  or moki  are not mentioned, is there a rational behind it? If so, can this rationale be expressed in the paper?
The tool is available on line both to use and to download and there is plenty of documentation that can guide first users. The paper is clear about the features provided by the tool and about the rationale behind future work.
Solicited review by Rinke Hoekstra:
This paper describes the WebProtege ontology editor that allows collaborative editing of ontologies through a GWT-based browser interface built on top of the collaborative protege infrastructure.
WebProtege clearly is the most advanced and mature collaborative ontology environment that currently exists, and has seen significant adoption by ontology developers in both industry and academia. The paper adequately describes its features, but does not dive very deeply into the technical details. This is mainly because many of its components have been previously published in other papers. However, it is a pity since a tools and systems submission to SWJ is one of the few places where authors can emphasise the perspective of technology rather than pure research.
I do have a couple of more detailed comments:
* The claim in the first sentence of section 1 needs to be supported by a reference or explanation/example.
* In section 2 you discuss related tools, and rightly note that the current emphasis is on ontology-based data-entry, rather than ontology development. However, this has not always been the case. Most web-based tools developed in the nineties did focus on ontology development (Ontolingua, WebODE, etc.). In my view, it is this branch of tools that WebProtege inherits from rather than the currently popular wiki-style data-entry tools. Furthermore, you contrast WebProtege with desktop applications that provide similar features, but do not really make a case for why the fact that WebProtege can run in a browser is such a big plus compared to these other tools. Just the fact that users do not need to install an application on their machine is surely not the only reason why you decided to develop WebProtege, is it? A comparison with the advantages of WebProtege over Collaborative Protege would be helpful here.
* Section 2, typo: "A number of collaborative ontology editors are, but they do now work in a web browser..." -> ".... is, but they do not work..."
* In section 3.1.1 you mention that users find the Properties View portlet very intuitive, and provide a footnote explaining that domain and range are often misinterpreted. Given the long history in Protege in emphasising intuitive and unambiguous ontology editing, do you think it is wise to support this view, if most users will misinterpret it?
* Section 3.4 states that you only provide a Protege 3.x-based service interface, and mention that developers "can easily add different back ends". Section 4 says that you have an OWL API implementation that is almost ready to publish. Perhaps it's good to mention this in section 3.4? Also, it is unclear from your description *how* difficult it is to add a different back end, given that from figure 3, even an OWL API backend will have to communicate with the (legacy 3.x) Collaborative Protege framework. Also, an explanation of why you still support Protege Frames would be nice. I have heard the argument that people find it 'easier' or more intuitive, but do you, as developers still believe in it?
* Section 6. I find your discussion of the usability studies a bit terse. Could you add some more details about the type of setup, and results? Also the web-based usability questionnaire provides very high scores, but these are not very convincing. What population filled in the questionnaire? On what basis (what experience) did they fill in the questionnaire? Were the respondents long time users, or did they just have a 2 minute glance at the demo?
I would like to see (even in a tools/systems paper) a more thorough discussion of why WebProtege is needed, and how it solves actual needs 'out there'.
* Section 7. After all this, I wonder what the basis is for you calling WebProtege a 'lightweight' ontology editor and knowledge-acquisition tool. Why is it lightweight? Because of GWT? It seems highly feature packed, and apart from the reasoning capabilities, just as 'heavy' as Protege 4.
In short, this paper is a good, but a bit shallow discussion of some excellent work.
Solicited review by Philipp Frischmuth:
This paper describes an adaption of the well-known Protégé ontology editor for the Web. The authors motivate the need for such a tool with an increasingly distributed and collaborative ontology development process. They have chosen a portal approach, since Web users are familiar with such tools. The user-interface employs the Google Web Toolkit (GWT) and the back end components are based on existing Protégé infrastructure. The authors provide a link to an online demonstration of the tool, which is publicly available.
The paper is well organized and the structure is in my opinion appropriate for a tool/system report. The authors start with a short introduction where they describe the motivation for WebProtégé. In section two they briefly present some related tools. A small remark regarding OntoWiki is, that the authors should have a look at . OntoWiki was developed with extensibility in mind. In fact a lot of the basic functionality is also realized as an extension. Thus I don't agree that WebProtégé is the only available Web-based knowledge engineering tool. The third section of the paper gives a detailed insight into the functionality of the tool. This is the longest part of the paper and in my opinion appropriate for this type of paper. It is followed by a shorter section, which describes the technical aspects of the architecture.
The authors also describe some real-world use cases where WebProtégé was used and adapted. The focus here is put on ICD-11 (11th revision of the International Classification of Disease). The paper is closed with a discussion as well as an outlook on future works and a short conclusion.
Since WebProtégé is based on the well-known Protégé tool, an adaption towards a Web-based solution is reasonable and the paper is in my opinion well written I would recommend to accept the paper.
Solicited review by Valentin Zacharias:
The paper "Web Protégé: A Distributed Ontology Editor and Knowledge Acquisistion Tool for the Web" presents the WebProtégé system - an extension of the well known Protégé Ontology Editor that enables collaborative ontology editing and knowledge acquisition on the web with a strongly customizable user interface. The paper presents the system and its motivation, related tools, system overview, architecture, use cases and future work.
WebProtégé is an important, relevant, interesting and innovative system. The paper is well structured, good to read and covers all important aspects of the system (as good as can be expected for such a complex system and such a short paper). No new scientific result is presented in the paper, but for a tool paper this is fine.
In my opinion the paper should be accepted with two minor revisions (see below).
The title calls WebProtégé an "Distributed Ontology Editor" - As it is a client-server system I don't think this is an appropriate or useful term (the authors also don't call it distributed anywhere else). I think the title should be "A collaborative ontology editor … " or "Web based" or similar.
Page 8 bottom left: "Stojanovich" should be Stojanovic
Related Work: I would have also expected the authors to mention the Web based skos editors such as PoolParty or our own Soboleo (which is actually mentioned in , which falsifies the statement by the authors that tools in  require an installation on a user's machine).
Page 4, middle of left column: "The customized tool presents the domain experts with simple forms familiar from other Web-based application, without being really aware that they are, in fact, editing instances of an ontology" - shouldn't this be: "The customized tool presents the domain experts with simple forms known from other web-based applications, without them being really aware that they are, in fact, editing instances of an ontology" (also note the added "s" for application")
Discussion and Future Work: To me it seems that WebProtégé with its modern architecture will become the new protege (once facilities to handle all reasoning on the server side are developed) and that very soon further development of the classical editing tool of Protégé will stop. I think many readers will think the same, hence I would appreciate some notes by the authors on whether this is the case or why not (offline functionality might by an easy answer, but internet access is becoming more ubiquitous by the day and it is getting simpler to build web applications that work offline - in the long run this is probably much cheaper than maintaining two independent tools).