|Review Comment: |
Within this submission, the authors propose an ontology (the CSO) for the description of arbitrary PaaS, IaaS, and SaaS cloud services. In addition, mechanisms for the verification of this ontology and an evaluation (in terms of checking if the ontology is sound) is proposed. As far as I can judge, the technical content of this paper is sound, so I am not going to comment too much on it. However, the authors fail to show that there work advances the state of the art in a manner that allows it to be published in a top journal like the Semantic Web Journal.
The paper is mostly well-written and easy-to-follow. There is quite a number of missing or superfluous articles and some issues with sentence structures, but apart from that, the language is fine.
The compilation of the CSO ontology is based on accepted best practices. However, I am missing a discussion of alternatives. Just one example: Why are the non-functional properties limited to availability and security? In the conclusions, the authors themselves state that there are other properties which could be taken into account. Why has this not been done already? As another example, I do not see why API support has been categorized into low, moderate and high based on the number of APIs offered. Surely, this is not a good metric to say if API support is good or not. What if the APIs are not well-documented? What if there is only one API, but offering a huge number of different functionalities? In general, a scientific paper should not only state what has been done, but also why something has been done this way.
The biggest issue I have with this submission is its missing delimination to the state of the art. Indeed, there have already been a number of approaches to provide a unified vocabulary for cloud services. Some of these are discussed in Section 2 of this submission. However, the discussion remains very superficial: In my opinion, the actual ontology presented in the submission is not really interesting, simply for the reason that there have been attempts before to provide something like this - it's more like "yet another cloud ontology". Instead, the authors should sell way better the verification and correctness checks they present in their work. However, this makes it necessary that it is discussed how such tasks have been done in the state of the art - not only with regard to cloud ontologies but with regard to arbitrary ontologies. Only by showing such advancements, the paper gains enough credibility of publication in a top journal.
Also with regard to the discussion of the state of the art, the authors claim at the end of Section 2 that the related work is missing 6 different aspects. However, this is only discussed very superficial by stating "The majority of the current proposals", "Some approaches", etc. Instead, the authors should have provided a table, clearly showing which of the discussed existing approaches already offer the discussed functionalities. In addition, the in-depth discussion mentioned in the last paragraph needs to be done. If these issues are not resolved in a major revision to this paper, I will surely recommend rejection in the next round of reviews.
Amongst other aspects, the authors highlight that the proposed CSO deals with cloud federation properties, which is not covered by the related work. The latter might be true, but I clearly have to say that I do not see how this submission really takes care of specific aspects of cloud federations. Sure, there is a part of the CSO which models federations, but this aspect is later on never really discussed or used again. What are the specifics of cloud federation with regard to ontology design? Do we need additional interference rules for them? For example, I could imagine that the capabilities of a cloud federation could be inferred from the capabilities of the single cloud services which are used to build a cloud federation. This would surely be interesting.
As another critical aspect, the evaluation is by far not sufficient. In Section 6.1, next to no details about the evaluation settings are given, i.e., which and how many cloud services have been available for retrieval. In Section 6.2, I don't really see how cloud federation plays a role, since the search results are limited to single cloud services again.
In conclusion, the paper has surely some potential, but there is a lot of work to be done before it can be accepted in a top journal.
* Vector-based graphics should be used to increase the scalability of the figures.
* The reference work in Section 1 is very weak, since there is not a single reference in the whole section.