Cloud Service Description Ontology: Construction, Evaluation and Interrogation

Tracking #: 1522-2734

Khouloud Boukadi
Molka Rekik
Hanêne Ben-Abdallah
Walid Gaaloul

Responsible editor: 
Freddy Lecue

Submission type: 
Full Paper
Cloud federation systems have recently emerged as a scalable delivery model that interconnects services from two or more cloud providers for load balancing and accommodating spikes in demand. One challenge in this delivery model is the complexity of service selection due to the heterogeneity of cloud service descriptions among cloud federations. To overcome this complexity, it is crucial to uniform cloud service descriptions. Towards this end, we propose a Cloud Service description Ontology (CSO) that will be modeled based on concepts from cloud standards. The proposed CSO covers functional and non-functional capabilities of infrastructure, platform and software services' providers. To populate CSO, we defined a set of semantic mapping rules to collect instances from cloud providers' web pages. In addition, to ensure the high quality of CSO, we propose an evaluation approach that detects and corrects consistency, redundancy and incompleteness errors. Furthermore, to show the correctness and inference ability of CSO, we present an experimental evaluation that measures the precision and recall ratios of querying CSO.
Full PDF Version: 

Major Revision

Solicited Reviews:
Click to Expand/Collapse
Review #1
Anonymous submitted on 18/Jan/2017
Major Revision
Review Comment:

Within this submission, the authors propose an ontology (the CSO) for the description of arbitrary PaaS, IaaS, and SaaS cloud services. In addition, mechanisms for the verification of this ontology and an evaluation (in terms of checking if the ontology is sound) is proposed. As far as I can judge, the technical content of this paper is sound, so I am not going to comment too much on it. However, the authors fail to show that there work advances the state of the art in a manner that allows it to be published in a top journal like the Semantic Web Journal.

The paper is mostly well-written and easy-to-follow. There is quite a number of missing or superfluous articles and some issues with sentence structures, but apart from that, the language is fine.

The compilation of the CSO ontology is based on accepted best practices. However, I am missing a discussion of alternatives. Just one example: Why are the non-functional properties limited to availability and security? In the conclusions, the authors themselves state that there are other properties which could be taken into account. Why has this not been done already? As another example, I do not see why API support has been categorized into low, moderate and high based on the number of APIs offered. Surely, this is not a good metric to say if API support is good or not. What if the APIs are not well-documented? What if there is only one API, but offering a huge number of different functionalities? In general, a scientific paper should not only state what has been done, but also why something has been done this way.

The biggest issue I have with this submission is its missing delimination to the state of the art. Indeed, there have already been a number of approaches to provide a unified vocabulary for cloud services. Some of these are discussed in Section 2 of this submission. However, the discussion remains very superficial: In my opinion, the actual ontology presented in the submission is not really interesting, simply for the reason that there have been attempts before to provide something like this - it's more like "yet another cloud ontology". Instead, the authors should sell way better the verification and correctness checks they present in their work. However, this makes it necessary that it is discussed how such tasks have been done in the state of the art - not only with regard to cloud ontologies but with regard to arbitrary ontologies. Only by showing such advancements, the paper gains enough credibility of publication in a top journal.

Also with regard to the discussion of the state of the art, the authors claim at the end of Section 2 that the related work is missing 6 different aspects. However, this is only discussed very superficial by stating "The majority of the current proposals", "Some approaches", etc. Instead, the authors should have provided a table, clearly showing which of the discussed existing approaches already offer the discussed functionalities. In addition, the in-depth discussion mentioned in the last paragraph needs to be done. If these issues are not resolved in a major revision to this paper, I will surely recommend rejection in the next round of reviews.

Amongst other aspects, the authors highlight that the proposed CSO deals with cloud federation properties, which is not covered by the related work. The latter might be true, but I clearly have to say that I do not see how this submission really takes care of specific aspects of cloud federations. Sure, there is a part of the CSO which models federations, but this aspect is later on never really discussed or used again. What are the specifics of cloud federation with regard to ontology design? Do we need additional interference rules for them? For example, I could imagine that the capabilities of a cloud federation could be inferred from the capabilities of the single cloud services which are used to build a cloud federation. This would surely be interesting.

As another critical aspect, the evaluation is by far not sufficient. In Section 6.1, next to no details about the evaluation settings are given, i.e., which and how many cloud services have been available for retrieval. In Section 6.2, I don't really see how cloud federation plays a role, since the search results are limited to single cloud services again.

In conclusion, the paper has surely some potential, but there is a lot of work to be done before it can be accepted in a top journal.

Further comments:
* Vector-based graphics should be used to increase the scalability of the figures.
* The reference work in Section 1 is very weak, since there is not a single reference in the whole section.

Review #2
By Paola Grosso submitted on 30/Jan/2017
Major Revision
Review Comment:

I read with this interest this article that focuses on the modelling of could services in a semantic manner. This is an important are of research and ontologies such as the CSO will prove, in my opinion, essential for services offering and integration at the various levels. My own work in the context of the international OMN collaboration and the OMN ontology shows the need and the effectiveness of such approaches.
Still, I found that the article needs revision and strengthening to convince me fully of the originality and solidity of the modelling effort, and ultimately of the actual usability of the ontology.

In essence my impression along the three dimensions I am asked to review us the following:
- originality: just sufficient, as it needs to be better supported
- significance of results: limited as there are no real ‘results’ to
- quality of writing: good.

In the following I will provide my comments and suggestions across a number of topics related to your article.

**Federation **
The main objection I have to make is about the actual coverage of federative support in your ontology. In the Introduction you state that your CSO specifies knowledge shared among cloud providers in a federation. Federation among providers requires specific agreement and policies, e.g. under which condition a provider can use resources and services from another, and to which level of automation and scaling this can happen. I do not see clearly where the CSO addresses this. This needs to be clarified much better. It seems that you very narrowly consider federation the capability of one provider to inquire about the services present in another (bullet #4 on page 3). This is not therefore supporting actual federation of cloud services, but simply the offering of a centralised database/knowledge base of individual services.
Figure 2 shows you CloudFederation class and a short text before describes its elements. This is insufficient to assess the strength of your modelling to support federation.
Can you provide instantiation example of this class, and information on how you actually derived the appropriate federation model?

** SaaS and PaaS **
In the related work section you state that most of the existing work focuses on IaaS systems. Your CSO shows you define a Service class, which in turn has PaaS and SaaS subclasses. From what you present the modelling effort for PaaS and SaaS is so limited that I would argue you have not really tackles either this aspects strongly. Can you better show this?

**Security aspects **
Significant work has occurred in the community in regard to security and compliance. The current description of the security aspects in the article are fairly weak and seem like a superficial handwaving to security measures required to assess service levels. I suggest you look at cloud compliance and self-assessment by CSA:
• Information Supplement: PCI DSS Cloud Computing Guidelines (February 2013) (
• CSA Consensus Assessment Initiative Questionnaire (CAIQ) v3.0.1
It would be interesting to see how much you ontology derives, extends or deviates from the above.

** Evaluation**
I would have expected a much more clear evaluation and a validation of the CSO with a plethora of actual cloud services. You state indeed on page 13 that for space limitation the population mechanism is out of scope. To me, this is essential to assess the usability of your work? For example. Let's consider for example AWS: how would you obtain information from the wide variety of their services? Collecting information is a difficult and complex aspect which is not clearly explained; you just say you populate the CSO wit 767 instances by collecting information from the web pages of the providers and well-known (which?) monitoring parties (page 29). This mechanism of filling with instances the CSO needs to be better explained.
In fact, on page 33 figure 9 your show out of the blue show that two cloud providers satisfy the requirements (AWS and MS Azure) but nowhere you showed which services are actually instantiated in your ontology.

** Related work**
The paper is reasonably grounded in real cloud technologies and real projects (although from FP7),. For completeness the related work section should cover more recent efforts, such as for example foe CYCLONE project (H2020).

**Relation to standard **
You refer to few standards CIMI, OCCI, TOSCA and provide references to their websites. Still you don’t provide evidence that you really looked at these standards. I suggest you provide evidence that you used those standards, and how they are reflected in CSO/ontology.

**Minor questions **
- Table1. Why do you make the oversimplification that Brokers, Providers and Users are distinct from each other. This seems to me a limitation that does not reflect actual cloud operations where a provider can be user of another cloud.
- You focus on description of virtual machines. The current trend in clouds is toward micro services and containers. How would you support this in the CSO?

Review #3
By Maria Maleshkova submitted on 08/Mar/2017
Minor Revision
Review Comment:

The paper presents an ontology for describing could services – CSO, especially focusing on covering the functional and non-functional properties of IaaS, Paas and SaaS services. In addition, the authors present an evaluation approach for determining the completeness of descriptions created using CSO and provide support for service selection based on functional and non-functional requirements. The work presents a significant contribution in the field and the introduced contributions address current problems that are partially or fully still unsolved. The paper is nicely written, easy to follow, with a clear logical order of the sections. The main points of improvement that can be recommended are not in terms of the contributions but rather in terms of improving the structure of the paper. There should be a better balance between the length of the sections and their importance in terms of the overall view of the paper. Some sections come a bit surprisingly, for example the introduction of the ontology concepts, while other are a bit two short – the use case evaluation section. It would be good to have a clear focus on the main contributions of the paper and a better balance in terms of the given detail and the corresponding significance. The paper is quite long and this would help to improve the readability.

Contribution-with there are two main remarks that need addressing:
1) The aspect of federation is very much emphasised in the abstract, introduction and conclusion, however, the main part of the paper does not really refer to federation and if it does, it reduces it to search. I would recommend once in a while to link back to the impact on federation of the individual contributions.
2) The reuse of existing concepts within the ontology is completely ignored. A number of the CSO have been already introduced in other ontologies, it is good practice to reuse those, instead of introducing new ones. Similarly, there should be links to concepts from other ontologies. This is also missing. This is needs to be addressed.

Important issues that need to be addresses:
1) Why does not the ontology link to and reuse existing concepts from other ontologies? This is a very important aspect that is ignored in the proposed work. There needs to be an explanation as to why this is not necessary or to how this will be implemented in the future. A good practice would be to publish and link the ontology on LOV.
2) The related work needs to be restructured. Currently there is not logical order of how the previous work is presented. Frameworks are mixed with ontologies, with discovery engines and from time to time there is a discussion of the disadvantages. Please restructure the related work section by listing ontologies, then frameworks followed by a comparison and a discussion of the current gaps. Missing work on early cloud service research - USDL / LinkedUSDL. Insert a table of comparing the related work, for example on page 6. In this way the following points can be understood more clearly. Currently they come a bit surprisingly.
3) On page 8 there is a big jump between the general view on cloud description ontologies and the description of the concepts. Figure 1 comes a bit surprisingly. How was the information that needs to be covered by the ontology determined? What competency questions were used? What are the requirements for designing the ontology? Why are not existing definitions of concepts reused, for example for provider? These questions need to be addressed before presenting the concepts of the ontology in detail.
4) Where is the ontology available and why do the concepts have no namespaces? Why are there no modules of the ontology, focusing on the different aspects, for example security, core, etc.?
5) How were the concepts of the Functional_Capability determined and what are their types? What was the systematic approach behind their definition.
6) Why not reuse concepts from other ontologies for Security?
7) Explain how the definitions that you have introduced in section 4. for example, High_Protection can be customised/adapted to meet specific use cases. How generally applicable and customisable are these definitions? This would also tie very nicely with a short explanation of how these were defined.
8) Are the inference rule formalisations (the rules in the boxes) necessary for each type or are just a few examples sufficient? Some of the rules are quire straight forward.
9) Page 21, how were these errors chosen? Why exactly these? What is the conclusion based on the fact that the restrainer does not detect any errors? So the ontology still might have errors?
10) Page 32, “good matching score” How do you define a good matching score? This is vary vague, add a reference or an explanation. Similarly, in this section it is not clear how the matching is implemented and what algorithms are used. Please provide a reference.
11) Shortly address where and how the cloud service descriptions are stored. Is there a service description repository?

Detailed comments:
1. Page 2. add a reference to IaaS, PaaS, Saas, etc.
2. Page 2. description heterogeneity also influences
3. Page 6. Point 2. proposed approaches ignore could standards — which standards are you talking about? specify. Also on page 7. Clearly describe what cloud standards you are talking about
4. Page 6/7 How do you compare ontologies with frameworks with engines? This needs to be explained.
5. Table 1. is not discussed.
6. Page 12, last sentence. Table 2 does not present axioms.
7. Avoid abbreviating it is as it’s
8. Page 15. “has no compatibility with” explain/define what you mean by compatibility.
9. Page 18 "if security is neither high nor low” why is this moderate? Why is it not undefined or incomplete? Will this not cause bad search results?
10. Page 19. second paragraph of section 5, the anti-pattern detections are visualised in the figure but the “recommendations” are not. This is a bit confusing at this point (clear later on).
10. Page 19. second paragraph of section 5, last sentence, they find out complement each other? What do you mean?
11. Page 23, section 5.2.2 “we propose three anti-pattern categories” why exactly these?
11. Page 26. Invalid VMType characteristic - rephrase the definition as anti-pattern. Currently it is formulated in a positiv way. Must have…
12. Listing 2, page 29. The listing is not really needed. The procedure is quire straight forward.
13. Page 30, “Moreover, since the extracted data den be incomplete.” rephrase, incomplete sentence.
14. Page 30, “If no error is detected, the ontology is considered as evaluated” As valid? The evaluation can be completed also if there are errors, so the ontology would be evaluated but not correct or valid.
15. Page 31, section 6. “CSO Interrogation” Is this a standard definition of “interrogation”? If yes, please provide a reference otherwise consider using a different term.
16. page 31 “cloud teachers” consider rephrasing.