How Ontologies Benefit Enterprise Applications

Paper Title: 
How Ontologies Benefit Enterprise Applications
Authors: 
Daniel Oberle
Abstract: 
This paper contributes an argumentation line for how technological features of ontologies lead to benefits for enterprise applications. Although many features are also available in precursory or alternative technologies, we claim that combinations of specific features are uniquely provided by ontologies. A careful elicitation of the available features therefore is a prerequisite for the argumentation line. As a second contribution, this paper reports on several challenges that frequently occur when trying to adopt ontologies in existing enterprise settings. These challenges have to be contrasted with the often overstressed benefits in Semantic Web literature. Together with reports for several SAP Research case studies, this paper channels back experiences in applying ontologies to the Semantic Web community. As a third contribution, we give several recommendations for future research directions based on the gathered experiences.
Full PDF Version: 
Submission type: 
Full Paper
Responsible editor: 
Krzysztof Janowicz
Decision/Status: 
Accept
Reviews: 

Accepted by editor

Reviews for previous versions below:

Solicited review by Michael Uschold:

The authors are to be commended for doing a lot of of hard work to address the reviewers' concerns. The rationale for the organization of the paper is much improved, and far less confusing in this version. Overall the paper hangs together much better. Modulo a few comments below, I'm ready to accept this paper as is.

Sec 3.2: This is an unusual definition and example for flexibility. I would argue that the ability to manually or automatically transform an ontology into an executable model for different specific purpose is highly flexibile, compared to starting with a DB schema which cannot be easily reused in a similar way. Yet you argue this as an example of Inflexibility. Reuse leads to flexibility. I'm confused.

"To the best of our knowledge, the idea
of having foundational ontologies and quality criteria
has not occurred in other disciplines."
DB normalization techniques are an example of quality criteria, you must be aware of them.

4.1 New business scenarios is super-broad as a category, and any of the technologies could combine in novel ways to achieve this. In particular, flexibility (which you leave out) is a major driver that makes new things possible that would have been too costly before to do. What good argument is there for leaving out anything in the top row of fig 3? Indeed, you mention flexibility in the BBC example, but it is not in the top row. This seems inconsistent.
--

I repeat: reuse leads to flexibility. Being able to combine existing pieces instead of building big things from scratch give much flexibility in what you can cost-effectively produce. This should be highlighted in the text and in fig 3, some how.
Also, increased productivity is not independent from new business scenarios, it directly facilitates it. In turn, improved enterprise information management increased productivity.

Solicited meta-review by Leo Obrst:

I largely concur with Michael Uschold's final review. If anything, this is a survey paper, since the research contribution is minimal. But even as a survey paper, it is incomplete. I think the paper as currently written would only minimally interest a business analyst, i.e., a somewhat technical person in industry, who was interested in laying out the arguments for the use of ontologies for internal enterprise applications. The paper will not appeal to more knowledgeable technologists, i.e., those involved in semantic technologies or ontologists, since really there is nothing new here for them. Ultimately, this may be a paper that is better for another venue, e.g., one that addresses emerging technologies for business applications. Personally, I hate to discourage the author, given his previous work, but would advise him to consider revising this, perhaps not for this journal, but for another venue.

An alternative is: if the paper was shortened (I would say to 10 pages), and the Semantic Web Journal created an occasional "Business and the Semantic Web" series, then this paper could be a potential candidate. This kind of occasional report actually might be a very useful addition to SWJ, especially if the journal had as a sub-audience some technically savvy business people.

But for now, even as a focused survey paper, there are gaps and errors:

1) p. 3, section 3.1: Class and property assertions are axioms too. We typically avoid mentioning that, so this characterization and discussion is ok if the audience is more general than ontologists and semantic technologists.

2) p. 3, section 3.1: Giancarlo Guizzardi has tried to bridge the gap between conceptual models and ontologies, and the two communities of conceptual modelers and ontologists, in many papers since his dissertation. I think the author should be aware of these.

Guizzardi, G. 2005. Ontological foundations for structural conceptual models. Ph.D. dissertation, University of Twente, 2005.

Guizzardi, Giancarlo, and Terry Halpin. 2008. Ontological foundations for conceptual modelling. Appl. Ontol. 3, 1-2 (January 2008), 1-12.

Here's his Google Scholar page: http://scholar.google.com/citations?user=nnfVBt8AAAAJ&hl=en.

3) p. 4, section 3.3 and other sections: There is no mention of the following paper (and presumed effort) about the ISO 15926 upper ontology:

Batres, Rafael; Matthew West; David Leal; David Price; Katsube Masaki; Yukiyasu Shimada; Tetsuo Fuchino; Yuji Nakag. 2007. An upper ontology based on ISO 15926. Computers and Chemical Engineering 31 (2007) 519–534.

Matthew West, formerly of Shell Oil Company, UK, has been a strong proponent of ontologies in the oil industry for many years. See: http://www.matthew-west.org.uk/publications.html. He is co-chair, along with Michael Gruninger, of the emerging Ontology Summit 2013: Ontology Evaluation Across the Ontology Lifecycle: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013.

4) p. 5, section 3.6: There were actually some quasi-standardized ontology and knowledge representation languages prior to the emergence of the Semantic Web language standards, including the Knowledge Interchange Format (KIF; http://en.wikipedia.org/wiki/Knowledge_Interchange_Format) and Ontolingua (http://www.ksl.stanford.edu/software/ontolingua/), both established during the DARPA Knowledge Sharing Effort of the early 1990s, when the notion of ontological engineering emerged.

Patil, Ramesh S., Richard E. Fikes, Peter F. Patel-Schneider, Don McKay, Tim FInin, Thomas Gruber and Robert Neches. 1992. The DARPA Knowledge Sharing Effort: Progress Report. In Proceedings of the Third International Conference on Principles of Knowledge Representation and Reasoning, ed. B. Nabel, C.Rich, and W. Swartout, Cambridge, MA. Oct 25-29, 1992. http://www.isi.edu/isd/KRSharing/kr92-ksharing.ps.

KIF later morphed into ISO Common Logic, also not referred to in this paper: http://en.wikipedia.org/wiki/Common_logic.

In addition, no mention is made of Cyc, which was the first ontology and reasoning engine: http://en.wikipedia.org/wiki/Cyc.

There were in fact other ontology web languages proposed prior to RDF/S, OIL, DAML+OIL, and OWL, e.g., XOL, OML/CKML, etc.:

Karp, Peter D.; Vinay K. Chaudhri; Jerome Thomere. 1999. XOL: An XML-Based Ontology Exchange Language. Pangaea Systems and SRI, International. http://www.ai.sri.com/~pkarp/xol/.

Kent, Robert. 1999. The Conceptual Knowledge Framework OML/CKML. KAW'99, Twelfth Workshop on Knowledge Acquisition, Modeling and Management, Voyager Inn, Banff, Alberta, Canada, October 16-21, 1999. http://sern.ucalgary.ca/ksi/kaw/kaw99/papers/Kent1/CKML.pdf.

And pre-Web languages, including the first combined reference for description logics and knowledge representation systems:

Special Issue on Implemented Knowledge Representation and Reasoning Systems, SIGART Bulletin Volume 2, Number 3, June, 1991. Note: This issue is always cited by everyone and remains the best single issue of the SIGART Bulletin ever, if you ask me, and is still useful; contains papers on every major KR system, including the description logics of the time: KRIS, LOOM, CLASSIC, BACK, Algernon, RHET, K-Rep, TELOS and the LILOG KR system.

The lack of these citations demonstrates to me that the author has not sufficiently considered the research literature.

5) p. 5, section 3.6: XML Schema is not comparable to or a "full-fledged substitute for" RDF(S). XSD is a structural language; RDFS is a semantic (ontology) language. The former addresses structure much like database schema languages do, and in particular, XSD addresses documents. RDFS is a simple semantic (ontology) language that focuses on classes and properties.

6) p. 6, section 3.7: under "First-Order Logics", there have been/are many reasoners under generally the rubric of theorem-proving, since the 1970s. Today's reasoners include Vampire, Otter/Prover9, etc. Even Cyc is a mostly second-order logic reasoner. See: http://www.cs.miami.edu/~tptp/CASC/, e.g., for theorem-proving competitions. In addition, there is the SAT (satisfiability) and SMT (satisfiability modulo theory) reasoners: http://en.wikipedia.org/wiki/Satisfiability_Modulo_Theories.

7) p. 6, section 3.7: an alternative to Figure 2 is the "Ontology Spectrum" of:

Obrst, L. 2003. Ontologies for Semantically Interoperable Systems. Proceedings of the Twelfth ACM International Conference on Information and Knowledge Management (CIKM 2003), Ophir Frieder, Joachim Hammer, Sajda Quershi, and Len Seligman, eds. New Orleans, LA, November 3-8, New York: ACM, pp. 366-369. http://portal.acm.org/citation.cfm?id=956863.956932.

8) p. 6, section 3.7: "Instance Retrieval" is not quite adequate, since this addresses querying. Another bullet should be added: Rule-Reasoning. You should not conflate instance retrieval with rule reasoning, since query languages generally do not perform inference, whereas rule-reasoning systems do.

9) p. 6, section 3.7: "Rete-based business rules" are expert systems, sometimes called "production rule systems". There are issues with expert systems.

10) p. 7, figure 3: I think there are some issues with this graphic. E.g., semantic (-web) services are not included directly on this; "querying" should probably be also under "Direct Interaction"; there are some missing level-demarcation lines in the figure; FO is not included under the column "Formality" for the row "Increased productivity of software engineering".

11) p. 8, section 4.2: Why not mention semantic wikis here?

12) p. 8, section 4.2: Other electronic commerce examples.

eClass and other business-to-business e-commerce efforts could also be mentioned here. For example, the VerticalNet effort of 1999-2001 reported on in:

Obrst, L., R. Wray, H. Liu. 2001. Ontological Engineering for B2B E-Commerce. In: Formal Ontology in Information Systems (FOIS): Collected Papers from the Second International Conference, October 17-19, 2001, Ogunquit, ME. Chris Welty, Barry Smith, eds. ACM Publishing, http://www.fois.org/fois-2001/index.html.

Obrst, L., H. Liu, R. Wray. 2003. Ontologies for Corporate Web Applications. Artificial Intelligence Magazine, special issue on Ontologies, American Association for Artificial Intelligence, Chris Welty, ed., Fall, 2003, pp. 49-62. http://portal.acm.org/citation.cfm?id=958676.

13) p. 10, section 5.1, 5.2, and other subsequent sections: Concerning ontology training, reference could be made to Ontology Summit 2010: Creating the Ontologists of the Future, http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2010, which is focused on training.

14) p. 11, section 5.3: mention should be made of OntoCom and its ontology cost model: http://ontocom.sti-innsbruck.at/.

15) p. 12, section 5.6: I disagree with a previous reviewer's comment, which suggested the lack of tools for domain experts was a/the problem with adoption of ontologies, and so suggest you back this revision out. Domain experts cannot build ontologies yet until they acquire ontologist skills, in much the same way that end users of databases cannot build databases: without training, the result is garbage. Before automated tools are available that encapsulate the knowledge and insulate domain experts from that knowledge to develop ontologies, 10-30 years will go by. The good news is that it will take domain experts much less time to acquire knowledge and skills in ontologies than it will take ontologists to acquire knowledge and skills in a given domain.

16) p. 15, section 6.3: Reid Smith, formerly of Schlumberger and now Marathon Oil, got his PhD from Stanford U, and did work in AI in the Gas and Oil community, which should have some value here: http://reidgsmith.com/. Similarly for Adam Farquhar.

Smith, Reid G., and Adam Farquhar. 2000. The Road Ahead for Knowledge Management: An AI Perspective. AI Magazine, Vol. 24, No. 4. AAAI. http://www.aaai.org/ojs/index.php/aimagazine/article/view/1528.

17) p. 16, section 7: There is also the OMG ODM effort, which maps UML to OWL, which could prove useful for improving the modeling of OO software engineers and/or translating their efforts into OWL or even Common Logic: http://www.omg.org/spec/ODM/1.0/.

Mention could be made of ontology repository efforts such as Open Ontology Repository (OOR; http://ontolog.cim3.net/cgi-bin/wiki.pl?OpenOntologyRepository), OntoHub (http://ontohub.org/), and BioPortal (http://bioportal.bioontology.org/).

Reference 93, Uschold, 2010 should be updated, per the NOTE below.

NOTES:

1) Note that the citation/reference for:
Uschold, M.: Making the case for ontology. ;Applied Ontology(2011)377-385

should be the following (per Erratum in Applied Ontology 7 (2012) p. 373. DOI 10.3233/AO-2012-0110):

Michael Uschold, John Bateman, Mike Bennett, Rex Brooks, Mills Davis, Alden Dima, Michael
Gruninger, Nicola Guarino, Ernst Lucier, Leo Obrst, Steve Ray, Todd Schneider, John Sowa, Ram Sriram, Matthew West and Peter Yim. Applied Ontology (2011), Volume 6, number 4/2011, pp. 377-385. IOS Press.

In addition, the detailed, working web site of Ontology Summit 2011, of which the above paper is the report of the Ontology Summit 2011 joint Communique, is the following: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2011.

Revised resubmission, currently under review. History of past revisions is below.

Previous round reviews - paper was submitted under title Ontologies & Reasoning in Enterprise Applications as survey paper, the last decision was "reject and resubmit".

Solicited review by Simon Scheider:
The new version of the paper has improved in clarity and argumentation. I consider it ready for publication.
I have only minor suggestions for improvement:

- 3.2 may be called, more adequately, "Flexibility in modeling"
- In section 5, a particularly important but missing challenge may also be enabling of information privacy in EIM, since it contrasts with the open and public web architecture of the semantic web. Enterprises host sensitive information and need a way to protect it against access.
- In 4.4, what I meant in my last review was not Enterprise Knowledge Management in general, but, more specifically, in-house documentation and archiving, since ontologies force developers to document their model upfront, not after the tool was built.

A last (lateral) comment to Michael Uschold's comments. There is not a single business study that I know of which is methodologically rigorous. However, as a matter of fact, this does not render business studies useless. Also, technical people are typically the ones that have no clue about added values and business benefits of their own technology. To me, the first two of Michael's "con" arguments (too little technical depth and bored technicians) seem therefore mislead.

Solicited review by Michael Uschold:

Review of resubmitted paper.
Most of the substantive issues in my first review have not been adequately addressed. I did not find the the author's response/explanations satisfactory in most cases. Lets consider the criteria from the call for papers.
1. likelihood of becoming well known introductory text
2. comprehensive and balanced
3. readability and clarity
4. importance to the broader semantic web community.

1. Introduction Text: I don't think this has much chance to become a well known introductory text. Even though I am expert in this field, I expect to learn a lot from a high quality survey article. I learned very little. Most of what is here is readily available elsewhere, other than the SAP applications. What I expect in a survey paper is the use of many articles and references as source material which is carefully distilled and presented back to the reader as a coherent story. In this case, there is not a compelling story, and there is not a lot of analysis. Too often, the references are not used as source material, they are quoted and used directly as the material itself without further comment.

TWO: Comprehensive and balanced: It is not comprehensive because there is really no attempt to survey applications that have been built in the enterprise. The focus on the enterprise minimal, with too much focus on ontologies and reasoning in general. This is problematic for a number of reasons.
1. there is systematic confusion between what is a business benefit and what is just a technology itself. For example, conceptual modeling is a technology, not a value add. The technology has been around for decades, building an ontology is doing conceptual modeling with a more formal notation. Web compliance is a technology that can have benefits, it is not a benefit in itself. The authors response to this was not helpful, they simply disagreed without justification. The argument for keeping conceptual modelling in was to save the structure of the paper. That is not a good reason. The paper is to serve the reader, it is not for the convenience of the author.
2. the presentation consists too much of many quotes that do not tell a consistent and coherent easy to follow story with a clear message.
3. terms are too often not introduced and clearly explained
4. the supposed value add features of ontologies and reasoning, are not unique to ontologies and reasoning (the authors say as much, and I agree). Conceptual modeling and web compliance, for example. This strongly undermines the case the authors are trying to make, and is one reason why this paper is unlikely to become a widely referened introductory text.
The presentation is very much unbalanced, favoring SAP applications and the oil & gas industry. There is mention of the BBC, which is good.

THREE: Readibility and clarity: the presentation consists too much of quotes that do not tell a consistent and coherent easy to follow story with a clear message. Also, while in a survey paper, it is somewhat common to have a consistent structured format for the material, this is also risky – it is easy to get bored as a reader. In this case, it did not work very well. The foundation of the paper is the value added features, which are problematic as noted above. Then they are applied to enterprise applications, but since the foundation is not solid, using it to structure the paper does not shed much light. Another problem with clarity is that many technologies are mentioned without any indication of how mature they are. For example, there is a reference to a 2003 paper about semantic semantic web services, as if it is a mature technology being used in the enterprise. In fact, it was an early research paper on the topic. This is confusing and misleading. It would be much better to organize the paper around many impactful real world applications of ontologies in the enterprise ,and then examine those to find common threads.

FOUR: Importance to the broader semantic web community:
This material is mostly about ontologies and reasoning, which is very broad and very adequately covered in existing literature. What might have been valuable to the semantic web community is a good survey on actual enterprise applications that are adding value now. The adopting of TOC practices from IT in general to semantic tgy in the enterprise is potentially useful.

FINALLY: there are surprisingly few references from 2010 onwards, only 10%. While 50% of the references are from 04-09. This is surprising, because most of the big impact enterprise applications have been happening in the past few years.

Revised manuscript after "reject and resubmit". Previous reviews below.

Solicited review by anonymous reviewer:

The revised version seems to provide much more focused and improved
description of use cases, which could benefit those who do not have much
experience with ontologies but consider using them for their projects. I
am ready to accept the paper as-is.

Solicited review by Simon Scheider:

This paper is a well written documentation of practical insights from business experiences with semantic technology. It is a business study (which should be treated as application report or survey paper), rather than a research paper. However, as such, it is a very valuable contribution. I even think we need many more such papers, which can serve as practical feedback loops to semantic web research, especially in order to orientate future research efforts with respect to user needs, and to provide arguments for adopting semantic technology.

Consequently, the paper needs to be evaluated with respect to clarity of argumentation and presentation, as well as with respect to community interest in the presented experiences and insights. With respect to both aspects, I am very happy with the paper as is.

I have some minor suggestion, which the authors may want to address:
In my view, ontologies serve to a large degree as a way to make hidden knowledge explicit, and a way to document and share it in a cheap way. Therefore, I would add to 4. Benefits for Enterprise Applications, that ontologies are a cheap knowledge preservation and documentation tool. In my view, knowledge preservation is one of the central challenges for business intelligence. Many companies are good in development and management, but only very few are actually able to preserve and store implicit knowledge of their developers and experts. Knowledge gets usually lost, besides the efforts of software engineers to standardize the documentation process. However, upfront modelling forces developers to upfront documentation.
The authors address this aspect in their first use case, but do not include it in the enterprise benefits.

Solicited review by Michael Uschold:

GENERAL COMMENTS:

PROS:
The problem being addressed is very real and very important. The breadth and scope of material covered and the bibliography is impressive. I like the idea of connecting value generators with specific use cases; that greatly improves clarity.
There is a very good job explaining some of the issues and challenges of implementing semantic technology in the enterprise.
A number of interesting use cases were mentioned, and I liked weaving the single industry, oil and gas, throughout all the different scenarios.

CONS:
Too little technical depth, all fairly abstract and general.
The intended audience is not clear. Technical people will be bored with all the stuff they know already, and business people will be bored with all the technology discussion that is not directly connected to clear business benefits that they care about.

I did not find that the paper achieved in its goal to help researchers and practitioners "identify which value-adding features they actually apply and contrast them with the challenges". The main reason is that that most of the discussion is at a very high level and does not clearly tie technologies to business benefits.

Reuse, for example, is not a business benefit. It can be more costly to reuse something than to do it over again. It depends on the context.

Flexibility is also not a business benefit per se, though it is much easier to connect to business benefits. For example, it can reduce maintenance costs, it can reduce time to market for developing new products, it can also lead to being able to do things that were not practical to do before E.g. the BBC web sites could have been done using conventional technology, but it would have been prohibitively costly to do so. Flexibility was a key.

Better to first introduce benefits that business people care about, and then work backwards to technologies that can facilitate them. For example, every business person wants reliable accurate systems. Using automated consistency checking helps increase reliability and accuracy of the ontology, which in turn will help increase the reliability and accuracy of systems built using it.

Most of the examples are several years old, there are many more recent ones. I get the impression that the paper was written a few years ago, and then briefly updated rather than being fully up to date from scratch. There are a number of high profile exampes of enterprise use of semantic technology that are not mentioned (Amdocs and Garlik to name two). The examples sometimes are not carefully distinguished between research prototypes vs. pilots vs. production, and at what scale.

The tone is one of bad news, "oh gosh, this is hard". It should be more upbeat. Do not ignore the challenges, but focus more on the positive.
---

SPECIFIC COMMENTS:
Abstract: is ok as a brief summary, but it could do with a bit more substance, e.g. include one or more conclusion or insights that were learned.

The term "the paper" seems to refer to another paper, but eventually I realize it refers to the submitted paper, not a referenced paper.. Suggest using "this paper" throughout.

The introduction is very abstract and generic, would like it to be more brief and to the point. The contributions are also very abstract. Quoting a common belief that the value add features of ontologies and reasoning are formally modelling a domain and then applying automated reasoning -- is fine, but it should be followed by an observation that value cast in such terms is quite unhelpful to a business person. They will ask: why do I care about formal modeling a domain and automated reasoning?

Sec 3: There are a lot of things that add value, but they often add value in the context of helping to build a better ontology. But there is no point in building better ontology until you have a clear argument for why having an ontology is better than alternate approaches. This is putting the cart before the horse.

Sec 3.1: To say that conceptual modeling is a value-adding feature of ontologies is unhelpful and unnecessary. An ontology is fundamentally about conceptual modeling, so it goes without saying. Having a model be closer to a domain expert's conceptualization of the domain makes it easier to understand, it does not make it useful. If it is not useful, no one needs to understand it. Start with the purpose of the model in the first place and show how those purposes are better served with ontologies and reasoning.

Sec 3.2 the argument for how ontologies and reasoning increases flexibility is not clear. There are many existing good arguments to choose from and analyze. IMHO, ontology evolution is not one of them. Why not apply ontology evolution strategies to conceptual models and get the same flexibility? Re-classifying instances is a good example of adding to flexibility, but it is only mentioned in passing. Also, flexibility can manifest in many different ways; these should perhaps be explored in more depth (see my comments above).

3.4 You seem to be acknowledging that the case for ontologies over other modeling approaches for reuse is weak. So why include it? In fact, an important point has been missed. Ontologies are easily reused, but database schema generally are not.

3.7 It is true that if you want formality, then ontologies are a good way to go, but where is the argument that business people care about formality? They don't, they care about results. So, how does formality give better results? That is the important question here.

4.3: Visualization is orthogonal to ontology technology, so it should not be highlighted as a benefit of ontology technology.

5.1: I disagree that technical integration is agnostic with respect to ontology technology. It is particularly exacerbated with ontology technology, since there are so few standards and so few vendors compared to mainstream technologies.
Fig 4: don't you mean "Ontology-agnostic challenges" for the left side of continuum?

6.1: Description is too generi; I cannot tell what FindGrid actually does, whether it is implemented in any deployed applications.

6.2, Again a very generic description. Saying that used the conceptual modeling value adding feature to describe carrier service products via building and OWL ontology does not seem illuminating in any way. Easier to just say: we built an OWL ontology for this. You could just as easily have built one in UML. There is no argument here that using OWL helped compared to other modeling languages.
Also, there is some speculation about what is possible that is not clearly separated from what is already implemented and adding clear value. It was nice that you explained the outcome at the end, why it was not productized.

Why no reference to the international effort making the case for ontology?
Many of the issues are addressed there.
Uschold, M.: Making the case for ontology. ;Applied Ontology(2011)377-385
For a graphic, see slides 25-27 in this talk version of the above paper: http://www.slideshare.net/UscholdM/making-the-case-for-ontology

This is a revised manuscript after a "reject and resubmit".

In the first round (reviews below), the paper was submitted as a full paper.

Solicited review by Rama Akkiraju:

This paper discusses the arguments and business case for using ontologies and semantic technologies in enterprise applications. As someone who had worked in this area and made unsuccesful attempts to commercialize the technologies developed in this space in a large enterprise integration company I agree with the arguments made by the authors to a large extent. The arguments made and business case discussed are valid. If I were to write a paper on my own experiences, the arguments would look very similar. In addition to the arguments made in the paper, I'd point out two more reasons for the lack of adoption of semantic technologies which need highlighting. Lack of adoption of semnatic technologies in the past few years is attributable to the lack of tools that offer higher levels of abstraction. OWL and other semantic languages are very complex to an average user/developer. The audience for most ontology editing tools even today is power users who know OWL and XML and logic programming. This is not the audience who is expected to create ontologies. Often, subject matter experts of domains are the most suitable people for authoring ontologies. They usually have no technical expertise in complex and deep computer science subjects. So, the entire tooling in ontology domain catered to the wrong audience to start with. This led to the lackluster adoption of semantic technologies.

Another reason for lack of adoption of semantic technologies is the need to build ontologies upfront. This in combination with tools that do not cater to the right audience, makes it very hard to get started with anything. Learning ontologies from domain dictionaries and to build them over time is looked at as a solution. This area of 'ontology learning' is not discussed in the paper. It needs a mention and citing of some papers. Here is one reference to ontology learning.

Hui Guo, Anca Ivan, Rama Akkiraju, Richard Goodwin: Learning Ontologies to Improve the Quality of Automatic Web Service Matching ICWS 2008: 337-344

Solicited review by anonymous reviewer:

The paper discusses the advantages and the challenges of using ontology & reasoning technologies in enterprise applications and presents two previous projects that set out to address some of the challenges. While the paper lucidly delineates several concepts – for example, the eight value-adding features in the first part of the paper, I think the paper does not include enough research contribution to warrant publish.

Firstly, the claims made in the paper are often made without referencing supporting evidence and data. As a result of the lack evidence, the claims in the paper sound speculative and subjective. For example, Section 4, which illustrates the benefits of using ontology, does not present any evidence aside from a few short introductory sentences that loosely reference existing projects.

Second, the paper's claims are often overly broad generalizations that need to be much more focused and specific. For example, Section 4 starts with the sentence, "arbitrary combination of value-adding features can create new business scenarios", and then presents two use-cases without any further elaboration. Rather than broadly referencing "new business", the paper should instead list the specific areas that would benefit and why. The description of the use cases in Section 4, for example, could be improved by discussing 1) problems that each use case sought to solve 2) how ontology-based solutions were implemented in each case 3) actual benefits acquired from the ontology-based solutions. Additionally, rather than vaguely claiming that ontology-based solutions "improves productivity and EIM" in Section 4, the paper should reference specific statistic that highlight improvements in key aspects of productivity and EIM that were achieved through ontology-based solutions.

Finally, the paper needs more insights and analysis, as opposed to simply enumerating the pros and cons of ontology-based technologies, which have already been discussed extensively in the existing literature. With that in mind, Section 6 in particular could be improved considerably. Instead of introducing the authors' past projects, it should thoroughly survey the existing solutions to the challenges and provide novel insights concerning what aspects of the challenges have been addressed and how, and which problems researchers still need to address, etc.

Overall, the paper needs to venture further into the realm of original analysis, and do more than simply reiterate what is already known. In order to achieve this, the paper should contain more concrete evidence and data from scientific methods and more insights and analysis from the author

Tags: 

Comments

This paper discusses the problems surrounding the use of ontology-based
technologies in enterprise applications. In particular, the paper summarises
various technical and non-technical benefits of using ontologies in
enterprises, and it discusses issues that need to be addressed in order to
achieve a wider acceptance of ontology-based technologies in enterprise
settings. Finally, the paper discusses several case studies conducted by SAP
to evaluate the benefits of using ontologies in enterprise applications and
identify areas for further improvement.

In general, this paper offers to the Semantic Web community several
interesting observations about the "real world". I particularly liked Section
6 that described the actual case studies. This section is very concrete and it
presents opinions based on real-world experience. In contrast, I was not that
fond of Section 3 and to a certain extent Section 4. These sections list
various benefits of ontology-based technologies in general (Section 3) and in
enterprises (Section 4). My main problem with these sections is that they are
rather abstract.

For example, Section 4 list the well-known benefits of reasoning, such as
instance classification and/or subsumption checking, but the discussion
remains rather abstract. In particular, it remains unclear whether these
reasoning services are really important for the actual use cases: the
potential benefits of reasoning seem to be often overstated in research papers
and project proposals. Thus, this section reads a bit like an introduction to
a technical paper for a logic conference, and it does not present a clear-cut
case for the use of reasoning in practice. While reasoning may seem like a
good idea on paper, applying reasoning in practice is fraught with problems
that were not discussed in the paper, which makes this part of the paper less
convincing.

I have similar comments about other parts of Sections 3 and 4. For example, I
doubt that many businesses will actually care about formality as such: this
aspect of a system is important only if it contributes to a business-critical
need, such as quality. Similarly, while ontology reuse seems like an easy sell
in theory, applying it successfully in practice is far from trivial and it
requires a considerable investment in ontology alignment and adaptation.

It seems to me that the paper would be more convincing if it focused on quite
specific and concrete use cases, based on which it would clearly demonstrate
the business case for using ontology-based technologies. In other words, I get
the feeling that Sections 3 and 4 could be shortened, while Section 6 could be
expanded. Furthermore, Section 5 could be moved after Section 6, and the goal
of this section would be to present the lessons learned on the basis of
concrete use cases.

In the rest of my review, I will give some more concrete comments (of mostly
linguistic nature) about various parts of the paper.

- abstract: "still share a touch of ivory tower technologies" does not sound
right in English

- abstract: "the paper lucidly delineates" -> "this paper clearly presents"

- The last sentence of the abstract is unclear and should be rephrased.

- Section 1, 1st para: "we did not see" -> "we have not seen so far"

- Section 1 in several places: "elusive" sounds a bit strange in this context

- Section 1, 2nd para: "conception" -> "sentiment"

- Footnote 1: I disagree to an extent with the sentiment that Rete-based
systems are not ontologies. Such systems can be understood as conceptual
models; furthermore, they have had way more impact in practice so far than
existing ontology-based systems. Thus, one might want to justify the
exclusion of Rete-based systems in a different way. One way to do it is to
simply say that the goal of this paper is limited to investigating the
benefits of OWL-based systems; while Rete-based systems could be conceived
as ontologies in some sense, their benefits could be analysed separately.

- Page 2: "more concrete as" -> "more concrete than"

- Page 2: "It is common belief" -> "It is a common belief"

- Page 2, several places: "the paper" -> "this paper"

- Page 2: "the contribution of the paper helps" -> "will help" or "should
help"

- Page 2: "allows to make" -> "allows one to make"

- Page 2, two places: "to use the ontology" -> drop "to"

- I found the first sentence of Section 3 confusing.

- Page 3: "allow to infer" -> "allow one to infer"

- Page 3: "allow to capture" -> "allow one to capture"

- Page 3: "gulf" -> "gap" (quotes are not needed here)

- Page 3: "automized" -> "automated"

- Section 3.3, Related Work: I did not find this paragraph convincing.
Semantic Web technologies such as SPARQL correspond to the relational
algebra in a straightforward way, and I do not see how such technologies
provide for more "direct interaction".

- Page 5: In my experience, foundational ontologies tend to be used very
little in practice, as they are often either too general or too specific for
a particular application.

- Page 5: "90ies" -> "90s" or "'90s"

- Page 5: "allow to publish" -> "allow one to publish"

- Page 6: The notion of sound and complete differs from certain use in the
literature. For example, resolution is deemed a sound and complete calculus:
it will never derive the empty clause from a satisfiable set of clauses (and
the calculus is thus sound), and if saturation terminates then one can
conclude the set of clauses to be satisfiable (and the calculus is thus
complete). Of course, resolution is not guaranteed to terminate in general,
which is generally considered a distinct property from soundness and
completeness. Hence, a more precise statement would be that "a sound,
complete, and terminating reasoner cannot be built".

- Page 7: It would be interesting to know what kinds of products the BBC is
building based on semantic technologies.

- In Section 4, it might be a good idea to distinguish "could be" applications
from real applications (i.e., ones that really exist in practice).

- Page 10: "Seldomly" -> "Seldom"

- Page 10: "of the problems" -> "of these problems"

- Page 10: "adaptors" -> "adapters" (although this is just my personal
preference: Wikipedia does not make a difference between the two)

- Page 10: I did not understand what is a "transport system".

- Page 11: "enduser" -> "end user"

- Page 11: "is also attributable" -> "can be attributed"

- Page 14: "such as store" -> "such a store"

In general, my problem with the paper is that I would have expected something different given the title. The paper promises "Ontologies and reasoning in enterprise applications", while most of those applications are talked about in conditional form, with a lot of "if" and "but". Given the three case studies in chapter 6, only FindGrid is (half) a success story, the other two are rather failure stories at a closer look. The essence thus seems to be: ontologies and reasoning are not productively used in enterprise settings. That said, chapter 7 seems rather weak, providing only little critical discussion and visions.

The main weakness in my opinion is chapter 5. Most of the points mentioned (build or buy, maturity of tools, enterprise scale performance, etc.) are not specific for ontologies and reasoning. If we make the experiment and replace "ontologies and reasoning" by, say, "3D CAD models" (and we can easily assume the oil and gas industry to work with those), most of the statements remain equally valid. Thus, they seem to be rather general statements, but no specific challenges for ontologies and reasoning. A look at [1] could provide some helpful insights here.

As the title is "Ontologies&Reasoning...", Linked Open Data does not appear in the paper. I can imagine that the authors want to avoid that area, since it would open another large set of challenges and visions, however, I feel like there are a lot of interesting business cases that could make use of LOD. When doing a major rework, it might be useful to think about including that aspect in the paper.

More specific points:
p.3: flexibility - the author states that a system without an ontology is less flexible and has to be restarted whenever the schema changes. I am a bit sceptic here. For example, the Amazon online store is (probably) not based on an ontology, but I do not assume that they restart their servers when adding a new product category.
p.4: "ontology evolution is not the same as database schema evolution because ontologies comprise data" - I do not get that argument, since databases contain instances as well
p.4: direct interaction with databases requires an additional layer because the end user does not want to deal with primary and foreign keys. That argument holds alike for ontologies, since end users do not want to deal with OWL restrictions either.
p.4: "ontology is created and used by only one person" - [2] states something different, stressing that an ontology is a "formalization of a SHARED conceptualization", i.e., there need to be at least two persons for having something to share. In a presentation, Hepp once used the sentence "Autists don't build ontologies", but there is no quotable source for that as far as I know.
p.5: I do not believe that there is no counterpart for foundational ontologies. I once had a book in my hands about high level content patterns in organizations which did not talk about ontologies, but I do not recall the title.
p.6: A more detailed view on the continuum can be found in [3]
p.6: The reasoning literature often says that subsumption checking, consistency checking etc. are different views on the same problems, see, e.g., [4]
p.6: I would have anticipated a more detailed description of the scenario.
p.7: The author mentions a "global database", but above, they state that ontologies are more than databases, so the wording should be chosen with greater care.
p.8: Fig. 3 is not self explanatory (e.g., why are no rules required for increased productivity of information workers?) and contains a number of unproven statements.
Sect.4 in general uses a lot of conditional, but few actual examples.
p.8/9: In a 2011 paper [5], we have shown that ontologies really have an effect for end users.
p.9: I am not sure whether disaster management is a typical enterprise application
p.10: the statement that ontologies and reasoning only work on toy data is quite bold. E.g., BigOWLIM and Virtuoso are capable of working with large scale data.
p.10: I do not immediately agree that the complexity of a language has an impact on performance. The author could be a bit more specific. The same holds for the statement that it is inclear how to put flexibility in software transport systems.
p.11: Actually, it is the openness and extensibility of ontologies which allows for specifying an ontology only as far as necessary (and let the open world assumption do the rest). Thus, the argument of upfront modeling costs does not make sense to me.
p.12: When talking about the different representations of ontologies, a reference to ontology matching and mapping would be useful. There are approaches that allow parallel use of different ontologies (see, e.g., [6])
p.12: I think it should be "# TCO drivers" in the formula. Furthermore, when multiplying three quantities, the product cannot be a cost. Thus, the price of a TCO should be in the formula.
Sect. 6 should discuss real developments and no "may" and "could", see above
p.15: How were the challenges such as scalability actually addressed? The same holds for the other two examples. I would have appreciated a more detailed discussion of challenges and trade-offs here.
p.16: the middleware was there before ISO 15926 was introduced, and it allows message exchange according to ISO 15926. This sounds somewhat twisted.

I know that most of what I wrote above does not sound too positive. However, I think that most of these points can be reasonably addressed, and that the paper will clearly be an interesting contribution once that is done.

[1] Rebstock, Fengel, and Paulheim: Ontologies-based Business Integration, Springer, 2008.
[2] Martin Hepp: Possible Ontologies. In: IEEE Internet Computing, Vol. 11, No. 1 (2007)
[3] Paulheim: Ontology-based Application Integration, Springer, 2011.
[4] Hitzler et al.: Semantic Web, Springer, 2008.
[5] Heiko Paulheim. Improving the Usability of Integrated Applications by Using Visualizations of Linked Data (2011)
[6] Ghidini et al.: On Relating Heterogeneous Elements from Different Ontologies (2007)

Section 2, page 2: Here you could refer to the PWC Technology Forecast 2009,
Semantic Web in the Enterprise, where Frank Chum of Chevron gives an inteview
about the need for shared ontologies in the oil and gas industry

Section 3: Please state in the intro that the featuresa nd subfeatures will be
later used in the evaluation in Fig 3 on page 8

section 3.1: Pls. ention that OWL provides a comprehensive variety of constructs
for describing classes and relations, and mention that this is on the other hand
quite costly (maybe mention OWL profiles)

Section 3.2.: "Flexibilty" is a rather blurry title for the subsection, I think "agile schema
developping" or "schema last" would be better here. According to [21], this is quite
often mentioned by analystists as a key benefit of ST, e.g. in
- Horowitz, P. (ed.): PwC Technology Forecast, Spring 2009.
- Final Demand driven Mapping Report. Public report D3.2 from the research project value-it.
- Stark, A., Schroll, M., Hafkesbrink, J.: Die Zukunft des Semantic Web. Think!innowise Trend Report, 2009.
- West, Dave: What Semantic Technology Means to Application Development Professionals.
Forrester Research, Inc Report, 10/2009.

Section 3.3.: To me, apart from the different kinds of direct interaction, a key benefit
of ST is that it better enables to use the vocabulary of the business user, instead of
cryptic, technica fieldnames in RDBMS.
If it comes to te interaction means, faceted UIs should be mentioned.
On the other hand, nattural language is IMHO *not* a benefit of ST, as NLP does not
belong to the core features of ST.

Section 3.4.: Reuse does not end on reusing *single* ontologies (for induviduals, communities,
or the world). More importantly: It is possible to *combine* ontologies, e.g. via mappings,
from different sources.

Section 3.5. "best practices" exists in nearly all IT-fields. Compared to the practices
in the RDBMS-field, I consider the best practices in ST rather weak ...

Section 3.7.: Note that the formal background of RDFS is neither captured by logic programming,
DL, or FOL.
When it comes to logic programmning, I don't think that FLogic is the best example here.
The different approaches to combine rulie-based reaasoning and DLs should be mentioned here.

Section 3.8.: I don't think that "consistency checking" etc are the main benefits of reasoning in ST
(e.g. this is pointles if RDFS-ontologies are modelled). Instead, specific reasoning facilities
like the possibility to express *transitive* relations are more interesting for ST.

Section 4.1.: I think the argument here is quite "blurry". To some extent, any new technology
can provide new business scenarios, So to me, "new business scnearios" are not a distinguishing
benefit of STs ...

Fig 3:
- I think that flexibility adds to new business scenarios
- i think that rules apply to the increased productivity of information workers as well
- think that the whole web compliance circle should be added to the increased productivity of
information workers (e.g. they better find information due to Q)
- I don't get how LP adds to the increased productivity of information workers
- I think S of Reasoning should be added to increased productivity of information workers and
Improved Enterprise Information Management

Section 4.3: For the perfomance of information workers, *collaboration* (e.g. via social
platforms, e.g. stramworks) becomes increasingly important. I'd like to see a paragraph
about the authors thoughts on how well (if at all) ST support collaboration.

Section 4.4: To me, the main benefits of ST w.r.t. Enterprise Information Management are:
- agile schema development
- ST support way better than RDBMS the federation of different (structured or unstructured)
data sources.

Sectiom 5: I like that section, as it makes clear that the benefits of ST come at a price ...
This section distinguishes this paper from all the ST whitepapers, as it addressed the
problems/downsides/challenges of ST as well. Nice!

Section 6:
Please make more explicit whether the case studies are prototypes, products, or whatever ...

Section 6.2: This example is not convincing, as
- it is somewhat outdated (2008)
- and, more importantly, as it turned out that the solution could not be productized due
to the ST challenges

Section 7: The discussion is somewhat puzzling, as it targets the challenges discussed in
Section 5, but it does not address and discusses any of the other sections. For example,
does the author expect more (business) use cases to show up in the future? Does the author
expect a stronger adoption of ST in an enterprise setting? etc ...