LOTED2: An Ontology of European Public Procurement Notices

Thank you very much for the authors to have taken really careful consideration of my review comments. Some of the sections of the paper have been extensively revised and the messages are brought out much stronger.

Modeling the legal domain using ontologies is a very challenging task. A lot of trace-off between expressivity and logical complexity has to be made, especially given the goal of the authors, which is about supporting the building of semantic legal applications. The authors have done a substantial amount of work and I truly believe that there are a lot of important lessons that others could learn, either for modeling the legal domain or about the actual strengths and limitations of Semantic Web technologies for supporting this very complex domain.

The revision is certainly a strong improvement, however, I still have a remaining concern and some minor comments.

*** The novelty of the work remains a little bit obscure.

By adding an example scenario in the revised manuscript certainly helps with understanding and appreciating the complexity of the legal domain that the authors work with. However, in my opinion, the novelty statement of the work could be improved.

Clearly, there are novelties in this work. At the moment, I found the following different novelty claims in the paper (and there might be others):

- in the abstract: “Unlike many other legal ontologies however, LOTED2 is designed to support the creation of Semantic Web applications”
- in the introduction (page 2): “a novel approach to the design of legal ontologies, which aims at describing the data through an effective representation of legal concepts, while also taking into account the issues arising from the integration of legal ontologies with other vocabularies and ontologies not related to the legal domain”
- in section 3.1 (page 6): “The work we are going to describe here, namely the LOTED2 ontology, could be indirectly considered a sort of ‘experiment’ carried out having in mind these questions, and thus, trying to give them an answer. ”

I think all of these statements are correct. But they are dispersed in different parts of the manuscript, which makes it really hard for readers to grasp the key messages. It will be great if the authors could say right in the front of the paper what the novelties are and what the contributions of this work are and consistently emphasize them throughout the paper.

*** Minor comments

*** On page 3, the paper says: “Compared to the original LOTED ontology, LOTED2 is designed specifically for supporting the modeling of European procurement notices and the description of the data extracted from the TED sys- tem, taking also into account the legal aspects featuring the domain.”

How about also emphasizing that LOTED2 provides a better modelling of the legal domain than LOTED, which is actually described in section 2.1?

** On page 6, what does this sentence “In a certain sense, a legal system can be considered as the whole of all the relations among these concepts” try to say?

*** on page 6, the second paragraph says: “On the other side, in the Linked Data trend, lightweight ontologies like FOAF seems to rule the scene and, given their complexity, there seems to be ‘no room’ for legal ontologies in the Linked Data scenario; they seem to be destined to dedicated and closed systems, performing complex tasks, but not integrating themselves into the web of data. Nevertheless, ontologies cannot be intended as a way to simplify the complexity of legal knowledge, in the name of the Semantic Web.”

There are a couple of issues in this paragraph:
- There are a lot of strong and seemingly arbitrary statements, like FOAF rule the scene, no room for legal ontologies, etc. I think any statements like such should be supported with evidences. Just because FOAF rules the scene (if this is a correct statement), it does not mean that we don’t need a proper ontology for the legal domain, do we?
- There are a lot “seem to” in this paragraph, which gives a dubious feeling to the statements, which should be rephrased.
- I would not agree with the last sentence. Ontologies should not be simply regarded as a way to simplify a domain. If the authors intended to make a comment about the difficulty to cope with the complexity of the domain given the limited expressivity of logic, then lease rephrase the last sentence.

*** on page 6, “Anyway, though the Semantic Web technologies can be helpful also for the needs related to the legal knowledge representation”.

“Anyway” is too slang.

*** on page 6: “Although LOTED2 does not inherit” … “it clearly refers to this core ontology as a source of patterns”

Unless it is really said “clearly”, I would not use this word here.

*** on page 7: “Besides this, almost obvious, consideration, another aspect concerning specifically the European tender notices deserves attention.”

I don’t understand this sentence. Please rephrase.

*** on page 7: “On the TED system is available a full version of each tender document in the original language”

… “what” is available “is” …. ?

The first review identified two issues to be further developed by the paper: (i) the degree of involvement of domain experts in the modeling activity; and (ii) the methodology for the design of the ontology.
Section 3.1 now specifies the domain expert’s contribution to the modeling task. Section 3.2 has been considerably extended to meet the second shortcoming. The design process is now explicit and it is clear how terms have been chosen and provided with a suitable semantics. The authors describe a "backward path" to recover the interpretation of decontextualized terms: Corpora seem to have been used only for concept interpretation and not for terminology extraction, since terms have been directly extracted from TED tabular summaries. This approach is interesting in that it effectively reuses existing terminological resources (like the TED tabular summaries) by recovering the semantics from the original source corpora. The method builds thus on previous term-oriented domain analysis and concentrates efforts mainly on conceptual modelling, striking a balance between precision and development costs. As a suggestion for further work, it would be interesting to evaluate the "backward path" method by comparing the coverage of its resulting ontology with other (more costly) methodologies which include as an initial step the semi-automatic extraction of terms from raw corpora. This kind of comparison would enable to make a cost-benefit analysis of the methodology suggested by the paper.

I thank the authors for their elaborate response, and for addressing many of my concerns in the revised version of the paper. The paper has improved in many respects, but I am still somewhat critical about the handling of two of my initial comments that I would very much like to be addressed.

First, the authors still fail to make *explicit* how the ontology contributes to solving the problems with TED. In their description of the underlying design decisions, as well as in the scenario in the conclusion, the authors primarily illustrate the complexity of the *problem* (scenario on p.17), and not how their *solution* solves that problem. In their response, the authors mention that they have tested the ontology against actual data: I would like this evaluation to be reported on in the paper. What does the instantiation of a tender or a bid look like? What information does LOTED2 infer? How does this help the user of a system that would be driven by LOTED2? How does the ontology help to "filter[ing] tender notices in such a way that the right information can reach exactly its potential users"? How does it alleviate skepticism of commercial parties with respect to public procurement?

I looked at the ontology myself (from [1]) and found a set of example instances, e.g. ProposedContract1, Tender1, Lot1 and Lot2 and related, that could be used to illustrate the data discussed in figure 8.

Second, and related, the discussion of the lightweightness of the semantics in LOTED2 compared to LKIF Core remains a bit muddy. This could again be addressed with a more lucid example of the application of the ontology. Also, this relates to the 'lessons learned'. Currently these are interspersed in the paper, in the discussion of all the design decisions, but a simple bulleted list in the discussion/conclusion section would help a lot.

In summary, I still think the paper could use some work with respect to the communication of results. Without an explicit evaluation and example, it is unclear to the reader (and the reviewer) how to assess the importance of the work.

[1] http://loted2.googlecode.com/svn/trunk/core/core.owl

p2.

"implies to rethink" is not proper English
"17% of EU Gross" -> "17% of the EU Gross..."
"governments open data" -> "open government data"
"public contracts notices" -> "public contract notices" (appears twice)

p5.

Why is section 3.1 not related work?

p6.

"like FOAF21 seems" -> "like FOAF21 seem"
"web of data" -> "Web of Data"
"Linked Data statement on legal content" -> "Linked Data statements on legal content"

"Anyway, " -> that's highly informal use of language, consider using a slightly more "proper" term.

p8.

"they provide sorts of conceptual" -> ...?

p9.
Section 3.3 could be inserted earlier on in the paper.
"compose LOTED2-core ontology" -> "compose the LOTED2-core ontology"
Layout of Section 3.4 is rather messy.

The link to LOTED2 redirects to a page not found message on the CIRSFID website.

p11.
"In this module are described the" -> "This module describes"
"Procurement Regulation module of LOTED2" -> "The Procurement Regulation module of LOTED2"

p16.
First paragraph of section 5 is not really proper English.