Developing an Upper Ontology of Behaviour: An Online Collaborative Approach

Tracking #: 3218-4432

Authors: 
Suvodeep Mazumdar
Vitaveska Lanfranchi
Fatima Maikore
Harriet M Baird
Fabio Ciravegna
Vyv Huddy
Paul Norman
Richard Rowe
Alex Scott
Thomas L. Webb

Responsible editor: 
Stefano Borgo

Submission type: 
Ontology Description
Abstract: 
Whilst there is a burgeoning body of data available on the relationships between behaviours (e.g. alcohol consumption, political behaviours, sleep, domestic violence, recycling) the community currently lacks a shared understanding of behaviour (e.g. how to characterise and differentiate behaviours) and the ability to integrate insights on the relations between behaviour across the existent data. This paper proposes a novel approach, based on online co-design to construct an `upper ontology of behaviour that specifies the characteristics of behaviour(s)' that users will be able to contribute to and keep updated. In this paper, we report on our experience of collaborative co-development of ontologies within a specialist domain using real-time interactive tools and reflect on the benefits and challenges of our approach. We also offer a set of recommendations for researchers interested in applying such methods to co-develop ontologies and present the co-created ontology. The work has major significance as it will create a dynamic, open access knowledge base that people can use to conduct their own reviews, creating an essential resource for behavioural science and, by extension, researchers across the wide range of disciplines that are involved in this field (e.g. psychologists, economists, sociologists, political scientists, and so on).
Full PDF Version: 
Tags: 
Reviewed

Decision/Status: 
Reject

Solicited Reviews:
Click to Expand/Collapse
Review #1
By Janna Hastings submitted on 25/Sep/2022
Suggestion:
Major Revision
Review Comment:

The manuscript describes a collaborative and participatory effort to co-design an ontology of attributes of behaviour. This is a welcome contribution as behaviour has been an under-represented domain within the larger scale of ontology efforts. The authors make many very good points about the scope and nature of behaviour as an entity, and about the process of co-designing an ontology through participatory online workshops. I applaud the authors for the work and for the approach that they have taken.

However, while a strength of the paper in general is its description of the ontolgoy development process, this is also its main weakness as a contribution of type 'Ontology description'. Despite the name of the paper, the bulk of the content is about the participatory workshops, their rationale, their process, and qualitative evidence about the experiences of the participants, which experiences are not directly related to the ontology either.

As far as I can tell, only 1-2 pages of the paper are dedicated to the description of the ontology. And this description then lacks many details that would be expected in an ontology description paper. My suggestion to the authors would be to acknowledge that this very nice paper is about the process rather than the outcome (changing the title accordingly), and perhaps create a different but more substantial contribution that is mainly about the ontology itself which then would be suitable as an ontology description article.

Specific comments according to the journal's review criteria are included below.

(1) Quality and relevance of the described ontology (convincing evidence must be provided).

The case for the relevance of the ontology is well made. The ontology includes very useful attributes for the description of behaviour. However, the paper does not include enough detail about the content included in the ontology in order to evaluate its quality. And as far as I can tell, the online OWL upper level file does not include further detail either - there are no annotated definitions etc. for the entities or object properties.

(2) Illustration, clarity and readability of the describing paper, which shall convey to the reader the key aspects of the described ontology.

The paper reads well, given that its main topic is the process followed in the ontology development. The description of the ontology is however too short and not well developed, and mainly exposed via a Protege screenshot.

Please also assess the data file provided by the authors under “Long-term stable URL for resources”. In particular, assess

(A) whether the data file is well organized and in particular contains a README file which makes it easy for you to assess the data,

The ontology is hosted in a GitHub repository, there is a README.

(B) whether the provided resources appear to be complete for replication of experiments, and if not, why,

N/A

(C) whether the chosen repository, if it is not GitHub, Figshare or Zenodo, is appropriate for long-term repository discoverability, and

Yes, GitHub is a suitable location.

(4) whether the provided data artifacts are complete.

The data artifacts appear to be complete as per what is described in the paper. However, they lack detail that I would expect in a fully developed ontology, which is also acknowledged through metadata and comments in the OWL files pointing to the preliminary nature of the resource.

Review #2
Anonymous submitted on 06/Oct/2022
Suggestion:
Major Revision
Review Comment:

The paper propose a solution to the problem: The community of behavioural scientists and related stakeholders "lacks a shared understanding of behaviour (e.g. how to describe and differentiate behaviours) and the ability to integrate insights on the relations between behaviour across the existent data."

The proposed solution is to "build a ‘collaboratively edited ontology’ that users will be able to contribute to and keep updated."

The paper is well written and generally well organized.

The use of github.com repository to store the ontology is well suited for this task, a collaborative remote process for designing and maintaining an ontology.

The paper has some content issues. Since it's being published in a SemanticWeb journal, the authors would benefit from using terms, background literature, and methods accepted by the SemWeb community. Since it was submitted as a 'Ontology Description', it would benefit from a more thorough description and analysis of the ontology. The proposed ontology is not that interesting, and the main focus and contribution of the paper is the process of ontology engineering by remote collaborators.

Ontology principles analysis could use more work. Why use ontology engineering proposed by [4] (Pinto and Martins)? This is OK, but some comparative analysis with others would be helpful, such as : Grüninger, M., & Fox, M. S. (1995). Methodology for the Design and Evaluation of Ontologies. International Joint Conferences on Artificial Intelligence Workshop on Basic Ontological Issues in Knowledge Sharing.

The ontology being developed does not look like an upper-level ontology, but a domain-level ontology. This should be addressed or changed throughout the manuscript.

The ontology itself was not analysed in any meaningful or recognized way to see if the collaborative design process was successful. Some concept clusters were given a label in Figure 1, which makes sense, but I would not say this proved the process successfully created a useful ontology. Table 2 provides statistics on upper-level and reused ontologies, but I am not sure how this is helpful. Some context would be useful, for example explaining how FOAF, OM, SEPIO, and Time ontologies were used in the designed ontology.

The only identifiable analysis that was done is on the correlation properties. Meaning, there was agreement on how correlated concepts were. While I welcome quantitative properties into ontologies, i.e. something that would benefit reasoning with weights, it is not clear how the weights were established. They are not probabilistic, but subjectively chosen by the subject matter experts.

I'm not clear parts of Section 4 and most of Section 5 are useful. Section 4 provides examples and discusses the types of questions and discussions made by participants. For example, the discussion on colours used is not meaningful, and could have been mentioned briefly, if at all. While the subsections in Section 4 are useful, the quotes are not adding enough to take up as much space as they do. A simple summary and discussion would be sufficient.

Section 5 reads like a general analysis of the online tools, rather than an analysis of how the tools are used for ontology engineering. I would think the same would be achieved with some pruning into a discussion paragraph.

While I believe this work is important, the ontology that was developed and the analysis of it are not well presented, or perhaps don't fit well into this category. I recommend a resubmission after major revisions.

Review #3
Anonymous submitted on 06/Nov/2022
Suggestion:
Reject
Review Comment:

This is perhaps the shortest review of a paper I've ever written: the text titled "Developing an Upper Ontology of Behaviour: An Online Collaborative Approach" does not meet the requirements of a research paper. It might be considered a report of an accomplished project, but even in this case, I would have mixed feelings if I were representing a funding organization. The methodology and tools described in the text have no signs of novelty. The "ontology" created in the project is a random set of concepts without proper definitions and axioms. It might be worth mentioning that the ontology metrics (p. 13) do not correspond with the metrics of the "ontology" I found under the link https://github.com/fatibaba/turbbo/blob/main/upperlevel/turbbo_upper_lev... even some properties have a different name in the text and the ontology, e.g., isHabitual (in text); isHabit (in turbbo_upper_level.owl).
And last but not least, the Related Work section contains only a description of collaborative tools. There is no mention of any ontology of action or behavior known on the market. The idea of ontology, and the semantic web in general, is to reuse and share. So, why start from scratch?
I do not recommend this text for publication in SWJ.