When Things Go Wrong: The SENSE Ontology for Explaining CPS Anomalies

Cyber-Physical Systems

I think this paper makes a good contribution in that it connects and extends existing ontological ideas to better fit the case of CPS. It takes some challenging ideas, such as explanation and causation along the way. In this sense, the vision is broad and aspiration. I like this. Some design decisions might be debatable, or could be achieved in more than one way (the authors say this) but I think the path taken here makes good sense in terms of what it enables. We wait a long time to see the results in Section 8, but when they come, they illustrate well the capabilities of the system

Nice introduction to the problem in sections 1 & 2. These sections are well-written, and introduce the main ideas and related work in a way that avoids becoming bogged down in details. It makes the work accessible to a broader audience.

Section 3 compares the ontologies that could be extended to support this work. I’m not familiar with the details here, but the presentation is clear and connects clearly with the needs & use-case the authors provide.

I like and support the idea of extending SOSA to support this work. It seems like a solid foundation for what you propose.

Section 4 provides the methodology for ontology design & implementation, using the LOT methodology.

Spell out CQ the first time you use it.

“While chowlk proved to be a useful tool for conceptualization, it lacked some capabilities to be able to use it as a full implementation tool.” Can you be more specific here? What did it lack? Why was this important? Too informal? No support for relation attributes? Etc.

In section 5, how would you fit into your 3 types of knowledge the case of an electric car that receives regular software updates, thus is not necessarily static, and might gain new features or ways of working, or ways or reporting. But I think falls into the systems knowledge category?

I really like, and value, the idea of tying together closely the ideas of causality, and explanation to events and observations. I can see all kinds off applications for this kind of connected logic system, from managing equipment in hospitals to better instrumenting the science process itself. This is good work.

I found 5.3 interesting. I can appreciate the choices here, and the need to not be too prescriptive in advance of the use-case.

Inferring Causality
In the domains I work in, correlation is not causation and we would not infer causality so lightly, but over time, based on repeated patterns of observation. I can appreciate it may be different here. But what if you define a new state causality and then encounter a new state in the future where this does not hold? Is it even possible to use an accumulation of evidence to support this kind of inference, as opposed to an observation? And what happens if you end up with two StateCausalities that seemingly contradict each other? Is this allowed? Surely it must occur?

I like the ideas around personalisation, and supporting user needs more precisely (5.4). User roles and access rights play a key role in clinical/medical systems, it is good to see them covered here. I know you get to evaluation later, but I think it would help to give an example of contrasting personalisation of the same information.

I’d like to see an example of an explanation presented in 5.5 in the context of a simple case.

The evaluation (Sect 7) seems reasonable to me. Good to see a mix of methods used here.

Section 8 works though use-cases in detail, explaining how the various parts of the ontology interact during operation. I think it might be useful to pull out the various instances events discussed and turn them into a narrative for the intro to the section, THEN show how they are each addressed. It think it would not only make it easier to follow, but also more compelling. As is, it is difficult to know where the explanation is going next, or why.
The authors do this in 8.3, I think it should be moved before 8.2.

Fig 8 is helpful and summarises the example well, making it clear how the system effectively translates events into actionable intelligence. Nice.

Section 9 could be a little more reflective. What still needs to be done? What could be improved? What decisions in the design might need revisiting for different applications than the ones shown here?

This paper presents the SENSE ontology, which aims to explain anomalies in cyber-physical systems (CPS) where sensor data is generated but lacks sufficient semantics for further analysis. Overall, the paper is well written and includes a clear introduction, a discussion of related work, a rationale for the ontology development, and a detailed description of the ontology supported by well-documented examples. In addition, the authors present data validation, ontology evaluation, and a proof-of-concept application.

Strengths:
•The development of the ontology follows a well-established methodology.
•The presentation of how the methodology is applied, as well as the description of the ontology itself, is clear and highly readable.
•Furthermore, the evaluation aims to guarantee the quality of the ontology.

Weaknesses:
•Some related work and existing ontologies could be revised to improve clarity.
•It would be worthwhile to include SPARQL query examples to demonstrate how the competency questions (CQs) are answered.
•Some claims, such as “domain independence” and “framework independence,” are difficult to justify without proof-of-concept applications in different domains and with different frameworks.

Some detailed comments are listed below:

•P4, Section 3: It is not clear how the authors decided to include these ontologies in the related work. Are they derived from an ontology survey? Are there other relevant ontologies that were excluded, and if so, why?
•P5, Table 1: The table mentions SAREF4SYST [10], while the text refers to SAREF4INMA [11].
•P5, line 17: The statements “SOSA and SAREF are designed for … CPS” and “The SOSA ontology … in CPS” are ambiguous. Readers may unintentionally assume that these ontologies were developed with a specific CPS focus, which may not be the case. It would be clearer to state that these ontologies are applicable to the knowledge representation of CPS to some extent.
•P5, line 38: Why are only two event-related ontologies mentioned, considering that more are surveyed in https://ceur-ws.org/Vol-3443/ESWC_2023_SEMMES_XPEvent.pdf?
•P6, line 13: “GitHug” → “GitHub.”
•P6, line 44: Readers may not be familiar with Chowlk. It would be useful to briefly introduce the purpose of this tool.
•P6, line 51: “OOPS” → “OOPS!”
•P7, line 31: Readers may not be familiar with MODL.
•P10, line 4 (Chowlk visualization): observedProperty should originate from AP1_Observation, and AP1_Observationshould be AP1_Observation1, according to the Turtle excerpt.
•P10, line 10: The term “RDF relations” is unclear. What is the semantic type of such a relation?
•P10, line 48: The CaTeRS scheme appears here for the first time. Is this related work that should be introduced in Section 3?
•P11, line 18 (Figure 5): OEViolationState is used in the Turtle excerpt, while OEViolationState1 appears in the Chowlk visualization.
•P12, line 4 (Turtle excerpt): containsAccessRights should have UserRole and AccessRight as its domain and range, respectively. However, the subject and object are not correct in the example.
•P12, line 14: containsMitigationOption should be used instead of hasMitigationOption.
•P14, line 15: The statement “which was a design decision by SOSA” should be revised and clarified. SOSA uses schema:domainIncludes as an annotation to represent multiple domains.
•P14, line 17: The SENSE ontology reuses a number of classes and properties from SOSA/SSN rather than explicitly importing the ontology.
•P17, Figure 8(b): requiresControlAccessRights is not present in the ontology.

Comments on the ontology file (https://github.com/wu-semsys/SENSE-Ontology/blob/main/ontology%20files/S...):
-rdfs:subClassOf should be used instead of rdfs:subclassOf for sense:ObservableProperty, sense:Event, sense:Observation, sense:Platform, and sense:Sensor.
-Some rdfs:comment entries should be revised, for example, sense:allowsControlAccessTo contains a typo (“ontrol access”).
-The comments for sense:associatedPlatformType and sense:associatedProperty are unclear, as there is no result-type class defined in the ontology.
-For the property cause, it is unclear how its usage is distinguished between an object property and a non-specific relation. The defined domain and range conflict with the comment. The same issue applies to the effect property.
-The domain of sosa:hasFeatureOfInterest is defined as sense:Observation. Why is sosa:Observation, which is already reused in the ontology, not used instead? Similar comments apply to sosa:hasResult, sosa:hosts, sosa:isHostedBy, sosa:madeObservation, sosa:observedProperty, sosa:observes, and sosa:usedProcedure.
-The comment for sense:hasUserType is unclear.
-The comment for sense:observedProperty is unclear due to the mention of ObservationCollection, which is not a class in the SENSE ontology.
-The label for qudt:hasUnit should be “has unit.”

Overall, the paper is well written as an ontology description paper and provides reasonable evidence demonstrating its quality and relevance. However, a number of issues should be addressed before the paper can be accepted by the Semantic Web Journal.

The paper presents, an ontology intended to support semantic explainability by modelling system knowledge and enabling the representation of causal relations among system states. The ontology is developed following the LOT methodology and is organized into several modules addressing different aspects of the domain. The paper also presents a proof of concept illustrating its application and reports a structural evaluation of the ontology.

The paper is generally well written and easy to follow. The ontology is openly available through GitHub, accompanied by documentation, and is identified by a persistent w3id URI, which is consistent with good Semantic Web publishing practices.

However, I believe that the paper currently falls short of providing convincing evidence regarding the ontology engineering process and the quality of the ontology, which are among the main evaluation criteria for ontology description papers in the Semantic Web Journal. Several important modelling decisions are insufficiently motivated, the application of the LOT methodology is only superficially described, and the evaluation does not adequately demonstrate that the ontology satisfies its intended requirements. As a result, it is difficult for the reader to assess whether the ontology effectively fulfils the goals stated in the introduction.

=== Major comments ===

1. The ontology engineering process is insufficiently documented
The paper states that competency questions (CQs) were elicited through workshops with domain experts, but the process through which they were derived remains largely unexplained.
Several important questions remain unanswered. Namely:
* At which stage of the ontology engineering process were the CQs defined?
* How many domain experts participated?
* Were the CQs produced during a single workshop or refined iteratively throughout the ontology development?
* Were they modified as modelling decisions evolved?

These aspects are important because the LOT methodology explicitly relies on competency questions to drive ontology engineering. Without describing their elicitation process, it becomes difficult to evaluate whether the proposed methodology has actually been applied as intended.

Moreover, the competency questions themselves are neither reported in the paper nor made available as supplementary material (e.g., in the GitHub repository). This omission makes it impossible to verify the claimed coverage of the ontology requirements.

2. The relationship between the stated objectives, competency questions and ontology design is unclear
The paper motivates the ontology as “a core component of a semantic explainability framework … designed as a foundation for modelling system knowledge and deriving causal relations between system states at runtime.”
However, the paper never demonstrates how these objectives are reflected in the competency questions nor how the ontology modules address them.
Consequently, there is no clear traceability between: (i) the initial requirements; (ii) the competency questions; (iii) the ontology modules; and (iv) the final evaluation.
Establishing this traceability would considerably strengthen the paper.

3. The application of the LOT methodology requires a more detailed description
Although the paper states that the ontology was developed following the LOT methodology, its actual application remains largely implicit.

In particular, it is unclear: (i) how the ontology modules were identified; (ii) why the ontology was modularized in its current form; (iii) how the competency questions informed the modularization; and (iv) which LOT activities produced the different modelling artefacts.

Since ontology engineering methodology is one of the core aspects expected in ontology description papers, the authors should provide a more explicit account of the engineering process.

4. Several modelling decisions require stronger justification
The paper describes the ontology almost exclusively at a textual level. While this improves readability, the lack of formal axioms makes it difficult to understand the intended semantics of the ontology and to assess the modelling decisions.
In particular, several important choices deserve additional justification. Namely:
- The ontology claims to reuse SOSA for modelling observations, yet the paper only briefly mentions this reuse without explaining which SOSA classes and properties are adopted or how they integrate with the proposed ontology. More generally, ontology reuse is discussed only superficially despite being an important aspect of ontology engineering.
- Similarly, the representation of causal, temporal and topological relations raises several questions. Since the ontology is intended to represent causal knowledge extracted from natural language, it is unclear why these relations are modelled as literal values rather than as ontology individuals or terms belonging to controlled vocabularies. This design choice appears to reduce semantic expressiveness and interoperability.
- Regarding topological relations, the adopted modelling solution also appears less expressive than the corresponding representation already available in SOSA. The paper should therefore better justify why an alternative representation was preferred.

5. The reasoning mechanism is insufficiently described
Section 5.3 discusses causal inference but does not explain how inference is actually performed.
It remains unclear whether inference relies on Description Logic reasoning, SHACL rules, custom reasoning procedures, or another mechanism altogether.
Since inference is presented as one of the intended capabilities of the ontology, this aspect deserves a more explicit explanation.

6. The ontology evaluation provides only limited evidence
The current evaluation mainly consists of structural verification and competency-question-based SPARQL validation.
While structural verification is certainly useful, it does not provide sufficient evidence regarding the quality of the ontology according to commonly adopted ontology evaluation criteria (see, for example, Gangemi et al. [1]).
Furthermore, the paper does not explain how the SPARQL queries were derived from the competency questions. Since the competency questions themselves are unavailable, it is impossible to assess whether the verification actually demonstrates that the ontology satisfies its intended requirements.

7. The proof of concept could better support the presentation
The proof of concept is presented relatively late in the paper. Introducing it earlier would help readers better understand the design motivations underlying the ontology.
In addition, the paper introduces three potential application scenarios but only evaluates one of them through the proof of concept. The rationale for selecting only this particular scenario should be explicitly justified.

=== Minor comments ===

- Page 6: “GitHug” should be corrected to “GitHub”.
- In the Introduction (page 2), the paper discusses the limitations of ontologies in dealing with real-time data. This limitation seems more appropriately associated with knowledge graphs or knowledge graph management systems rather than ontologies themselves, which primarily define conceptual schemas.

1. Gangemi, A., Catenacci, C., Ciaramita, M., Lehmann, J. (2006). Modelling Ontology Evaluation and Validation. In: Sure, Y., Domingue, J. (eds) The Semantic Web: Research and Applications. ESWC 2006. Lecture Notes in Computer Science, vol 4011. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11762256_13