Review Comment:
This is an extension paper from a Rule+RR paper published in 2023 by the same authors. The extension involves leveraging RDF and SPARQL 1.2 to write and validate data governance policies. The topic is relevant as there is a need for systems to aid organizations in keeping track of their obligations in an effective and efficient manner.
Even if the paper is relevant and the extension from the original paper is considerable, I have three main issues with the manuscript, which are as follows:
1. The introduced definitions are not sufficiently readable or easy to understand. Even if the paper is intended for policy experts, database professionals implementing systems using GUCON should be able to follow it. The authors can check my detailed reviews to see concrete improvements, but the most important weak points I see in this regard are as follows:
- There is an underutilized running example. The authors spend a lot of time on their health-related example, which is then largely unused. Most definitions in Section 3.2 should be accompanied by a concrete example from the scenarios described.
- Most definitions are either too informal or overloaded with symbols. For instance, a knowledge base is defined as an RDF graph describing the actual set of knowledge. I don't see this as relevant to distinguish the concept of KBs from RDF graphs if they are the same. Then, the difference between an Action Pattern and an Obligation pattern seems to be the appendage of the = suffix. It is not clear what this "O" means or why these concepts must be understood as different. The concept of condition satisfiability should be right after Definition 6, where conditions first appear (and are rather informally introduced as the definition is for obligation rules)
2. The authors use a rather outdated version of RDF/SPARQL-star. Even a year ago, the working draft had stopped considering subject nesting of triples. This is tricky, given that RDF 1.2 is still being worked on, but the risks of relying on non-standard notation are that these things are not settled and may need constant updates. In my detailed comments, I outline the updates that would make the paper acceptable.
3. The evaluation using Thörn's criteria is rather qualitative, even if it could be more mathematical/quantitative. In particular, with regard to formalness, correctness, and usability. Formalness is said to be achieved because of the expressive power of SPARQL. This is done rather informally in the paper. As per correctness, the authors link some test cases, but there are no quality metrics associated with them. The authors don't explain what the aim of the tests is (accuracy, coverage, robustness, succinctness, etc.). Finally, regarding usability, the authors simply argue that the model should be easy to use, with no validation from actual users. In data governance, there are several roles for people interacting with data; even a small poll of people in those roles could provide some (more objective) evidence for the authors' claims. The authors mention that turtle should be intuitive and user-friendly. Is there evidence that people managing these data policies are proficient in turtle/rdf. And even more, RDF 1.2?
Detailed comments:
- The abstract ends with "we evaluate the extended model and its prototype implementation", but there is no mention of what this evaluation found.
- Page 1, line 47: operationalization [of] these
- from page 2 on, there is a typo in the running authors. It says "N. Akaichi et al.", I guess it should be "I. Akaichi et al."
- Page 4, line 1 (and in some other places), it says "graph patterns expressions"; it should be "graph pattern expressions."
- When the I and L sets are introduced, they must be countably infinite.
- To comply with RDF 1.2, blank nodes should be considered in the paper, and thus introduced here too.
- Definition 3 is too informal, and there is no actual difference between an RDF graph and a KB
- Are sets N, C, R pairwise disjoint? Would it matter? Are they countably infinite, too?
- The difference between action and obligation patterns (other than the bold O) escapes me.
- There is an example below Definition 7; however, I argue that it should follow the format of the definition (i.e., be of the form O(n,c,r) and say what n, c, and r correspond to in the situation.
- Definition 8 appears way after condition satisfiability is informally defined, right after Definition 6.
- Page 5, lines 23-24, the authors say that the subset of variables condition is there to ensure that all variables in the obligation pattern are "already bound" by the condition. At this point, there has been no talk about the evaluation of conditions.
- Page 6, line 30. store -> stores. Also, in the example of Dr. Smith being a doctor, I would argue that this fact could, for instance, be revoked and thus change over time.
- Page 7, line 15. I am not sure if \oplus is the standard notation for an XOR.
- Definition 15 says that O\in D. D has not been introduced afaik.
- Definitions 14--23 also need concrete examples based on the scenarios.
- Definition 18 should remind the authors what "the contents" of OR are (i.e., cond t_start, t_deadline, etc.)
- Figure 2 should be divided into three subfigures, and each of them should be explained in the caption. As it is, the figure is too small, and there is not much text explaining what is going on there.
- Definitions 24 and 25 are outdated. For over a year, RDF 1.2 has not considered nesting of triples in the predicate, only on the object (see https://www.w3.org/TR/2025/WD-rdf12-concepts-20250109/).m These definitions, and examples (in Listings 1--4 should be updated to present nesting only in the subject, and ideally use the rdf:reifies predicate with blank nodes.
- Page 9, lines 37 and 45. These examples should be updated to the most recent RDF 1.2 draft.
- I am not sure that Algorithms 1 and 2 are really needed. Algorithm one simply evaluates (which is what I assume the get_mappings function does) all rules and labels them according to their status with respect to their timelines. Algorithm 1, in fact, opens the door to questioning the complexity of this process: the larger t, the larger the snapshot and thus the slower the evaluation. This is not discussed by the authors.
- Page 10, lines 39 and 45. These examples should be updated to the most recent RDF 1.2 draft.
- Page 11, line 19. Should the mentioned KB be a temporal KB?
- I think that Section 5.3 should appear before 5.2, as in 5.2 the authors talk about policies applied to KBs, but other than from the Definitions, the reader has never seen a policy in RDF 1.2 (these appear in the listings of Section 5.3)
- Listings 1 and 2 are crowded. Just use more vertical space. Also, all these listings should be updated to the latest draft of RDF 1.2.
- I think more concrete evidence should be provided for the completeness, correctness, and usability dimensions of the evaluation, as mentioned in my initial comments.
- I think that the empirical evaluation can be kept as is. I don't believe that updating the system would yield times that are too different. However, for the sake of the paper's impact, I recommend that the authors update the implementation as much as possible to the latest RDF/SPARQL 1.2 draft.
- Table 2 is quite hard to understand. I assume the second column contains the number of rules in the policy, and the third column contains the number of triples in the KB?
- As I mentioned before, since the snapshot gets all triples annotated with a time lower than or equal to t, the snapshots get larger as time passes. Yet, the evaluation does not consider increasing values of t. This limitation should be explicitly addressed and/or evaluated through dedicated experiments.
- The x-axis labels of the charts in Figure 6 have the same issues as Table 2.
|
