Building a Knowledge Graph of Distributed Ledger Technologies

This manuscript was submitted as 'Ontology Description' and should be reviewed along the following dimensions: (1) Quality and relevance of the described ontology (convincing evidence must be provided). (2) Illustration, clarity and readability of the describing paper, which shall convey to the reader the key aspects of the described ontology. Please also assess the data file provided by the authors under “Long-term stable URL for resources”. In particular, assess (A) whether the data file is well organized and in particular contains a README file which makes it easy for you to assess the data, (B) whether the provided resources appear to be complete for replication of experiments, and if not, why, (C) whether the chosen repository, if it is not GitHub, Figshare or Zenodo, is appropriate for long-term repository discoverability, and (4) whether the provided data artifacts are complete. Please refer to the reviewer instructions and the FAQ for further information.

This paper introduces a knowledge graph and an ontology for distributed ledger technologies (DLT). The authors looked beyond the technical aspect of DLT. They extended the knowledge graph to include security considerations (such as threat models and vulnerabilities), application domains and relevant compliance standards and regulations. The authors hope and expect that the knowledge graph and the ontology improve the understanding of distributed ledgers, not just focusing on a single aspect of technology but a wide range of use cases.

Strength:
1. Although there are typos here and there in the manuscript, overall, the paper is well-written and easy to understand.
2. Regarding the ontology design, my overall evaluation is that the ontology looks correct to me and follows a sound methodology (except for a missing step described below).
3. The URL and website hosting the ontology look stable and well informative.

My concerns for accepting the paper "as is" speak to its scientific merits and the significance of its contributions as rationalized below:

1. While ontology design in its infancy was considered a scientific endeavour, after many years of ontology design efforts for various domains, if the requirements for the domain are known, then the design of an ontology per se is a straightforward task. As an analogy, I think the authors agree that no scientific journals in the relational database community will accept a very well-designed database, even for a sophisticated domain, as a scientific contribution. While I appreciate the efforts of the ontology design, I do not see any scientific novelty in this work.

2. The second issue I have with the paper is its lack of user involvement in the design process. In fact if the authors had defined the project as application-oriented research (as opposed to knowledge-oriented research), they could have added elements of novelty by user involvement. For example, the authors states " Such a knowledge graph improves the overall understanding of distributed ledgers, reveals their strengths, and supports the work of security personnel, i.e. analysts and system architects." I do not see anywhere in the paper where this claim is being assessed, has this ontology been used by a specific community and has the impact been reported?

3. Having points 1 and 2 in mind, the competency questions reported in Section 3.3 seems to me superficial. Why are these queries and not further queries? For example, currently, the designed ontology only describes the static state of a DLT instance; what about the dynamic aspect? what if, over time some of the cryptographic schemes turn invalid, how the ontology can handle these cases?

4. I'm glad that the authors indicated the study limitation in the very last sentence of the paper as "While it was possible to gather information on the technical components of DLT systems, existing standardization documents, and potential use cases in different industries, this requires intensive research, extensive manual work, and expert knowledge in the domain." Addressing this limitation is exactly what translates to a research contribution in ontology design nowadays and not the design itself.

5. Some minor issues:

5.1. I think it's not a desired habit to design yet another ontology from scratch. there are plenty of ontologies out there and any new ontology should make use of the existing ones. This should be part of the ontology design methodologies.

5.2 The authors state "Existing ontologies and vocabularies typically focus on a limited sub-group of distributed ledger systems or build an ontology for a single distributed ledger product or application." A thorough comparative analysis is needed to support this claim.

5.3 there are typos in the paper, e.g., P1, Line 47 why both references and footnotes?, P2, Line 5, Sentence finishes without a period. P2, Line 44: "There are many more blockchain applications and cryptocurrencies are merely one aspect of the broader picture." incorrect parallel structure.

This manuscript was submitted as 'Ontology Description' and should be reviewed along the following dimensions: (1) Quality and relevance of the described ontology (convincing evidence must be provided). (2) Illustration, clarity and readability of the describing paper, which shall convey to the reader the key aspects of the described ontology. Please also assess the data file provided by the authors under “Long-term stable URL for resources”. In particular, assess (A) whether the data file is well organized and in particular contains a README file which makes it easy for you to assess the data, (B) whether the provided resources appear to be complete for replication of experiments, and if not, why, (C) whether the chosen repository, if it is not GitHub, Figshare or Zenodo, is appropriate for long-term repository discoverability, and (4) whether the provided data artifacts are complete. Please refer to the reviewer instructions and the FAQ for further information.

---

The authors present a KG of distributed ledger technologies (DLT) that includes an ontology of relevant concepts and relations. The recent focus on cryptocurrency and blockchain has belied the variety of existing DLT technologies. The targeted KG focuses on a broad perspective of DLT, their security considerations, real-world applications, and standardization and legal perspectives. A systematic approach to constructing the ontology was used, and the KG was evaluated using a set of competency questions.

The topic is quite interesting and suitable for the semantic web journal. However, I feel there are some major problems that should be fixed before publication.

(1) Quality and relevance of the described ontology. Most importantly, after trying some competency queries, the contents of the KG seem rather underwhelming. There are only 9 types of components (many more consensus mechanisms exist, notably, proof-of-stake, which greatly improves upon proof-of-work), 3 (!) DLT systems, and 2 use cases with specialized DLT systems - this is a part I was especially interested in. Multiple classes are not instantiated (OrganizationalControl, Mitigation, ..). As a result, it seems that not all competency questions can actually be answered (e.g., I3 (no smart car, public transit use cases), S5), which seems to imply that the evaluation proposed by the authors was actually unsuccessful.

With regards to the ontology design, the choice for an RDFS ontology is not motivated. It seems that, if OWL had been used, multiple properties could be inferred using OWL axioms. For instance, isUsedFor could be derived from isSpecializedFor & hasBusinessSector; hasUseCase and hasBusinessSector seem inverse versions of each other; threatens (range DLTComponent) could be inferred from canExploit & hasVulnerability, and threatens (range DLTSystem) could be inferred from canExploit & hasVulnerability + hasComponent. By only asserting core knowledge, and then deriving inferred knowledge, one can keep the "core" ontology more compact, and avoid redundancy and inconsistency. The "controls" and "mitigates" properties are a bit unclear - giving some examples of controls and mitigations would help.

- spelling
As a matter of style: instead of "in [1]", consider using "X et al. [1]..."; "Reports like [7]" -> "Reports as by Deshpande [7]", especially in cases such as "by the authors of [25]" -> "by Werbach [25]", etc.

"describes the ontology In detail, and it explains" ->
"describes the ontology in detail, and explains"

"a data structure that (cryptographically) links blocks of data into a list of blocks" ->
"a data structure that (cryptographically) links blocks of data into a chain of blocks"

"3. Second, we define" ->
"3. We define"

"we instantiate the developed ontology by with" ->
"we instantiate the developed ontology with"

"Which normative references do exists" ->
"Which normative references do exist"

"of an industry form an initiative" ->
"of an industry to form an initiative"

"Domain: IndustryInitiatives" ->
"Domain: IndustryInitiative"

"in a knowledge graph: While" ->
"in a knowledge graph: while"

(2) Illustration, clarity and readability of the describing paper. The paper and KG scope is rather inconsistently defined throughout the paper, making it difficult to grasp what the exact focus is. Section 3.1 describes the scope differently from the introduction, where the focus lies on security; mentioning more generally "factors which affect the implementation and operation of distributed ledgers". Section 3.2 then introduces 3 major areas of technology, business & market use, legal & standardization; this really seems to belong in section 3.1. Then, Section 3.3 introduces 3 different categories, namely technology & security, industry & application, and standardization & regulation. Even if they were synonyms (don't think they are) it's unsure why the same terms are simply not used consistently. It is also rather unclear what standardization means in this context; interfaces, capabilities, underlying technology, interoperability with other DLT or IT systems, ...?

A) - C) and (4): the associated data arteficts are satisfactory, and the authors did a great job on the online documentation and data files. The online SPARQL endpoint is very useful. However, note that the default example in SPARQL endpoint (https://dlt-ontology.github.io/query/sparql.html) does not yield any results - one needs to add "DLT Knowledge Graph" as well. And, the way T1 is written does not yield useful results (components are blank nodes). It would even be more useful if all competency questions can be selected from drop-down.

This manuscript was submitted as 'Ontology Description'.

In terms of completeness and organisation of the resource in the chosen repository: Excellent.

In terms of Illustration, Clarity, and readability. Sufficient. I have one remark about the introduction of a Knowledge Graph in the title and as part of the justification. This addition makes the paper difficult to assess as one does not know how central the knowledge graph is to the contribution. Knowledge graph papers have different assessment criteria.

In terms of quality I think it's OK, but I don't see enough relevance. My main concern is that it is not clear how one goes from the collected information to the competency questions, in other words from steps 2 and 3 of the methodology. At the moment, we just have that some desk research has been done that justifies the importance of the general categories of the competency questions, but not how the specific competency questions were developed based on that information. I think it boils down to the need of more details on the methodology: Is it the questions were in the discussion of the papers? is it that you encoded somehow the information of the desk research in a way that then led to the questions? This would greatly strength the argument on the relevance of the ontology.

This paper tackles an important area: describing Distributed Ledger Technologies with an ontology to lessen misunderstandings. The authors have carried out a survey of literature, setup competency questions, created an ontology, instantiated the ontology and evaluated the model.

Quality and Evidence

The main issue I have with this paper is that it does not capture the main concepts that DLT practitioners and researchers use to discuss the area. The biggest is **consensus mechanisms**. A great deal of research and discussion is centred on how blockchains reach consensus. The most well known blockchain, Bitcoin, uses proof of work and this consumes a lot of power. Other consensus mechanisms have now been proposed and deployed including: proof of stake, proof of time, proof of authority. Other concepts which are not represented include: payment mechanisms, smart contracts, token systems, decentralised organisations, mining, transactions.

I realise that this paper is on DLTs rather than blockchains but the 'main action' is currently within the blockchain field and this ontology is at a level above anything that would be of interest to the main community.

I would at least to expect concepts related to, for example the work of Vitalik Buterin such as https://ethereum.org/en/whitepaper/ to be included.

Illustration, clarity and readability

In general the paper is readable however it is never a good idea to have headings which cover 2 or more concepts with an 'and' in the middle. See Table 1 with 'Technology and Security', 'Industry and Application' and 'Standardisation and Regulation'.