Human-based Consensus for Trust Installation in Ontologies

Tracking #: 2479-3693

Christoph Summerer
Emanuel Regnath
Hans Ehm
Sebastian Steinhorst

Responsible editor: 
Oscar Corcho

Submission type: 
Full Paper
Blockchain technologies enable a decentralized peer-to-peer network to reach distributed consensus on transaction data that is written into a blockchain. This data is then considered to be a single source of truth, trusted by the entire network. Many approaches focus on writing financial transaction data into the blockchain, which can be easily verified and validated by machines to reach distributed consensus. However, there exist also other types of data which requires human thinking and collaboration for validating and finding consensus. This is the case for ontologies, which are important building blocks for Semantic Web content but are currently difficult to validate and maintain and would therefore benefit from the guarantees provided by blockchain. In this paper, we propose a novel protocol to represent the human factor on a blockchain environment. Our approach allows single or groups of humans to propose data in blocks which are verified and validated by other humans. Only if human-based consensus on the correctness and trustworthiness of the data is reached, the new block is appended to the blockchain. Our experimental results show that this human approach is an alternative to conventional approaches but significantly extends the possibilities of blockchain applications on data that cannot be verified and validated automatically but requires human knowledge and collaboration.
Full PDF Version: 


Solicited Reviews:
Click to Expand/Collapse
Review #1
By Bogdan Iancu submitted on 22/Jun/2020
Major Revision
Review Comment:

Although the article introduces a consensus-based blockchain model that can help implementing the trust layer of the semantic web, the paper requires, to my opinion, some major revisions before it can be accepted. Except some minor English editing, the quality of writing is good. The originality and significance of results can be improved in the revised version.

The first concern is related to the added value. The idea of using a vote-based consensus is not new (see "Proof of Vote: A High-Performance Consensus Protocol Based on Vote Mechanism & Consortium Blockchain" by Kejiao Li et all, for details). How is this paper different? In my opinion this paper should be discussed in the Related Work section.

Second, the block content is not described in great detail for ontologies. What is the content of a block from the blockchain, an RDF triple? An OWL file? What happens if someone wants to delete an existing node or relation? Do all the participants have equal rights (Create-Read-Update-Delete) or are you considering adding a rights management model also as a future plan?

I agree that the model can be used for anything else than ontologies, but having into consideration the topic of the journal, more details about how it can be applied for an ontology are needed. Maybe a specific example can make things clearer.

Why the 2/3 majority is considered? What are the reasons behind the decision? Why not 50% or 75%?

Related to the token system, cannot this affect the democracy of the network? Isn't it possible for a small number of participants to control the entire network? How can the others fight back against this?

I suggest also to add a remark about Wikidata or DBpedia which are ontologies based on consensus (it is true that they do not use blockchain technologies, but the information is made public on Wikipedia based on consensus).

I will also list some minor changes that need to be made, bellow:

- page 1, line 17, abstract - "Blockchain technologies... into a blockchain"; a definition cannot use the defining term as explanation, please consider rephrasing;
- page 1, right column, lines 38-42 - "This human factor not only influences the way of applying blockchain technologies for such content that cannot easily be classified as wrong or right, but also the way of reaching distributed consensus on it" - citation needed or rephrasing;
- page 2, left column, lines 38-40 - "there is currently no mechanism to integrate human verification and validation into a blockchain architecture" - the article specified earlier proves contrary (in the semantic web context maybe?);
- page 3, left column, lines 40-47 - OWL is more than RDF triplets; please consider rephrasing;
- page 3, left column, lines 47-49 - In my opinion there are enough standardized ontologies. Please see LOD cloud ( for details;
- page 4, left column, line 21 - "Should a two-thirds majority of human..." - please consider rephrasing;

Review #2
Anonymous submitted on 12/Oct/2020
Review Comment:

This manuscript proposes the inclusion of human-based decisions in the context of the operations of a blockchain, and its application in the development of ontologies, so that ontologies can be built and evolve considering the changes that are being proposed in a blockchain block and their analysis and evaluation by humans, so that they reflect the consensus usually needed in ontology development. The approach is implemented, and as the authors discuss, it is not only to be considered as a contribution in ontology development but also as a contribution into how blockchain technologies and approaches can be modified when human-based interactions/consensus are needed.

I will first reflect on the part of the contribution that focuses on improving or providing an alternative for distributed ontology development using this approach. I think that this contribution is not sufficiently strong (nor sufficiently evaluated) for being published as a journal paper, for the following reasons:
- The authors make several claims about ontologies without clear references or studies that support them, and that given my experience in ontology development and governance I would rather say that may be considered incorrect. For instance, in the abstract the authors comment that "ontologies are difficult to validate and maintain". And later on, in the paper, the authors say that "since there are currently only a few standards available, these ontologies are under constant development and there is no standardized way to install trust into them yet". And later they claim "There are very little standardized ontologies at the moment, which results in a dynamic further development of existing ones". Given the current state of the art of ontology engineering, I find it difficult to agree with these statements if no further references are provided. I cannot see where the difficulties in evolution and maintainance of ontologies lie, apart from usual difficulties in the evolution and maintainance of any other artefact. I cannot see either the causal relationship on number of standards and constant development, and the need for trust, or the fact that there are very little ontologies (if you take a look at some domains, you may see that this is not really a problem.
- As a result of some of these claims, the authors propose a way to solve those problems (which I cannot consider as real problems in the world of ontology engineering and governance) by using a blockchain approach, which is modified with the creation of a human-based consensus step. I would have find it interesting to have a clear discussion (and corresponding evaluation) of different types of human-based consensus and how they affect the creation and evolution of ontologies. I think that such an approach would be very valid for the audience of this journal. However, this is not the case. Only one approach for obtaining consensus is proposed, and no formal qualitative evaluation of how this approach performs wrt others is provided.
- As a final reflection on this: how is this approach not covered already by the development of ontologies that is being done in standardisation groups (e.g, W3C groups), where all decisions are being recorded in a source code repository (e.g., using git), where the different versions of the implementation code are being stored and can be traced, where the consensus is obtained in group calls, videoconferences and f2f meetings? How is this approach different to that, and how can it ensure that it provides more trust, as the authors claim?
- Finally, the evaluation is not clear at all on which research hypotheses are being evaluated (I would have expected evaluations on the quality of the ontologies generated, with a larger corpus of ontologies and not just with a single one whose implementation is not made available publicly, on the feelings of the participants in the ontology development process, on the amount of conflicts generated, on the most common types of agreements and disagreements, etc.)
- I can also mention that in terms of incentives, I agree that tokens can be provided, but this is not considered to be a problem in current ontology engineering practices. People who participate in ontology development have their name included in the author or contributors list, as a usual practice. Why do tokens solve this?

I think that these weaknesses make the paper unsuitable for publication in this journal.

On the second core contribution, the installation of human-based consensus in the process of blockchain, I like the idea and I think that it is novel (although I am not an expert here), but this core contribution would need further development and at the same time would be out of the scope of this journal.