Explainable multi-hop dense question answering using knowledge bases and text

Tracking #: 3143-4357

Authors: 
Mohsen Kahani
Somayeh Asadifar
Saeedeh Shekarpour

Responsible editor: 
Guest Editors Ontologies in XAI

Submission type: 
Full Paper
Abstract: 
Much research has been conducted extracting a response from either text sources or a knowledge base (KB). The challenge be-comes more complicated when the goal is to answer a question with the help of both text and KB. In these hybrid systems, we address the following challenges: i) excessive growth of search space, ii) extraction of the answer from both KB and text, iii) extracting the path to reach to the answer. A heterogeneous graph is utilized to tackle the first challenge guided by question de-composition. The second challenge is met with the usage of the idea behind an existing text-based method, and its customization for graph development. Based on this method for multi-hop questions, an approach is proposed for the extraction of answer ex-planation to address the third challenge. Evaluation reveals that the proposed method has the ability to extract answers in an ac-ceptable time, while offering competitive accuracy and has created a trade-off between performance and accuracy in comparison with the base methods.
Full PDF Version: 
Tags: 
Reviewed

Decision/Status: 
Reject (Two Strikes)

Solicited Reviews:
Click to Expand/Collapse
Review #1
Anonymous submitted on 22/Jun/2022
Suggestion:
Reject
Review Comment:

The authors promise an explainable question answering system. The explanation aspect was supposed to be explained in section 5.1.3, which is nowhere to be found.
Moreover, my previous feedback was not addressed.

Review #2
By Carlos Badenes-Olmedo submitted on 23/Jun/2022
Suggestion:
Minor Revision
Review Comment:

Regarding the response letter, I think it is not completely aligned with the current version of the article. For example, the authors mention that "In evaluation section we added 'Links are established between words in Wikipedia sentences and DBpedia knowledge base entities using SMART and REL[1]'", however that text and references does not appear in the article. It is also stated that "This operation is called PE for short (See Figure 3)", but that figure does not contain that information, it seems to refer to Figure 5, but in that figure PI is mentioned, not PE. I would appreciate the authors to check the status of the article before writing the response letter.

Regarding the content, the paper now facilitates reproducibility since it publishes the source code of the methods. The description of their proposal has also been improved, mainly the expansion of the graph, and it is in line with the results of the evaluation. In this regard, it is necessary to specify, either in the title or in the abstract, that this method requires not only the natural language question, but also the entities previously identified in the knowledge base. Minor errors have also been corrected.

Review #3
By Ivan Donadello submitted on 04/Jul/2022
Suggestion:
Major Revision
Review Comment:

I really appreciated the authors effort in addressing the reviewers comments. However, now the paper is really verbose and descriptive on technical parts. Many parts are more tailored for a technical paper than for a scientific paper. Another concern is on the contribution of this paper. This has been answered in the answers to my previous questions:

"The theory section discusses the theoretical issues that led to the proposed method for solving the challenges. The first challenge is the inability of hybrid systems to answer questions whose answers do not exist in the knowledge base and must be extracted from the text. To solve this problem, an information retrieval based system was used. The next major challenge is the inability of the existing hybrid systems to explain, which was made possible by the probabilistic method. In addition, query decomposition is not performed in existing hybrid systems. The results showed that by performing this operation, which is based on the function of the human mind, the efficiency of the system increases."

This sentence clearly expresses one of the main paper contributions. I would write it in the paper. In addition, the second finding provides an useful insight on the contribution of this paper, it would be useful to read it summarized at the beginning of the paper. I suggest to make the paper less technical by selecting in Section 5 only the scientific important points and leaving the technicalities in a separate section/appendix. In addition, I would pair Fig. 2 with a running example. Such a running example could be use even in Section 5 to better explain the single modules of the system. In addition, a single running example going along the whole paper would better help the reader than many single examples.

Minors:
- page 3, the sentence "Efforts have been made to" is repeated;
- Section 5, "The motivation for using these 3 axes is given in Section 7.3" I would summarize such a motivation here.
- Section 5, last paragraph, triples and facts are used interchangeably.
- Section 5, last paragraph, the sentence "In addition, ... a single sentence" is not clear.
- Section 5.1, second sentence, correct in "This architecture is shown in Figure3. Figure 4 illustrates ..."