CAMS-KG: a Classical Arabic Morpho-Semantic Knowledge Graph

Tracking #: 2063-3276

Ibrahim Bounhas
Nadia Soudani
Yahya Slimani

Responsible editor: 
Guest Editors Knowledge Graphs 2018

Submission type: 
Full Paper
In this paper we propose to build a morpho-semantic knowledge graph from Arabic vocalized corpora. Our work focuses on classical Arabic as it has not been deeply investigated in related work. We use a tool suite which allows analyzing and disambiguating Arabic texts, taking into account short diacritics to reduce ambiguities. At the morphological level, we combine Ghwanmeh stemmer and MADAMIRA which are adapted to extract a multi-level lexicon from Arabic vocalized corpora. At the semantic level, we infer semantic dependencies between tokens by exploiting contextual knowledge extracted by a concordancer. Both morphological and semantic links are represented through compressed graphs, which are accessed through lazy methods. These graphs are mined using measure BM25 to compute one-to-many similarity. Indeed, we propose to evaluate CAMS-KG in the context of Arabic Information Retrieval (IR). Several scenarios of document indexing and query expansion are assessed. That is, we vary indexing units for Arabic IR based on different levels of morphological knowledge, a challenging issue which is not yet resolved in related work. We also experiment several combinations of morpho-semantic query expansion. This permits to validate our resource and to study its impact on IR based on state-of-the art evaluation metrics.
Full PDF Version: 

Reject (Two Strikes)

Solicited Reviews:
Click to Expand/Collapse
Review #1
By Milan Dojchinovski2 submitted on 15/Feb/2019
Review Comment:

First of all, I would like to thank the authors for considering the comments and implementing them in the paper. The authors have improved the paper since the last version, however, there are still three crucial problems with this submission:

From my previous review: 1) The work is not well positioned - the related work is scattered across the paper and thus it is difficult to clearly identify the problems with the existing solutions and the contributions of the author's work

The authors reorganized the paper and made changes, however still the work is scattered around the paper and not consolidated. This adds a lot of difficulties to understand what are the problems and how you solve them. The authors also make a statement on page 2:
“Nevertheless, our literature review (cf. Sections 2.3, 3.1 and 4.2.1)...” which also shows that the related work is all around the paper. It is fine to comment on the related work in different locations in a paper, but there should be a single location where you review the related work and identify gaps.

From my previous review: 2) The paper is difficult to read and follow - the motivation, the data source, the generation process, the data model and the evaluation are not clearly presented. More importantly, due to the lack of simple examples, it is difficult to understand and interpret particular parts from the paper.

The authors added an example (Figure 2) which illustrates the KG creation process. However, the figure is not discussed and explained. For example, what do you do for pre-processing step?
Also, the data model is still unclear. I would expect the model would be described in Section 3 (KG spec), while it seems it is described in Section 2 (Arabic morphology). It is still difficult to read and understand easily the contents of the paper.

From my previous review: 3) the evaluation is not satisfactory - the authors primarily focused on experiments in order to evaluate the benefit from the generated knowledge graph. However, they do not evaluate the quality of the created knowledge.

The authors added more info on the qualitative evaluation (Section 5.4) but it is very unclear what and how has been evaluated. I expected an evaluation of the quality of the created KG. For example, how much of the extracted knowledge is correctly extracted?

Some additional remarks:

- The abstract still does not clearly state the contributions of the paper.

- The authors included references to some relevant work (section 4.2) however the authors do not discuss the relation of these works (e.g. MMoOn/Ontolex-lemon) to their work.

- The paper lacks info on the statistics about the knowledge graph, license, download link, query interface, etc.

Minor remarks:

Page 2

- We are trying to contribute
-> Are you “trying” or you managed to “do it” successfully.

- we propose to build new resources
-> We build/created,... you do not propose

Fig 1. The title of the figure is: “Fig 1. CAMS-KG construction and mining process.” but the figure illustrates something else.

Overall, the authors have put significant effort in implementing and creation of the knowledge graph. In other words, I find the technical part very strong (although direct access to the dataset has not been provided). However, the quality of the paper is not sufficient to justify publication. The three key problems are 1) positioning/motivation/related work, 2) clarity/readability, and 3) qualitative evaluation.
Considering my comments above I do not recommend acceptance of the paper.

I sure hope the authors will not be put off by the above remarks, and instead, to see the above-detailed points as a good solid path to follow, which should guarantee a high-quality publication.

Review #2
Anonymous submitted on 19/Feb/2019
Major Revision
Review Comment:

This manuscript was submitted as 'full paper' and should be reviewed along the usual dimensions for research contributions which include (1) originality, (2) significance of the results, and (3) quality of writing.

-------- overall review (scale: Excellent, Very good, Good, Fair, Weak)
originality: Very good
significance of the results: Good
quality of writing: Good

This paper tackles an interesting problem (semantic interpretation of Arabic corpus), with focus on improving Information Retrieval by Query Expansion. This problem is challenging, given the complex morphology of Arabic language. CAMS-KG tries to combine morphological and semantic features for a superior QE method. The paper could be written more fluently, and the results could be analyzed better. There are some ambiguities in terms of design choices and evaluation results which I have explained in detail later.

-------- detailed review
summary of the paper:
This paper introduces a morpho-semantic knowledge graph from vocalized Classical Arabic corpus. The authors go over the morphology of Arabic language in detail, and go over the principal processes that form the Arabic lexicon: Derivation, Inflection, and word construction. This analysis reveals some of the challenges of semantic interpretation of Arabic language. The authors then discuss state of the art resources for morphological and semantic analysis of Arabic language. CAMS-KG combines existing tools for morphological analysis (Ghwanmeh) and disambiguation (MADAMIRA), and implement a concordance builder tool, and KG representation for morpho-semantic features.
Given an arabic corpus, it extracts the morpho-semantic features and stores them in a KG (nodes: (root, verbed pattern, lemma, stem, vocalized word, non-vocalized word, links: morphological, e.g. the flectional relation between a lemma and a stem) which then is used for NLP or IR tasks (morpho-semantic query expansion). BM25 ranking is used for retrieving related documents for given query.
The authors evaluate CAMS-KG on two large datasets (Tashkeela, and ZAD), and compare their morpho-semantic QE with several baselines (use of different morphemes, nodes and links in the KG, and with different indexing methods), and other morphological/semantic QE methods, on 25 queries from ZAD dataset.

1 - The paper tackles a challenging problem of combining morphological and semantic features for Arabic lexicon, in order to improve information retrieval from Arabic corpus.
2 - The authors review previous methods in details, and explain the challenges with examples which is insightful for the reader.
3 - The experiments contain many different scenarios and the IR metrics are presented for different baseline methods, as well as state of the art systems.

Major Comments:

The revisions the authors made in this version of the paper covers some issues in the previous version, specially adding sections 2.3 and 2.4, adding Fig 1, and changes to sections 3.1.3 and 4.2.2. However I believe the write up of the paper should be revised. Some of the major necessary changes I would suggest is as follows:

1- Sections 2.3 and 2.4 are very informative to the reader, but is written in a verbose manner and is very hard to follow. It is hard to keep track of the names of the tools, and what they do, and it is confusing for the reader. There are also contradictory statements, for example in page 1 paragraph 2 it states "Soudani et al. (2016) [78] concluded that MADAMIRA outperforms all the other NLP tools essentially the stemmers of Khoja and Garside (1999) [90], Ghwanmeh et al. (2009) [83], Fraser et al., (2002) [8] and Darwish et al. (2009) [47] stemmers." and in paragraph 3 it states "From the previous cited work of Ben Guirat et al. [85], we glean that Ghwanmeh stemmer achieves the best results compared to other light or root-based stemmers.". The authors should add some insights on why the contradictory conclusions are reported and make a conclusive argument. The current write up is only reporting previous work and leaves the reader confused. I suggest that the authors summarize the capabilities in a table similar to Table 2.

2- The statement "Their results uncover that the use of a morphological disambiguation tool (i.e. MADAMIRA) and lemma-based indexing enhances remarkably and in an optimal way the IR system performance compared to other tools." is made very late in section 2.4. Since the focus of this paper is improving IR systems by proposing CAMS-KG, this should be argued early on. using Ghwanmeh instead of Sebawai is not motivated.

3- Section 5.3.1 is not conclusive, the argument "Nevertheless, the fact that the former approach performs better than the latter is explained by the fact that it retrieves more words from CAMS-KG." needs to be explained. The result in table 6 suggests that morphological-expansion degrades both precision and recall, which is strange (0.74 without expansion vs. 0.694 with expansion), this needs to be explained.

4- I suggest that the contributions of the paper be clearly summarized at the end of section 1.

Minor Comments:

1- For Fig. 5, somewhere in the text or caption it should be stated that the number of relevant documents for Q23 is 0.

2- the order of systems in fig. 4 and fig.5 is different and confusing.

Overall, I think the write up of the paper needs revision in order to clearly underline the contributions and experimental results takeaways.

Review #3
Anonymous submitted on 21/Feb/2019
Minor Revision
Review Comment:

This manuscript was submitted as 'full paper' and should be reviewed along the usual dimensions for research contributions which include (1) originality, (2) significance of the results, and (3) quality of writing.