Review Comment:
I would like to thank the authors for their effort to improve the paper. The writing has improved a lot in the revised version, although there are still many grammar errors and typos that need to be fixed.
The strong points of the previous version are still valid, so I am going to focus this time on the weaker points that still exist and need to be addressed, in my opinion.
Before that, I would like to add to the previous strong points the excellent reproducibility and commend the authors for their efforts on that.
More important points:
- Section 2 - matching targets: you introduced matching targets, as per another reviewer's request, but they are not really well-defined. Moreover, you mention them next in the definition of the annotation tasks, as if they are given as input, whereas it's a part of the tasks to identify them. This also makes the task definitions unclear. I am fine with keeping the notion of matching targets, but you should be extra careful on how you define and use them.
- Section 2 - Assumption 1: "The system could return incorrect answers if table elements are not available in the knowledge graph." -> what do you mean "could"? 1. That it only returns errors when the table elements are not in the KG? 2. That it could also return a correct result (impossible)? 3. That it could also return no annotation result, which would be the ideal scenario, but it sometimes mistakenly returns something? 4. That it always returns something? 5. None of the above/something else? Please clarify.
- Section 2 - Assumption 3: Please elaborate a bit on what exactly "independent" means (e.g., not part of a relational database, no joins?).
- Section 3: In the revised version, you changed "learnable weights" to always-equal weights (w = 1). I don't see a point in keeping the weights in all the equations, when they are always equal to 1.
- Section 4: I would move Section 4 to an Appendix, as it breaks the flow and looks more like a section from a demo paper.
- Section 5: I don't see why Section 5 is a separate section and not just a part of the intro (e.g., placed just after the contributions bullets). In fact, many parts are also repeated in the intro.
- Error analysis: In my previous review, I pointed out that the "Error Analysis" section was focusing on the benchmark errors and not the MTab errors, which would have been more interesting. The authors removed the error analysis section in the revised version and created a new section about improving the benchmark, which is good. However, I am still missing an actual error analysis of MTab. Your results show that F1 is not always 1 (which is natural, it's not a weak point). It is interesting to check those cases that MTab does not get right and, if possible, explain why this happens, using examples or patterns of errors (again, using the cases where the original or revised benchmark has the correct ground truth annotation).
- The writing has improved, but there are still many errors. Some are, indicatively, listed next.
Minor points:
- Abstract does not end with a period ('.').
- page 1, line 45, left column: "knowledge graphs (KB)" -> "(KG)"
- page 1, lines 34-35, right column: "an semantic annotation system for tabular data designed" -> "a semantic annotation system for tabular data, designed"
- page 2, lines 25-27: "Wang et al., (...) complexity. [5]." -> "Wang et al. [5], (...) complexity."
- page 3, line 1 (and elsewhere): You are still comparing to your arXiv paper [7], not the SemTab paper. My previous comment was not to pick one or the other, but cite BOTH and compare to both together (if they are identical), or to each of them separately (if they are different).
- page 3, lines 40-51, left column: I would use the phrase "We denote (something) as (symbol)", instead of "(something) is (symbol)". I would also say that " $\in Triples$, consist of a subject (...), a predicate (...), and an object (...)". Finally, your subscripts and their definition is a bit confusing. For i and j you use = 1...N and 1...M, respectively, but for j_1 and j_2, you use [1,M]. I would prefer the latter notation, and perhaps use j and j' (resp. c_j and c_j') instead of j_1 and j_2, as j is already a number.
- page 3, right column: You should be extra careful with the problem definitions and assumptions. For example, the CEA task is not to annotate THE table cell with an entity, but many (as many as possible) table cells.
- page 3, line 49, right column: "representing" -> "represent"
- page 4, Assumption 4: "and the entities related to cell values are of the same type" -> "and the entities related to cell values of the same column are of the same type"
- page 4 (and elsewhere): when referring to specific steps, capitalize 'S', e.g., ("Step 1")
- page 4, line 36, left column: "We perform the four processes" -> "We perform the four following processes"
- page 4, line 48, left column: extra white space between "(NE)" and "."
- page 4, line 51, left column: "a entity-name" -> "an entity-name"
- page 4, lines 1-4, right column: why are objects, wars and books not associated with a SpaCy tag?
- page 5, Section 3.3: Please explain what type of index you are using? It looks like an elasticsearch index.
- page 7, Equation 10: missing ')'
- page 9, line 26, left column: delete this line "where t_e (...)", as t_e is already introduced
- page 9, Section 4.1, line 21: change to "We provide the five following APIs:"
- page 9, Section 4.1: replace "The use could (use this API to)" with something like "This API does that"
- page 9, line 30: "responded object include" -> either "objects" or "includes"
- page 10, lines 29-30, right column: "statistic" -> "statistics"
- page 13, lines 32, 33, right column: please cite here as well the ESWC 2020 paper ("SemTab 2019") that introduced the metrics AH and AP.
- page 14, lines 9-10, left column: "We compare MTab4D with the others reported results from SemTab 2019" -> "We compare MTab4D with other systems, using the results reported in SemTab 2019"
- page 15: Section 7.1 start on the right column, while the left column is blank
- page 18, Assumption 2: Please elaborate a bit more on what "classifying table types before matching" means
|
