Review Comment:
This paper presents a study with approaches for mapping data into RDF. I know the difficulty of performing such a study with users and how much time is required, however, in my opinion the current version of the paper cannot be accepted in Semantic Web Journal. In particular, the novelty and the contributions are not clear to me, and the paper seems to be more of a technical report. Moreover, the presentation needs several improvements in most of the parts of the paper and the structure should be reorganized. For the mentioned reasons (as it is explained in more detail below), my decision for the current version of this paper is “reject”.
Strong points
S1. The study can be of primary importance for the users of YARRRML and SPARQL Anything.
S2. A real user study that needs a lot of time for preparation and can be time consuming whereas user behaviors are described and several recommendations are given
S3. There is a Long-term Stable Link to Resources, which contains all the required details and files of the study.
Weak points
W1. The novelty and contributions are not clear
W2. The presentation and the writing of the paper needs to be improved.
W3. The paper is too dense, containing so many details and it seems to be more of a technical report than a full research paper.
W4. The participants are few, a better option could be at least the same participants to have done all the tasks for both systems.
W5. The structure of the paper needs to be revised, several pieces of information are repeated, and in many sections too many details are given.
Abstract and Introduction
First, please use present instead of past (e.g., the study uses, not the study used). Concerning the abstract, please also mention in the beginning why it is important to map such unstructured data to RDF. Therefore, please be clear in the abstract, describe (in brief) the problem, the motivation and the contributions.
Regarding the introduction do the same, i.e., provide more details why it is important to map such sources, and the motivation. There is no need to give so many details about the different techniques (YARRRML and SPARQL Anything) in the intro. Instead, you can add a subsection (e.g., in Section 2) having a background for each of these techniques.
Moreover, there are no research questions in this paper (only for future work), the contribution and the novelty (now, it is not clear), whereas in my opinion it is not acceptable to say “we believe it to represent the state-of-the-art”, i.e., if there is such a reference stating that, you can add it.
Related Work
Here, add a background for the techniques that you use in the study. Moreover, add in the end of the section a section for comparing your work with the related approaches, for highlighting the novelty of the presented work.
Section 3
This section needs to be fully revised. I appreciate how difficult and time consuming it is to perform such a study with real users, however, please use figures and tables for representing all the statistics about the users, in the current version it is quite difficult for the reader. Or at least use some bullets.
Section 4
Similarly to Section 3, again there is too much information in the text. One solution is to extend Table 2 by adding more information for each question, e.g., to include the objective and a link to the corresponding figure.
Sections 5 and 6.
Again the text is too dense, it is helpful that you have figures with the solutions, however, in my opinion the best way is to include some comments in some lines of the figures instead of writing so many details in the text, e.g., in Section 5, from “Both Mappings … changing the effect”, you can describe all these details by adding some comments in the right side of the code of Figure 9.
The same holds for the other figures
Sections 7 and 8
It is nice that you analyze the errors, but again the section should be revised to include only the required information, since most of the information included in those sections are also described in Tables 2 and 3. So in the next version of the paper, please provide a smaller version of the text and keep the Tables.
Section 9
The two different paradigms should have been compared earlier in this paper, e.g., in a background section (e.g., before related work).
Section 10.
In my opinion the same users should have done both parts of the study (for both YARRRML and SPARQL Anything). Moreover, 9 participants is a small number although the same repeated problems can occur. Table 5 should be moved to section 3.
Section 11.
Concerning the recommendations, in my opinion they can be helpful for the users that use the studied techniques. However, again the text is too dense, you can just provide a table showing the key recommendations and future developments.
Section 12.
The conclusion is quite big and repeats the same things from the previous sections. Please be more precise, just add 2-3 paragraphs with the most important parts of the paper and the final conclusions. Moreover, I do not agree with having the research questions for the future study in the conclusion section. One solution could be to have a subsection in Section 11, stating that by performing the study, we discovered more research questions for the future.
Other Minor issues
have tag item → have tag items
Uses of SPARQL Anything. →Users of SPARQL Anything
| 
