Review Comment:
The paper presents an approach and a tool for visual exploration of linked data, which provides visual representations of resource sets that help gain insights about those resources.
The authors gave a good overview of related works pointing that the other approaches do not make it possible to navigate linked data directly while benefitting from aggregation techniques, sub-selections etc.
Pros:
+ important topic, fitting well in the scope of the Semantic Web Journal
+ good narrative
+ good idea
+ set-oriented exploration, such as identifying correlations, observing distributions, comparing & contrasting groups of resources
+ builds nicely on the experiences and results of previous works in the area
+ minimal effort from users before they can start browsing
+ S-Paths is distributed as an open-source project (requires registering/signing up to the git service of INRIA)
Cons:
- some too much generic descriptions - lack of precise definitions, algorithms etc.
- some information is given only by means of examples and not by an exhaustive list
- the evaluation setup was not always clearly communicated to users
Further comments:
***Abstract:***
"amenable to visual representation" -> is this measurable?
"set-based navigation"-> it is not precisely defined in the paper what it is
***1. Introduction***
"Most linked data browsers employ a follow-yournose strategy" -> reference needed
"Properties that provide relevant descriptions of resources are not necessarily direct properties of those resources." -> any (quantitative) evidence? Examples?
"They can be several hops away in the RDF graph, depending on how abstract the dataset’s model is and on what ontologies it employs"-> can this be an artefact of serializing OWL to RDF?
Fig.1 contains very small images, hardly readable.
***3. S-Paths***
"S-Paths is designed to support users in the exploration of linked datasets"-> this sentence would benefit from the stating what S-Paths is (even if previously mentioned)
"semantic path" -> It is unclear whether a "semantic path" is being introduced by this paper, or if it has been introduced previously. In the former case, it would be better to provide a phrase like "in this paper we introduce semantic paths". In the latter case, there should be a citation or a section with preliminaries to delineate what is the contribution of the current paper with respect to previous work.
"A semantic path is a set of resources related to a set of values by a sequence of RDF statements." ->this definition is a bit vague, as it does not say in a detailed way about the relation, e.g. are RDF statements arbitrary? It would benefit from formalization
"S-Paths provides a collection of such templates"->where they can be found or a list of them? It would be useful to point to such list in this place of the manuscript or inform that it will be described later in the paper in Section X
"Considering paths that can be indirect, and not only first level properties, mechanically results in aggregation steps to the set of results."-> what does that mean that it *mechanically* results?
"The full analysis is performed only when S-Paths gets set up with a new set of graphs." ->what is a full analysis? The analysis with respect to all the characteristics?
"S-Paths provides a set of views: map, image gallery, timeline, statistical charts, simple node-link diagrams, etc." ->I recommend to remove "etc." and provide a full set of the views or a reference to the full set (e.g. included in a table).
"Once semantic paths for a given resource set have been characterized,"->it is unclear exactly how they are characterized, there haven't been any algorithm presented before this point of the paper, only a generic description
"These are used in multiple places in the interface, e.g., next to the resource selection menu, in the view configuration menu, in the axes’ legends, and whenever a semantic path is displayed."->again, I would prefer more exact description than only kind of an example (with use of "e.g.")
"They serve as entry points into the data, constituting what we consider a priori to be reasonably-coherent groups of entities."-> what is that the authors consider "reasonably-coherent groups of entities". This should be precised/formalized.
Fig.6 and Fig.7 are much too small in my opinion.
http://s-paths.net is down.
***4. Illustrative Scenario****
The images at Figure 8 are too small, non-readable.
***5. Evaluation***
In general, when users do exploratory search it is often a pre-requisite to solving some task.
The users were given tasks indicatively. At the same time the users were asked to explore the dataset in an open-ended fashion. There is perhaps some inconsistency or lack of explicit instructions on the goals with regard to the guidelines the users were given. Therefore I think that the quantitative results presented in "5.3. Task Success and Task Time" are not valid or reliable.
"They also had to tell whether they would have been able to answer those questions before the experiment."-> I am not sure this is the right setup.
I would expect such setup in which one group of the users is divided into two groups where one is answering the questions before and then the subgroups change the roles solving another, but similar configuration of the task, i.e. actually evaluating how well the task is solved with and without the proposed system and not asking how the users think they would solve the task. The baseline might be raw data?
***6. Limitations*
I appreciate that the authors provided the limitations section.
6.2. Data Processing: The authors claim they have tested S-Paths on several datasets, but it is not described what was the setup of the testing, and what was tested exactly.
Overall, I am rather positive about the paper, regarding its narrative, motivations, topic and contribution, which is very in line with the scope of the Semantic Web Journal.
However, I would expect more precise definitions (clear, even if not overly formal) and more rigorous style of writing (at places I have indicated in comments) and fixing a broken link to the demo.
Another major remark is that the paper might perhaphs be submitted to a category of "Reports on tools and systems"?
|