Review Comment:
Overall evaluation
Select your choice from the options below and write its number below.
== 3 strong accept
== 2 accept
== 1 weak accept
== 0 borderline paper
== -1 weak reject
== -2 reject
== -3 strong reject
-2
Reviewer's confidence
Select your choice from the options below and write its number below.
== 5 (expert)
== 4 (high)
== 3 (medium)
== 2 (low)
== 1 (none)
4
Interest to the Knowledge Engineering and Knowledge Management Community
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor
2
Novelty
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor
2
Technical quality
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor
2
Evaluation
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 not present
2
Clarity and presentation
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor
2
Review
Please provide your textual review here.
The paper/manuscript describes the development of a `recommender component' for an authoring environment for authors of accompanying texts for (visual) museum objects. The component collects concepts (mainly/only?) from Freebase which are related to an object of visual art with the aim to provide material - tags representing concepts - and some structure to create a relevant information context, comprising stories (and other information e.g. date of birth and (eventually) death of the author). A simple model of narrative texts consisting of a small number of meta-concepts which take roles in narrative texts. The notion of theme is the major organiser. A theme is derived by selecting the most covering and frequent concepts from the `concept space', i.e. the set of `domain' concepts that are associated with the object as retrieved from Freebase. A story describes events and events occur at some place/time, which is the setting of the event, allowing for time-lines of events. The model distinguishes 6 properties: time & place (called here by the authors: location, start/end time) and agent, activity and tag. The authors report a `lightweight' evaluation, comparing performance in answering 3 questions about two painters. The effects of the recommender are compared with direct Freebase access and with two biographical texts as sources for information.
The work reported is premature for publication: not only for a journal article, but also for a conference (EKAW) publication/presentation. For the following reasons:
Evaluation:
Starting with the last issue: the evaluation is too light to be called evaluation. In each condition only 2 subjects participated, so that leaves one degree of freedom for any (statistical) comparison over three (dependent) test-questions. That makes this experience rather a try-out for a pilot study. The design is too simple to allow for much control (e.g. capturing variance due to individual differences; repeated measurements, etc.).
System design and functionality:
The design/architecture of the recommender component is not described, and from the text I get the impression that it is rather flat: a simple user interface to the underlying information management system (Drupal). The authors describe Navigation paths (figure 8), which can be viewed as dependencies that have to be respected in the use and specification of the various recommender (meta-) concepts (settings, themes, etc.). These may be hardwired in some architecture, but thus far I have read this as a kind of map for the users. Figure 8 presents a diagram of the Navigation paths. According to this diagram the author starts with a story and collects `concepts' (i.e. create a `concept space'). As there are no arrows back to `story' I assume this is the `reader' mode of the authoring environment. Anyway, the manuscript is utterly vague about the functionalities of the `authoring environment': it is unlikely that for both roles (author, reader) the same functions are available, or is the `recommender component' no more than a collector of associated terms? What does the recommender recommend and to whom? In other words, the user requirements are not explicit therefore it is difficult what functionalities to expect, nor are functionalities differentiated for the types of users.
Theoretical grounding:
The model as described is not explicitly structured. The verbal descriptions of its meta-concepts (event, setting, theme, etc.) suggest a theoretical foundation but lack coherence as there is for instance no way to decide whether the event attributes of Location and Begin/endTime are not the same as those for the notion of Setting. In other words, any event has a setting, but some settings are more settings than other settings. Indeed stories have levels of granularity, and the scope of an event may in fact be a theme (e.g. the First World War is an event that is the theme of millions of books, movies). Since Tomashevsky the narrative literature has had an explosive development, starting at the 70-ies by text-linguistics in which text-structures for narrative and expository discourse have been analysed, and a richer, but not undisputed vocabulary to describe text structures has been developed. For instance, themes are not simple concepts but rather propositions, expressing not only a topic, but a particular view (e.g. qualification) or intention of the author on that topic: a message. Aside from the fact that there is a huge literature on the notion of `event' (philosophy, linguistics, AI, in particular as in top ontologies), events do not have properties but roles which are governed by actions (`verbs' e.g. in case grammar approaches). That it concerns roles instead of properties can be derived also from the statement at p7 "For example, an artist may be the creator of certain artworks and the subject of others." Agents may take the role of actor but also of recipient, beneficiary, etc. There is a real proliferation of thematic roles available in the (psycho-)linguistic literature. In other words, there is a lack of grounding in the literature that makes the proposed meta-concepts arbitrary and not well defined. For instance, the notion of theme has an operational definition (global, frequently occurring in text) but that does by no means cover its meaning. For instance, the theme may even be implicit as it can be inferred by the reader, For instance, journalists are taught that it pays to have the reader draw its conclusions from an article instead of `paternalistically' give the moral away explicitly.
Nature of contextual stories:
If stories play an important role in informing viewers of objects of art on the context of the work, it stands reason that one should indicate what kind of stories make up such contexts. It seems reasonable that these narratives are not the coherent narratives with a plot, etc. of stand alone stories. Their functions are not described in the paper (illustrating concepts related to the object? explaining why things are as they are?…) It is not clear whether these stories should be supportive for digesting information or play a role by itself.
Two minor textual remarks:
p7 "The agent, location and tag properties are equivalent to the involvedAgent, involvedObject and atPlace properties of the LODE ontology."
Probably you mean:
"The agent, location and tag properties are equivalent to respectively the involvedAgent, atPlace and involvedObject properties of the LODE ontology."
p2 "The CIDOC CRM ontology [9], facilities an event-based approach to the representation of heritage and cultural knowledge."
-> ""The CIDOC CRM ontology [9], facilitates an event-based approach to the representation of heritage and cultural knowledge."
|