Using event spaces, setting and theme to assist the interpretation and development of museum stories

Tracking #: 759-1969

Authors: 
Paul Mullholland
Annika Wolff
Eoin Kilfeather
Evin McCarthy

Responsible editor: 
Guest Editors EKAW 2014 Schlobach Janowicz

Submission type: 
Conference Style
Abstract: 
Stories are used to provide a context for museum objects, for example linking those objects to what they depict or the historical context in which they were created. Many explicit and implicit relationships exist between the people, places and things mentioned in a story and the museum objects with which they are associated. We describe a simple interface for authoring stories about museum objects in which textual stories can be associated with semantic annotations and media elements. A recommender component provides additional context as to how the story annotations are related directly or via other concepts not mentioned in the story. The approach involves generating a concept space for different types of story annotation such as artists, museum objects and locations. The concept space is predominantly made up of a set of events, forming an event space. The concept spaces of all story annotations can then be combined into a single view. The events of a concept space can be visualized by the story reader or author. Narrative notions of setting and theme are used to reason over the concept space, identifying key concepts and time-location pairs, and their relationship to the rest of the story. Story setting and theme can then be used by the reader or author to assist in interpretation or further evolution of the story.
Full PDF Version: 
Tags: 
Reviewed

Decision/Status: 
[EKAW] conference only accept

Solicited Reviews:
Click to Expand/Collapse
Review #1
Anonymous submitted on 25/Aug/2014
Suggestion:
[EKAW] conference only accept
Review Comment:

Overall evaluation
Select your choice from the options below and write its number below.

== 3 strong accept
== 2 accept
== 1 weak accept
== 0 borderline paper
== -1 weak reject
== -2 reject
== -3 strong reject

1

Reviewer's confidence
Select your choice from the options below and write its number below.

== 5 (expert)
== 4 (high)
== 3 (medium)
== 2 (low)
== 1 (none)

4

Interest to the Knowledge Engineering and Knowledge Management Community
Select your choice from the options below and write its number below.

== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor

4

Novelty
Select your choice from the options below and write its number below.

== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor

4

Technical quality
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor

3

Evaluation
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 not present

2

Clarity and presentation
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor

4

Review

This paper presents an interface to generate a concept space for museum stories that can be interpreted through the narrative notions of theme and setting. I found overall idea of the interface compelling and a potentially useful tool to enrich museum experiences through application of existing knowledge graphs. I recommend some changes, however, as described in the following.

In the paper the authors only discuss using Freebase for the core knowledge graph used by the recommender component. One caveat is that although Freebase is great a resource, it is also a general-purpose knowledge base. Thus, it will not be a very good source for specialized domain knowledge, which is something that might be very useful in understanding museum stories. Some discussion of the choice of Freebase as well as other potential knowledge sources is merited.

The workflow described in the paper is quite extensive and although it is described well in the text, it would be really helpful to include figures that schematically show how a concept space is generated, and how themes and settings are identified.

My main reservation with the paper in its current form is that the evaluation seems somewhat anecodotal. The number of participants is really too low to make any statistical judgment, and the questions quite narrow and perhaps chosen based on the annotated pages for the two artists. A much broader selection of information sources, with questions developed by experts on those sources might be a better indicator. Furthermore, the evaluation can be broken down in two ways: 1) looking at the usability of the recommender component, and 2) looking at the interpretability of the annotated story results.

In section 4, the notions of coverage and frequency to the determine themes from a concept space seems reasonable. I would like to see a somewhat more formal explication of the scoring mechanism. In fact, the paper as a whole would benefit greatly from some more formality at each step, using consistent symbols and set notation for ideas such as concept, concept space, theme, story, story annotation, setting, and so on.

Minor comment:
pg. 2 "Frist World War"

Review #2
Anonymous submitted on 27/Aug/2014
Suggestion:
[EKAW] reject
Review Comment:

Overall evaluation
Select your choice from the options below and write its number below.

== 3 strong accept
== 2 accept
== 1 weak accept
== 0 borderline paper
== -1 weak reject
== -2 reject
== -3 strong reject

-2

Reviewer's confidence
Select your choice from the options below and write its number below.

== 5 (expert)
== 4 (high)
== 3 (medium)
== 2 (low)
== 1 (none)
4

Interest to the Knowledge Engineering and Knowledge Management Community
Select your choice from the options below and write its number below.

== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor

2

Novelty
Select your choice from the options below and write its number below.

== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor
2

Technical quality
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor
2

Evaluation
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 not present

2

Clarity and presentation
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor
2

Review
Please provide your textual review here.

The paper/manuscript describes the development of a `recommender component' for an authoring environment for authors of accompanying texts for (visual) museum objects. The component collects concepts (mainly/only?) from Freebase which are related to an object of visual art with the aim to provide material - tags representing concepts - and some structure to create a relevant information context, comprising stories (and other information e.g. date of birth and (eventually) death of the author). A simple model of narrative texts consisting of a small number of meta-concepts which take roles in narrative texts. The notion of theme is the major organiser. A theme is derived by selecting the most covering and frequent concepts from the `concept space', i.e. the set of `domain' concepts that are associated with the object as retrieved from Freebase. A story describes events and events occur at some place/time, which is the setting of the event, allowing for time-lines of events. The model distinguishes 6 properties: time & place (called here by the authors: location, start/end time) and agent, activity and tag. The authors report a `lightweight' evaluation, comparing performance in answering 3 questions about two painters. The effects of the recommender are compared with direct Freebase access and with two biographical texts as sources for information.

The work reported is premature for publication: not only for a journal article, but also for a conference (EKAW) publication/presentation. For the following reasons:

Evaluation:
Starting with the last issue: the evaluation is too light to be called evaluation. In each condition only 2 subjects participated, so that leaves one degree of freedom for any (statistical) comparison over three (dependent) test-questions. That makes this experience rather a try-out for a pilot study. The design is too simple to allow for much control (e.g. capturing variance due to individual differences; repeated measurements, etc.).

System design and functionality:
The design/architecture of the recommender component is not described, and from the text I get the impression that it is rather flat: a simple user interface to the underlying information management system (Drupal). The authors describe Navigation paths (figure 8), which can be viewed as dependencies that have to be respected in the use and specification of the various recommender (meta-) concepts (settings, themes, etc.). These may be hardwired in some architecture, but thus far I have read this as a kind of map for the users. Figure 8 presents a diagram of the Navigation paths. According to this diagram the author starts with a story and collects `concepts' (i.e. create a `concept space'). As there are no arrows back to `story' I assume this is the `reader' mode of the authoring environment. Anyway, the manuscript is utterly vague about the functionalities of the `authoring environment': it is unlikely that for both roles (author, reader) the same functions are available, or is the `recommender component' no more than a collector of associated terms? What does the recommender recommend and to whom? In other words, the user requirements are not explicit therefore it is difficult what functionalities to expect, nor are functionalities differentiated for the types of users.

Theoretical grounding:
The model as described is not explicitly structured. The verbal descriptions of its meta-concepts (event, setting, theme, etc.) suggest a theoretical foundation but lack coherence as there is for instance no way to decide whether the event attributes of Location and Begin/endTime are not the same as those for the notion of Setting. In other words, any event has a setting, but some settings are more settings than other settings. Indeed stories have levels of granularity, and the scope of an event may in fact be a theme (e.g. the First World War is an event that is the theme of millions of books, movies). Since Tomashevsky the narrative literature has had an explosive development, starting at the 70-ies by text-linguistics in which text-structures for narrative and expository discourse have been analysed, and a richer, but not undisputed vocabulary to describe text structures has been developed. For instance, themes are not simple concepts but rather propositions, expressing not only a topic, but a particular view (e.g. qualification) or intention of the author on that topic: a message. Aside from the fact that there is a huge literature on the notion of `event' (philosophy, linguistics, AI, in particular as in top ontologies), events do not have properties but roles which are governed by actions (`verbs' e.g. in case grammar approaches). That it concerns roles instead of properties can be derived also from the statement at p7 "For example, an artist may be the creator of certain artworks and the subject of others." Agents may take the role of actor but also of recipient, beneficiary, etc. There is a real proliferation of thematic roles available in the (psycho-)linguistic literature. In other words, there is a lack of grounding in the literature that makes the proposed meta-concepts arbitrary and not well defined. For instance, the notion of theme has an operational definition (global, frequently occurring in text) but that does by no means cover its meaning. For instance, the theme may even be implicit as it can be inferred by the reader, For instance, journalists are taught that it pays to have the reader draw its conclusions from an article instead of `paternalistically' give the moral away explicitly.

Nature of contextual stories:
If stories play an important role in informing viewers of objects of art on the context of the work, it stands reason that one should indicate what kind of stories make up such contexts. It seems reasonable that these narratives are not the coherent narratives with a plot, etc. of stand alone stories. Their functions are not described in the paper (illustrating concepts related to the object? explaining why things are as they are?…) It is not clear whether these stories should be supportive for digesting information or play a role by itself.

Two minor textual remarks:
p7 "The agent, location and tag properties are equivalent to the involvedAgent, involvedObject and atPlace properties of the LODE ontology."
Probably you mean:
"The agent, location and tag properties are equivalent to respectively the involvedAgent, atPlace and involvedObject properties of the LODE ontology."

p2 "The CIDOC CRM ontology [9], facilities an event-based approach to the representation of heritage and cultural knowledge."
-> ""The CIDOC CRM ontology [9], facilitates an event-based approach to the representation of heritage and cultural knowledge."

Review #3
Anonymous submitted on 01/Sep/2014
Suggestion:
[EKAW] combined track accept
Review Comment:

Overall evaluation
Select your choice from the options below and write its number below.

== 3 strong accept
== 2 accept
== 1 weak accept
== 0 borderline paper
== -1 weak reject
== -2 reject
== -3 strong reject

1

Reviewer's confidence
Select your choice from the options below and write its number below.

== 5 (expert)
== 4 (high)
== 3 (medium)
== 2 (low)
== 1 (none)

4

Interest to the Knowledge Engineering and Knowledge Management Community
Select your choice from the options below and write its number below.

== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor

4

Novelty
Select your choice from the options below and write its number below.

== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor

3

Technical quality
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor

2

Evaluation
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 not present

3

Clarity and presentation
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor

5

Review

I enjoyed reading this paper as it is clear, stimulating, and straightforward to its scope.
I'd like seeing it presented at EKAW (with the issues reported below), and probably it might be a good contribution to SWJ, in case the authors would take into account some significant improvements needed (also reported below).
On the positive, I like the way some narratological notions have been smoothly transferred in an application based on a minimal semantics such as Freebase. I also like the way of approaching KR problems, empirical, social and considering cognitive issues.
On the negative, the plain talk of the paper seems to hide any knowledge representation/engineering detail. The model of museum stories described in the paper looks basically informal, and Freebase data are reused just as topics emerging from annotations, and populating a "concept space", from which "themes" are extracted to build a "context" for a museum story. However, no clear semantics is given to Freebase data, topics, concept spaces, themes, and contexts. They are "managed" as fragments of annotations for the sake of a nice application. In other words, as it is described this is a knowledge management application, with only some flavor of semantics and knowledge engineering.
For the EKAW version, I would like to see a more explicit position on the formal aspects (if any) of the model, and how the "reasoning" claimed by the authors can actually qualify as such. Moreover, I need more quantitative data from the observational study.
For a possible SWJ version, I definitely need the previous, plus an extension that provides links to semantic web and its best practices. For example: how the knowledge collected by reusing and reasoning over Freebase can be reintroduced as novel SW data? How the narratological notions of event, setting, and theme are related to the literature on event representation and extraction, topics vs. concepts, etc.?