Quantifying the Salience of Musical Characteristics From Unstructured Text

Tracking #: 730-1940

Authors: 
Gopala Koduri
Xavier Serra

Responsible editor: 
Guest Editors EKAW 2014 Schlobach Janowicz

Submission type: 
Conference Style
Abstract: 
Music is a discerning window to the rich diversity of the world. We hypothesize that identifying the differences between music from different cultures will lead to richer information models representative of them. Using five music styles, this paper presents a novel approach to bring out the saliences of a given music by rank-ordering its characteristics by relevance using a natural language text corpus. The results agree with the cultural reality reflecting the diverse nature of the music styles. Further, to gather insights into the usefulness of this knowledge, an extrinsic comparative evaluation is performed. Similarities between entities in each music style are computed based on a salience-aware semantic distance proposed using the knowledge acquired. These are compared with the similarities computed using an existing linked-data based distance measure. A sizable overlap accompanied by an analysis of experts' preferences over the non-overlapping portions indicate that the knowledge acquired using our approach is indeed musically meaningful and is further complementary in nature to the existing structured information.
Full PDF Version: 
Tags: 
Reviewed

Decision/Status: 
[EKAW] reject

Solicited Reviews:
Click to Expand/Collapse
Review #1
Anonymous submitted on 10/Aug/2014
Suggestion:
[EKAW] reject
Review Comment:

Overall evaluation
Select your choice from the options below and write its number below.

== -1 weak reject

Reviewer's confidence
Select your choice from the options below and write its number below.

== 3 (medium)

Interest to the Knowledge Engineering and Knowledge Management Community
Select your choice from the options below and write its number below.

== 4 good

Novelty
Select your choice from the options below and write its number below.

== 3 fair

Technical quality
Select your choice from the options below and write its number below.

== 3 fair

Evaluation
Select your choice from the options below and write its number below.

== 4 good

Clarity and presentation
Select your choice from the options below and write its number below.

== 2 poor

Review

[Overview]

The paper describes a novel approach for obtaining the salient characteristics of music styles using knowledge obtained from Wikipedia, particularly its category structure.
The technique starts by constructing a graph for each music style and then runs the PageRank algorithm in order to weight the nodes in the graph based on their structural importance. The top-ranking nodes in the PR vector are taken as salient features for the corresponding music style. Two experiments are carried out in order to evaluate the approach against DBrec, a music recommendation system that also uses Wikipedia, from two perspectives: objective and subjective. The former evaluation involves comparison in terms of statistics such as correlation whereas the latter is carried out with the help of experts that are asked to judge the outputs from the two systems. An analysis of the results is provided at the end.

[General comments]

The paper is about an interesting topic. However, there are two main concerns:
(1) The paper lacks clarity at some points, especially where the core approach is being described. For instance, the authors suffice to describe the graph construction, which is an essential part of their approach in some few lines (at the end of Section 3).
(2) None of the two experiments show that the performance of the proposed approach is considerably better than the previous work. For the objective experiment, having 60% of correlations as positive and the other 40% as negative is not a proof for “strong agreement”. Also, in the case of the subjective experiment no considerable improvement is observed with respect to DBrec, the comparison music recommendation system.

Finally, you might want to briefly discuss in the paper how your work is different from a similar work of Pilehvar et al. (2013) that obtain the same salience vectors (which they call semantic signatures) with the help of PageRank on semantic networks.

[Minor comments]

- Section 2: MIR, the first usage of this acronyms; please provide the full form in the first usage.
- Section 2: “One class of approaches take advantage [...]” -> takes
- Section 3, last paragraph: not clear; more detailed description is required on how the graphs were constructed and what the numbers in Table 2 show exactly.
- Section 4: “[...] three steps: entity linking. [...]”, entity linking might be confused with named entity disambiguation whereas it does not seem to be the same takes; if so, please use a different naming.
- Section 4: “Pagerank” -> PageRank
- Section 4: “dsizeesirable” -> desirable?
- Section 5: “[...], we propose a salience-aware distance (SASD)”; please clarify: at this point it is not clear what distance are you measuring (distance between what)?
- Eq. 5, Why those quotients (½ and 1/4)?
- Fig. 1 better to appear on the next page.
- Section 6.2: “Table. 6” -> “Table 6”

Review #2
Anonymous submitted on 25/Aug/2014
Suggestion:
[EKAW] reject
Review Comment:

Overall evaluation
Select your choice from the options below and write its number below.

== 3 strong accept
== 2 accept
== 1 weak accept
== 0 borderline paper
== -1 weak reject
* -2 reject
== -3 strong reject

Reviewer's confidence
Select your choice from the options below and write its number below.

== 5 (expert)
== 4 (high)
* 3 (medium)
== 2 (low)
== 1 (none)

Interest to the Knowledge Engineering and Knowledge Management Community
Select your choice from the options below and write its number below.

== 5 excellent
== 4 good
== 3 fair
* 2 poor
== 1 very poor

Novelty
Select your choice from the options below and write its number below.

== 5 excellent
== 4 good
== 3 fair
* 2 poor
== 1 very poor

Technical quality
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
* 2 poor
== 1 very poor

Evaluation
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
* 2 poor
== 1 not present

Clarity and presentation
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
* 2 poor
== 1 very poor

Review

This paper proposes an approach to identify the salient and discriminating aspects of a music style compard to other styles, e.g. as a basis to improve retrieval and recommendation.

The approach extracts Wikipedia pages related to the music styles under consideration (Baroque, Carnatic, Flamenco, Hinudstani and Jazz) and constructs a graph consisting of edges indicating how often one entity is references from a given entity (node).
A measure of salience is proposed to compute a salience signature for each music style, relying on PageRank and IDF-style category weighting. The approach is evaluated on a recommendation task with respect to a baseline relying on a Linked Data based Semantic Distance measure.

Overall, the approach is rather straightforward and quite adhoc. The main design choices are not well justified, in particular why the measure has been defined as it has been defined. Further, the novelty with respect to existing methods is not well justified. The discussion of related work is rather superficial, glossing over many related approaches at best.

It is also not clear what is specific about this method that makes it suitable for the music domain. The method seems generally applicable to other domains. In this sense it is odd that the authors emphasize the application to the music domain rather than the method per se. In fact, the application to discover salient and discriminating properties of music styles is in my view only one of many possible application domains on which the authors happened to choose to evaluate their method. Unless the methods proposed is really specific for the music domain - which in my view it is not - the authors should focus on the description of the method per se and the general type of problems that it has been designed for, presenting the application to the music domain only as one application and to provide a proof-of-concept and evaluation of their approach.

Review #3
Anonymous submitted on 02/Sep/2014
Suggestion:
[EKAW] conference only accept
Review Comment:

Overall evaluation
== 0 borderline paper

Reviewer's confidence
== 2 (low)

Interest to the Knowledge Engineering and Knowledge Management Community
== 3 fair

Novelty
== 4 good

Technical quality
== 4 good

Evaluation
== 3 fair

Clarity and presentation
== 3 fair

Review

The paper introduces an approach for quantifying the salience of musical characteristics in different music styles. Unlike the existing approaches which use linked data to compute distance measures between music style entities, the authors' approach extracts this knowledge from unstructured natural language text corpus. The approach is evaluated in a comparative evaluation in which the authors' approach is compared with a linked data based approach.

Overall the paper is well-written and technically sound, however certain parts should be extended with more details and examples to make the paper more clear (see comments).

Comments and points for improvement:

Introduction
- It is not very obvious at this point of the paper what the difference between entities and characteristics are as some of the examples of entities are intuitively closer to characteristics of musical styles (e.g. chords and scales).

Related work
- The relation between the work in the paper and related work in the field should be extended.

Data
- How were the entities for music styles selected?

Vichakshana
- typo, "it is dsizeerisable"
- "We have empirically chosen the value for the threshold to be three." How?

Salience-aware semantic distance
- The section should be extended and more details about the measure should be included. For example, details on what D1, D2 and D3 are and why they are weighed in such way.

Methodology
- More details needed on why yield and overlap are important to compare.

Evaluation
- The evaluation done in the paper compares the authors' approach (SASD) to an existing approach (LSSD). While this kind of comparison is valuable, there is not much information about how good the authors' measure is. The authors do test the usability of their approach but still with respect to the other approach. My opinion is that a more extensive usability test of the approach on its own should be done e.g. testing how much the experts agree with the ranked characteristics.
- In the user study, were there any differences with respect to the music style?
- Evaluation of other music styles