Reviewed

Submission in response to http://www.semantic-web-journal.net/blog/semantic-web-journal-special-ca...

Revised manuscript after "accept with minor revisions" - now accepted. The reviews from the first round are below.

Solicited review by Francois Scharffe:

The paper presents the pilot Europeana dataset. The dataset is important, rich and complex. It is a pilot as lessons learnt will enable to revise the publication. The paper is well written and gives a good overview of the dataset structure.

two minor remarks:
- Section 2 it is said that semantic markup is available on Web pages. It would be good to cite the technology used for the markup RDFa ? schema.org ?
-dereferencable -> dereferenceable

Solicited review by Dave Kolas:

This paper describes a prototype Linked Data version of the Europeana dataset.

* Quality of the dataset

The Europeana data on the museum / library resources is aggregated from a number of holders of the physical resources, thus the original providers have motivation to make the data accurate. It is possible that the aggregation of many sources means that some sources produce different subsets of data for the schema. The schema addresses the problem of multiple potentially conflicting records about a resource with proxies. It is not clear whether this is a better or worse approach than reification or named graphs for this purpose, but it appears sufficient. The other schema modeling is reasonable, though light on the interlinking (as noted in the paper). The authors do a good job of linking to other datasets, though it would be interesting to see percentages as well as raw links.

* Usefulness (or potential usefulness) of the dataset

This dataset could be potentially useful to a large number of people involved in or interested in the arts in Europe. It could also be combined with travel applications to know where to see particular works of interest. The prototype nature of the dataset leaves out much of the content currently in the non-linked-data Europeana dataset, somewhat mitigating its utility for the moment.

* Clarity and completeness of the descriptions

The paper is written clearly and concisely. The main classes in the data model are described well, and there is a good diagram of how these classes interact. An example record with properties might have been nice however.

Solicited review by Amit Joshi:

The paper is about the Europeana linked open data which contains open metadata with more than 2.4 million text,images, videos and sounds related to books, paintings, films, museum objects and archival objects throughout Europe. Data is gathered by Europeana from multiple data providers. Metadata is obtained from data providers, formatted according to ESE XML Schema and then converted to EDM for generated linked data version. The dataset is live and can be accessed either by downloading data dumps or executing SPARQL queries against the dataset. The significance of such unique dataset being open is, without any doubt, high. However, the paper has following weaknesses:

1. Use of provider proxy and Europeana proxy is not clear. Is it even required?
2. It would be good to provide examples of the items/resources in a dataset that uses existing ontologies and connects to other LOD datasets.
3. Number of references is very few (only two). Please revisit earlier sections and add additional references (ex: linked data principles)

Submission in response to http://www.semantic-web-journal.net/blog/semantic-web-journal-special-ca...

Solicited review by Axel Polleres:

This paper presents a preliminary description of a dataset wrapping information from the Institute for Development studies.

The dateset is potentially interesting, but the authors make no clear attempt to showcase possible uses.

The quality of the dataset is not yet clear, particularly, there is no comment on the sustainability of the project, beyond the exercise of creation of a wrapper, even the URI (http:///idswrapper.appspot.com) is marked as preliminary,

As for usefulness (or potential usefulness) of the dataset, it would be nice if the authors gave some concrete examples, e.g. of possible applications and queries that they envision with this dataset.

As for clarity and completeness, the examples and figures of Section 3.3 all seem to be cut off in the PDF. There is not query-, browse-, or search functionality as of yet, it seems and when I go on the Webpage, I have no guidance how to navigate this dataset and find out how it could be useful for my purposes.

In summary, this work is a potentially useful project in a preliminary stage which is welcome, but probably too immature for the purposes of this special issue.

I recommemnd the authors to deploy it further and find adopters, seek feedback in workshops, etc. first and target a journal again when the dataset has proven usefulness in some sense.

Solicited review by Philippe Cudre-Mauroux:

This short paper describes a wrapper used to convert and interlink metadata from the Institute of Development Studies (IDS) into Linked Data. Overall, I found the paper interesting and well-written. More specifically, the architecture of the wrapper is compelling: it dynamically converts identifiers to linked data, calling the IDS REST API and creating links to further Linked Data resources on-the-fly taking advantage of the Java restlet package deployed on Google's AppEngine. Also, the authors give a nice overview of the state-of-the-art in international development APIs and data dumps. A few interesting points are mentioned in the paper but should be in my opinion discussed in more detail, namely: i) to what extend would it be possible to mine information directly from the text of the 53'000 research documents? ii) since the URI scheme is human-friendly (it includes the literals corresponding to the "collections"), why not adding the label of the resource itself as well? (having the label in the URI is imho useful in many situations) iii) it would be really interesting to have some information on the efficiency and effectiveness (e.g., precision and if possible recall) of the various linkage services and finally iv) why only develop propriety client applications based on this data? Wouldn't a SPARQL endpoint / RDF dump be possible? Please explain in the context of your project.

Solicited review by Norman Heino:

This paper describes a dataset about research results from development studies as made available by IDS (Institute for Development Studies).
The software component used to convert IDS data to RDF (called 'IDSWrapper') is also described in brief.
Like the original IDS data the converted set is about documents, organizations, categories, countries and regions of research focus.
The data is on the fly enriched with links from Lexvo, DBpedia, IATI, and Geonames which helps in understanding the data once found.

I found the data to be of medium to high quality.
Some values are obviously wrong or missing, but most of the data seems plausible.
What I particularly like is that some properties have been replaced by or linked to more common ones.
Replacement has been done for rdf:type and dcterms:language, while others like ids:date_created or ids:cat_parent have been linked via rdfs:subPropertyOf relations to Dublin Core, FOAF, or SKOS vocabularies.

Usefulness of the dataset is a bit hard to evaluate since it consists mainly of metadata about research articles.
The value obviously lies in the articles themselves and the dataset's raison d'être is making it easier to find those articles.
As the authors note their implementation lacks a search feature but the technical reason given is a bit unconvincing.
Why are search queries not just forwarded to the original search API?
The paper quickly mentions a client application that could be used to 'browse through the IDS documents'.
I would like the authors to elaborate a bit more on the potential that is gained through RDF here.
For instance since the categorization properties are derived from SKOS this would enable generic SKOS browsers to be able to browse articles by category hierarchy: something which is not possible through the IDS interface.

The paper is clearly written and provides a usage example as well as the bigger picture on supporting development practitioners.
Other than a few typographical errors and minor issues no editorial revision is needed.

Typos
-----
* Abstract: "as well as its medata" => "as well as its metadata"
* page 2, section 3, par. 1: "different type of entities" => "different types of entitites"
* page 2, section 3, par. 3: "and based on those to establish links" => "and based on those establish links"
* page 3, section 3.3.3, par. 1: "example of such link" => "example of such a link"
* page 4, par. 1: "by using applying their URI scheme": remove one of using, applying

Other issues
------------
* Why are Fig. 1--5 presented as screenshots? To improve presentational quality I suggest reaplacing them qith real tables.
* For standard namespaces you should use the standard prefixes as per recommendation (you do so for rdf but not for owl)
* The vocabulary still uses the "http://example.org#" namespace; an obvious oversight

Submission in response to http://www.semantic-web-journal.net/blog/semantic-web-journal-special-ca...

Revised manuscript after an accept pending major revisions, now accepted for publication. The reviews of the original submission are beneath the second round reviews.

Second round reviews:

Solicited review by Aba-Sah Dadzie:

The authors have done a good job of addressing the review comments. I'd recommend acceptance for the special issue, with a few minor corrections/additions. With regard to the specific requirements for this call:

* Quality of the dataset - this is well described and pointers to sample queries allow the reader to directly access the linked data.

* Usefulness (or potential usefulness) of the dataset - clearly contributes to the arts and cultural heritage. Further work planned by the authors indicates potential for further use and added value enabled by the conversion to LD.

* Clarity and completeness of the descriptions - the revised paper addresses concerns expressed in the 1st review. The use of existing standards and extensions to these are well described and referenced. The process followed in generating the dataset and the overall aims are also more clearly described.

* Name, URL, version date and number, licensing, availability, etc.
Version information and licensing information missing

* Topic coverage, source for the data, purpose and method of creation and maintenance, reported usage etc.
* Metrics and statistics on external and internal connectivity, use of established vocabularies (e.g., RDF, OWL, SKOS, FOAF), language expressivity, growth.
* Examples and critical discussion of typical knowledge modeling patterns used.
- Addressed in sufficient detail

* Known shortcomings of the dataset. - fairly well addressed, but see point below.

_____________________

Additional points to address

p.2 - "Although this approach ensures a level of consistency and interoperability between the datasets from different institutions it creates a disconnect between the cultural heritage institute original metadata model and the Linked Data version."
This begs the question "why"? Also, this appears to be contradicted in the final paragraph in this section.

p.5 - " Finally 34 persons were linked to persons in DBpedia. This is a relatively low number as 1) most of the Amsterdam Museum people are not notorious enough to appear in DBPedia" - do you need to be "notorious" to appear in DBPedia? "Famous" or "noted", maybe, but notoriety is normally considered to be negative or at best not complimentary.

Language & Presentation

Generally well written but needs a spelling & grammar check and proofread for minor errors. Among others,

p.4 - "proxies-aggregation" -> "proxy-aggregation"

Formatting of URLs

Some of the URLs them break because the formatting is splitting them and/or inserting whitespace at delimiters - requires the reader to copy the full URL and delete the white space inserted to reach the intended address.

Solicited review by Philippe Cudre-Mauroux:

This second iteration corrects the minor flaws of the first version of the paper (which I already liked actually). From my perspective, this paper is ready for publication.

Solicited review by Fabien Gandon:

my previous comments were addressed.

First round reviews:

Solicited review by Fabien Gandon:

This paper presents the Amsterdam Museum Linked Open Data set
The access point, content, metrics, statistics, modeling rationale, etc. are provided by the paper, which is in my opinion a very good contribution to this CFP.

"with suffix 'proxy-', 'aggregation-', 't-' or 'p-' for proxies, aggregations, concepts and persons respectively (eg. am:proxy-22476."
Don't you mean prefix ?

"There are also 34 links to DBpedia."
Any reasons for such a low number?

Solicited review by Aba-Sah Dadzie:

The paper describes the generation of the Amsterdam Museum Linked Dataset, as part of the Europeana project, to make more accessible information about the collection and people related to the various objects and the museum as a whole. The Linked Data was created by editing a "crude" RDF dump, to, among others, ensure interoperability with other cultural heritage data and the Europeana Data Model.

A few examples of use of the source data are given, and the authors discuss the benefits that the conversion to Linked Data is expected to bring. A specific example is the creation of a mobile tour guide. An overview of the data structure and work to promote interoperability with domain-specific models and more general standards are detailed. The work reported includes web services for querying and URLs from which to browse the data. The authors note the need to periodically regenerate the dataset to capture changes in the source data.

I have a few reservations about this paper. While the authors provide a good amount of information about the dataset and the technology used to create it, it reads more like a project report that ticks off a list of deliverables than a description of the linked dataset and the design process followed - what it should be. I would suggest, where Europeana is first mentioned, that the authors give a brief description of the project as an introduction to its relation to the generation of the linked dataset (that it is a project the authors are involved with is not obvious till the end of the paper). This should make it easier to understand the impact on Europeana, based on lessons learnt from the process followed - this is to a large extent a pointer to potential reuse, further enrichment and maintenance of the Linked Data generated, that is wasted. Also, the authors conclude with future work on Europeana, not the Amsterdam Museum dataset.

A critical discussion of design choices and the knowledge modeling is missing. There is no comparison with related work - the four references are all self-citations pointing to more detail on specific aspects of the work reported. While I don't expect a detailed literature review in a short paper, obvious areas where a review could be carried out are the usage section, the description of the model used and how this relates to or is an improvement on other similar, domain-specific datasets (Linked Data or otherwise). I acknowledge other models and schemas are mentioned - but these simply refer to URLs to a project page or other rather than indicate why they constitute a good or balanced decision.

DETAILED REVIEW

p.1
"While larger cultural heritage institutions such as the German National Library or British National Library have the resources to produce their own Linked Data, metadata from smaller institutions is currently only being added through large-scale aggregators such as Europeana."
This statement is open to debate - I would amend it to something like " smaller institutions often depend on large-scale aggregators such as Europeana." AND back the claim with an appropriate citation.

"published it as "five-star" Linked Data" - "five-star" should be cited, using Berners-Lee's "Linked Data - Design Issues" article (http://www.w3.org/DesignIssues/LinkedData.html)

p/2
"The Amsterdam Museum Linked Data set implements best practices that -, the together with its methodology and tools- Europeana is keen on adopting for its future workflow."
Need evidence to back this claim. Also, is there something missing after the hyphen?

I don't understand the resource URI derivation. A suffix terminates a word; however the example given has "proxy" (the suffix) followed by a numerical code. A complete example might be useful here.

"We used purl.org URIs since for this conversion we were not in the position to use the Amsterdam Museum namespace for our Linked Data server." - why not?

It is not completely clear what the RDF relations and the conversion of the language information at the end of S2.1 are till later in the paper - forward-referencing (annotated sections of) Fig.2 would be useful here.

p.3
"In total the object metadata consists of 5,700,371 RDF triples of which many have a thesaurus concept or person resource as object." - How many is "many"? - the word is too vague to be meaningful.

"Two Amsterdam Museum classes am:Exhibition and am:Locat were defined as rdfs:subClassOf of the EDM class edm:Event."
Are these two classes particularly meaningful or are they simply meant as examples?

"Most term-based thesauri, including the AM thesaurus, have a more or less uniform structure (ISO 25964) making" - what does the ISO standard mean or refer to here?

p.5
"Linked Culture Data web" - is this referring to a particular initiative?

==============================

Figures & Tables

The caption of Fig. 1 is too long (especially compared to the figure content) - I would suggest working the description into the main text and providing a more concise caption.

Figure 2 is referenced in the text before Figure 1 - Figure 2 should be brought forward.

Figure 2 caption - "... with their super-properties and -classes in italics." Does "-classes" imply "SUPER-classes" - if so this must be explicitly written, a "-" works as a shortcut for suffixes, not prefixes.

Citations & Bibliography

Footnote 3 (URL) goes to an administrator login
Footnote 4 (URL) displays a tiny XML file (to do with diagnostics)

While http://purl.org/collections/nl/am IS redirected to http://semanticweb.cs.vu.nl/europeana, the resource itself is not found.

The 'Object Re-use and Exchange (ORE) model' should be cited properly, using whichever is most appropriate of the papers by Lagoze & Van de Sompel et al.

Language & Presentation

A small number of typos (not listed here) will be caught by an auto spell check.

All acronyms must be expanded at first use. This is especially important for those not in wide use. E.g.,
- OAI-PMH interface (p.2)
- RDA Group 2 metadata standard (p.2)

English uses "," as a 1000 delimiter - unlike some other languages which may use ".". Simply because this paper is in English it should use the "," convention. More importantly, usage should be consistent - this paper uses both. (p.2,4) This gets even more confusing on p.4 where "." is used in a sentence as a decimal point, and then also used as a 1000 delimiter.

p.3
"These properties are mapped to RDA Group 2 elements using 20 rdfs:subProperty relations were defined." -> "These properties are mapped to RDA Group 2 elements using 20 rdfs:subProperty relations."

p.5
"Where the current Linked Data pilot of Europeana (data.europeana.eu) focuses on producing a Linked Data set based on the already-ingested metadata consisting of a minimal set of Dublin Core properties."
This is not a sentence.

Solicited review by Philippe Cudre-Mauroux:

This short paper describes the Amsterdam Museum LOD. The paper starts by describing the modeling and conversion methodologies (basically, the metadata and vocabulary were exported from OAI-PMH as XML, converted to RDF, curated using rewriting rules, and finally interlinked). Mapping onto the Europeana Data Model (EDM) was carried out using subclasses and subproperties on the one hand, and a proxy-aggregation pattern (supporting EDM's multiplicity of providers by separating object metadata and provenance metadata) on the other hand. The resulting LOD is available online over HTTP (serving HTML/RDF-XML/Turtle by content negotiation) and as a GIT repository.

Overall, I liked the paper and found the description of the modeling/export process as well as the description of the data itself interesting and technically sound. The methodology used to produce the LOD is rather standard, but nevertheless compelling and gives a down-to-earth, pragmatic account of how to export local cultural data and map it to EDM. I inspected a number of entities online using the HTML interface and all the metadata I looked at appeared to me as correctly and precisely exported, hence supporting the claim of the authors (i.e., preserving the original richness of the data being LODified). The paper would in my opinion be stronger if it included i) more information about the linking process (e.g., high-level description of the Amalgame alignment platform, precision/recall results for the automatically created links) and ii) additional detail on the update procedure (how efficient/scalable is the overall export procedure? Is there any bottleneck? Would incremental updates be possible? etc.)