Neural Language Models for the Multilingual, Transcultural, and Multimodal Semantic Web

Tracking #: 2244-3457

Dagmar Gromann

Responsible editor: 
Guest Editor 10-years SWJ

Submission type: 
A vision of a truly multilingual Semantic Web has found strong support with the Linguistic Linked Open Data community. Standards, such as OntoLex-Lemon, highlight the importance of explicit linguistic modeling in relation to ontologies and knowledge graphs. Nevertheless, there is room for improvement in terms of automation, usability, and interoperability. Neural language models have achieved several breakthroughs and successes considerably beyond Natural Language Processing (NLP) tasks and recently also in terms of multimodal representations. Several paths naturally open up to port these successes to the Semantic Web, from automatically translating linguistic information associated with structured knowledge resources to multimodal question-answering with machine translation and multilingual text-video knowledge representation with embeddings. Language is also an important vehicle for culture, an aspect that deserves considerably more attention. Building on existing approaches, this article envisions joint forces between Neural Language Models and Semantic Web technologies for multilingual, transcultural, and multimodal information access.
Full PDF Version: 


Solicited Reviews:
Click to Expand/Collapse
Review #1
By Philipp Cimiano submitted on 11/Jul/2019
Review Comment:

This paper provides a nice overview of applications of neural language models in the context of the Semantic Web supporting multilingual, transcultural and multimodal transfer, adaptation and localization. The paper is very well written and generally very clear.

I have a few suggestion on points where the paper could be expanded:

In the Transcultural Section, the paper would benefit from more examples of transcultural transfer and /or adaption. The section is low on examples, making it rather abstract. On page 5 top, left column, what is "a connection of cultural evolution and language as well as connections to knowledge representation". Can the author spell this out a bit?

On the other hand, all paragraphs "Challenges and Opportunities" list exemplary works and specific tasks. I would have liked to see a more general characterization of the open problems and opportunities if they are overcome. I.e. what benefits to SW systems or users have if "broad coverage in terms of language and visual-manual modality" is established? I was hoping to see the "big questions" and "big impacts" to be addressed rather than listing only some exemplary works.

On page 5 top, the authors talks about enabling "transcultural query-answering". I would welcome an explanation or example to illustrate this concept.

Overall, I liked the paper and found it very informative as an overview paper. I recommend this for publication in the special 10 year issue.

A few typos:

Section 5

withing cultural => within cultural

Such a cognitive frameworks => framework

Section 6

also considering mutisensory => multisensory

Review #2
By Claudia d'Amato submitted on 31/Jul/2019
Minor Revision
Review Comment:

The paper surveys approaches grounded on Neural Language Models and envisions joint forces between Neural Language Models and Semantic Web technologies specifically for: multilingual, transcultural,
and multimodal information access.

The paper focuses on extremely important problems and traces interesting research directions. Still I would suggest some improvements from the reader perspective in order to make the main message more clear and straightforward. In the following more detailed comments are provided.

The description of the paper contributions and goals provided in the abstract could be a bit more focused, currently they result a bit vague. Similarly, until the end of section 1, the actual goal of the paper does not result fully clear to the reader.

In section 1 there is a jump from the motivations arguing for the necessity of multi-lingual support to the reference to specific solutions. A more curated connection between these two aspects would be beneficial. For instance, the content reported at the end of the 2nd column (page 1) could be anticipated before introducing specific information concerning neural language models (NLMs).

Section 2 provides an appreciated and quite clear overview of the basic solutions at the state of the art. At the end of section 3 some discussions concerning the value added provided by the adoption of NLNs are somehow expected. This aspect seems to be actually treated in section 4. An explanation or even a reference to it should be provided at the end section 3.

It is not clear the reason why section 4 is opened with a contrast to multimodal information access that does not seem to be extensively treated in the previous section, where actually most of the focus seems to be overall on multilingualism.

In section 4, the challenges and opportunities reported for “machine translating SW” and “machine translation for reasoning” result a bit vague. A more detailed discussion would be appreciated. A similar comment applies to section 5 specifically concerning challenges and opportunities for “cultural heritage”.

Line 45, 2nd column: “Spaio-temporal” —> “Spatio-temporal”

Review #3
By Freddy Lecue submitted on 19/Aug/2019
Minor Revision
Review Comment:

This paper presents a vision for multilingual, transcultural, and multimodal information access through joint Neural Language Models and Semantic Web technologies.

A great paper with a concise view with clear next challenges to be addressed by the joint community. Nice potential direction for the ML community to integrate some of the Semantic Web / linked data community work - definitely worth having a look from both communities.

I would recommend adding another challenge on lighting the semantic representation for the ML community to integrate semantic representations more easily in the learning process - go beyond RDF/S / Sparql i.e., towards more knowledge graph-like domains where semantics is represented in a more lightweight form. Something that might become mainstream in a near future.

Great picture of the field of NLMs and NLPs - I would recommend to skip some fo the details related to autoencoders and transformers, as they do not serve the rest of the paper. Reference would be more appropriate.

- Semantic Web reference missing in Introduction
- NLP reference missing in Introduction
- structure or unstructured -> structured or unstructured
- knowledge graph reference missing in Section 2