WYSIWYM -- Integrated Visualization, Exploration and Authoring of Semantically Enriched Un-structured Content

Tracking #: 589-1796

Ali Khalili
Sören Auer

Responsible editor: 
Guest editors Semantic Web Interfaces

Submission type: 
Full Paper
The Semantic Web and Linked Data gained traction in the last years. However, the majority of information still is contained in unstructured documents. This can also not be expected to change, since text, images and videos are the natural way how humans interact with information. Semantic structuring on the other hand enables the (semi-)automatic integration, repurposing, rearrangement of information. NLP technologies and formalisms for the integrated representation of unstructured and semantic content (such as RDFa and Microdata) aim at bridging this semantic gap. However, in order for humans to truly benefit from this integration, we need ways to author, visualize and explore unstructured and semantically enriched content in an integrated manner. In this paper, we present the WYSIWYM (What You See is What You Mean) concept, which addresses this issue and formalizes the binding between semantic representation models and UI elements for authoring, visualizing and exploration. With RDFaCE and Pharmer we present and evaluate two complementary showcases implementing the WYSIWYM concept for different application domains.
Full PDF Version: 

Minor Revision

Solicited Reviews:
Click to Expand/Collapse
Review #1
By Heiko Hornung submitted on 20/Dec/2013
Review Comment:

Regarding the previous version, the authors addressed the majority of my comments, with the exception of some comments related to the UI categories/techniques as well as the usability evaluation presented in the paper. For an "HCI-focused" journal I would have requested additional minor revisions, but regarding this journal and special issue my suggestion is to accept the paper.

Review #2
Anonymous submitted on 31/Dec/2013
Review Comment:

In the new version, none of the remarks made by the reviewer has been addressed by the authors.

The remarks are as follows:
(1)A more sophisticated definition of WYSIWYM model is needed.
(2)Figure 4 needs more clarification.
(3)A more comprehensive evaluation (including comparison evaluation) is needed.

Review #3
Anonymous submitted on 26/Jan/2014
Minor Revision
Review Comment:

The paper suggests that the problem of "semantic gap" between unstructured and structured content may be solved if the two can be authored, visualized and explored in an integrated manner. A formalism called WYSIWYM including the binding between UI elements and semantic representation models is proposed to reach the possible integration. The contributions of this paper include:
1) Propose a formalism called WYSIWYM (What You See is What You Mean), which implies a methodology of how to develop and evaluate a UI interface that users can view, explore and author semantically enriched unstructured content in a user-friendly way. The idea is interesting and somewhat original.
2) Take a comprehensive survey of different UI categorizations and UI techniques used in representing semantic models ranging from tree-based to graph-based and even hyper-graph-based, and draw a detailed binding table between varies of UI techniques and different semantic representation models. The survey requires great effort and maybe useful for tool builders to develop UI.
3) Develop two tools, RDFaCE and Pharmer, which try to show WYSIWYM formalism can help to build semantic editors with high usability. Though the article does not tell the reader what is WYSIWYM and what is not intuitively, the demos and corresponding articles on the web site provide some idea.

Though it is impressive and interesting, the following issues require further consideration:

1) The UI techniques listed are a little limited. Just list a few: 1. Techniques such as animation are not included. 2. "Associating" UI category only includes "Line connectors" and "Arrow connectors". For hierarchy relationship, nested items can be shown inside one another, the technique is widely used as suggested in reference paper [6]. 3. The "zooming" elements vary on different applications, it may be further categorized.
2) The demos mainly demonstrate highlight typed-text and some “callouts”, The UI that shows semantic representation model, such as tree, graph and hyper-graph, is not included. Furthermore, if the tree, graph and hyper-graph denote "structured content", it seems it is still unsolved about how to integration the "unstructured and structured content".
3) There is a gap between the formalism and the tools. e.g. How the formalism and different bindings guide the tool building process or helps to evaluate the tool usability?

In spite of the above issues, it may be a useful survey and a startup material for different readers to understand the key point of building UI interfaces of semantic-enriched content.