Generating ontologies from Intelligent Tutoring System courses. A generic approach

Paper Title: 
Generating ontologies from Intelligent Tutoring System courses. A generic approach
Authors: 
Hector Escudero, Ramón Fuentes-Gonzalez
Abstract: 
In recent years a great effort has been done in order to create Intelligent Tutoring Systems that get close to the human teaching. Some of the handicaps of the systems already created is the impossibility of sharing the courses between different Intelligent Tutoring Systems and the difficulty to create them. Whereas a great amount of SCORM-compliant learning objects is being created for being imported in educational systems, there is a research current that pleads for ontologies as domain knowledge representation systems. We have created a generic and extensible authoring tool that creates courses for different intelligent tutoring systems. The authoring tool allows the creation of courses for different types of intelligent tutoring systems and saves those courses as ontologies. This allows the reusing of domain models between different Intelligent Tutoring Systems. This paper focuses in the ontology creation and populating process.
Full PDF Version: 
Submission type: 
Full Paper
Responsible editor: 
Decision/Status: 
Reject
Reviews: 

Solicited review by Renata Guizzardi:

In this paper, the authors intend to propose a method to develop Intelligent Tutoring Systems (ITS) based on the use of ontologies. Apparently, the method provides some advantages in the sense of supporting and easing the development of ITSs. However, it is clear to me that they do not understand what an ontology is and what it can be made for. They apply the so-called ontologies as generic exporting mechanism from one ITS to another, so the models serve more as a course formatting method than a real ontology. This can be grasped by the reader throughout the paper, and it is also clearly understood from the following exerpts from the Introduction:

"We have created an authoring tool where the user can configure the characteristics of the ITS where the course will be represented, and create the course using it. Then, the course will be saved in a generic way, so it can be reused in the same authoring tool to create a course for a different ITS."

"It will be explained why have we cosen ontologies as saving format"
It is important that the authors take into account that ontologies are meant to provide semantic interpretation of (for instance) a domain. Thus, the model must be precise and complete enough. That is why so many ontological models have some kind of formalization behind them, to provide such semantic precision. The models that you apply in this work are no different than concept maps, database models or relational models. It is a fact that in some systems, ontologies are represented by such models. However, the representation format is different than the ontological model per se.

In several parts of the text, it becomes apparent that the ontology definition you use is flawed. For instance:

- In section 2, you mention that "Text", "CheckBox", "Combo" and "List" are basic types in your ontology. However, this does not seem like ontological concepts at all. They are data structures and not domain concepts.

- When you claim that "The first idea to save the courses in a generic way was to create an XML language for it.", it becomes more than clear that you do not apply a real ontology because XML cannot be efficiently applied as an ontology representation mechanism, simply because it lacks the ability to make semantic distinctions.

- In section 3, you also say "The course creation process is based in a set of concepts that have relations between them. This could be seen as an ontology." This is a too simple definition for an ontology! As aforementioned, an ontology is much more than that.
Other problems with the paper:

- The title gives the impression that the paper talks about a method to develop ontologies. But in fact, the paper concentrates on a method do develop ITSs using ontologies.

- In section 2, you mention some tools that are not explained in the paper. These are "Iris Shell" and "AHA!" Since the SW Journal covers many areas, you must explain what these tools are, otherwise it is hard to relate to them only by using the provided references. After all, the paper should be self-contained.

- In the Introduction, the paragraph starting with "In order to solve these problems, we propose…" needs more structure, e.g. separating "first" and "second part" in different paragraphs or using bullets.

- In section 3, in the paragraph starting with "This approach would fulfill…", you make the discussion assuming the reader understands about the editor's structure. I do not understand anything about this next concept issue!

Important references for you to understand better about ontology and its applications are

N. Guarino and P. Giareta (1995). Ontologies and knowledge bases: Towards a terminological clarification, In Mars, N. (Ed.) Towards Very Large Knowledge Bases: Knowledge Building and Knowledge Sharing, IOS Press, p. 25-32.

Mittadis D. Lytras, Athanasia Poiloudi and Nikolaos Korfiatis (2003). An Ontology Oriented Approach on eLearning: Integrating Semantics for Adaptive eLearning Systems In Proceedings of the ECIS 2003.

There are English typos here and there:

- Abstract: "This allows the REUSING…" should read REUSE
- Introduction: "This is the first part of our ongoing research, the creation of an authoring tool capable of CREATE courses" should read CREATING.
- Section 2, paragraph 1: "convertir" should read "CONVERT"
- "… why have we COSEN ontologies as saving format…" should read CHOSEN.
- Conclusion: "sematic" should read "SEMANTIC"

Solicited review by Adila Krisnadhi:

The authors' proposed the following general idea (which is quite nice): (i) separate the course (which is seen as a collection of knowledge) and the ITS (which acts as the interface); (ii) model a course in an ontology generic enough so that it can be used by different ITSs. Modeling the domain of a course as a concept map provides a way to obtain a generic representation of a course. Moreover, hiding the gory details of the ontology creation from the user saves him/her from a headache of hard-coding the ontology itself. The authors provide some examples which are intended to demonstrate how this objective can be achieved. The authors justification that the objectives are attained is presented through a narrative throughout Section 3, followed by a working example in Section 4.

Judging this paper along dimensions for research contributions, however, does not at all convince me that the paper should be accepted. First, the general idea as summarized above, although nice enough, does not give a strong contribution from the point of view of originality. The possibility of re-using knowledge is precisely the main motivation of people using ontology for ITSs and this is, I believe, also the driving motivation for earlier works in this area (including the ones referenced in this paper). The main obstacle is thus to use ontologies in such a way that knowledge that is obtained from a course can be re-used as much as possible (thus the need to create sufficiently generic ontologies), while at the same time, to avoid the pitfall of burdening the lay user with the task of creating them.

I feel that the main challenge addressed by this paper is more of engineering nature, rather than conceptual research. If the paper is able to provide a strong, robust, working solution, then I would agree that it would be a significant advance. Unfortunately, the paper's content does not show this. I can understand that deeper discussion about the proposed system has already been published elsewhere, so this paper should only contain a short overview of it, but a robust and working solution also need a justification in the form of rigorous and systematic experimental studies, especially since the authors claim that their system can generate ontologies for ANY type of ITSs. Clearly, this is severely lacking in the paper. Short examples are insufficient, especially due to the engineering nature of the problem's solution.

The authors used OWL Lite as the language for the ontologies to which the courses are saved. But subsequent explanation does not offer any insight as to whether expressivity of OWL Lite is really needed. No part of the examples show the use of specific constructs and vocabulary from OWL Lite. One usually uses OWL Lite (or any OWL fragment for that matter) if either there is a need to perform reasoning over the knowledge, or at least, to express complex logical constraints/statements about the domain being modeled. This is completely missing from the paper. Section 4 only mentioned expressing "is-a" relation and throughout the paper, I can only deduce that the authors need a language to express explicit, simple relationships between concepts. If this is the only motivation, then I believe, RDF or at most RDFS framework would be sufficient. This also fits the fact that only SPARQL (and no reasoning) is used (aside from the modeling language) for doing anything with the ontology.

I see that Section 4 does not convey anything meaningful except a collection of Java-like code snippets. If an extended example is intended here, then the authors need to present explicitly the knowledge related with the course in the example. This includes the structures, concepts, relationships, as well as other specifics and peculiarities. How exactly the separation between the course component/content which will be saved into an ontology and the part which is deemed as acessories/representation (thus no need to be saved) should be clarified. The authors also need to point out which part of the knowledge that needs specific constructs from the chosen ontology language. The overall resulting ontology should also be presented. If there is any use of SPARQL, the authors should present the use cases and scenarios or otherwise, particular needs within the system. Writing down the SPARQL as a Java code snippet should also be avoided. If reasoning is done anywhere, this should be explained clearly, including the motivation and details of how it is done. Since the authors intend that the course can be re-used by any ITSs, there needs to be a detailed discussion on how and why this is actually achieved in the real cases.

Again to emphasize my point earlier, some systematic experimental studies to justify the authors' claim can be included after describing the example. Moreover, not only backing up the claim the re-usability of the created courses through the ontologies but also other aspects such as ease of use, compatibility, user satisfaction, etc. The authors did point out that this is going to be part of future work. However, this paper in the current version is pretty weak without these experimental studies.

================================

Minor issues/typos:
- Throughout the paper, the usages of reference numbers as words should be avoided, especially as the subject of a sentence. For example, there is the following sentence:
[5] and [28], for example, have used concept-maps in tutoring systems to organize knowledge successfully.

The above sentence is better written as follows:
Chang, et al. [5] and Shih, et al. [28], for example, have used concept-maps in tutoring systems to organize knowledge successfully.

- Throughout the paper: several usages of the word 'de' where actually 'the' was intended.
- Throughout the paper: when referring to a figure, use 'Figure' with capital F, instead of small letter f; also, write 'in Figure 1' instead of 'in the figure 1'
- Throughout the paper: if possible, put all figures at the top of a page, so they look nicer and tidier.
- Second author affiliation: "Automatics and Computing" is written twice.
- Abstract, line 1: 'that get close to the human' --> replace 'get' with 'are'
- Abstract, line 2: 'is the impossibility' --> replace 'is' with 'include' or 'are'
- Abstract, line 4: 'for being imported' --> replace this phrase with 'to be used'
- Abstract, line 4: 'there is a research current that' --> move 'current' before 'research'
- Abstract, line 4: 'ontologies as domain' --> replace 'as' with 'to represent the corresponding'
- Abstract, line 5: 'knowledge representation systems' --> delete 'representation systems'
- Section 1, page 1, left column: 'research on prototype systems' --> replace 'prototype' with 'prototypical'
- Section 1, page 1, right column: 'However, ITSs go beyond' --> replace 'However' with 'Furthermore'
- Section 1, page 2, left column, 2nd paragraph: 'authoring tool capable of create courses' --> replace 'create' with 'creating'
- Section 1, page 2, left column, 3rd paragraph: 'why have we cosen' --> replace 'cosen' with 'chosen'
- Section 2, page 2, right column, 3rd paragraph: 'Iris Shell ([1],12)' --> the reference numbers should be [1][12] or [1,12]
- Section 3.1., page 4, left column, 1st paragraph: 'decisions in some points of the process' --> replace 'some' with 'certain' or 'particular'
- Section 3.1., page 4, left column, 1st paragraph: 'Most times' --> replace with 'Most of the time'
- Section 3.2., page 4, right column, bottom paragraph: 'based on based on' --> delete one 'based on' here
- Section 3.2., page 5, right column: fix this: 'referencia a http://www.w3.org/TR/rdf-schema'
- Section 3.2., page 5, right column: 'the the concept created' --> replace with 'the created concept'
- Section 4, page 7, left column, 1st paragraph: 'Irirs Shell' --> 'Iris Shell'
- Section 4.1., subsection title: should be 'Configuring the ITS'
- Section 4.2., subsection title: should be 'Creating the course'
- Section 4.3., subsection title: should be 'Saving the course'

Solicited review by anonymous reviewer:

The authors might want to learn more about ITS. Woolf's book:
"Building Intelligent Interactive Tutors: Student-centered strategies for revolutionizing e-learning" and many papers presented at AIED- and ITS-related conferences are good materials.

The key issue of ITS is in its intelligent adaptation to learners on the fly. So, not only data/knowledge structure but also its control mechanisms are to be investigated. Examples include learner modeling, tutoring strategy, adaptive feedback, intelligent interaction, etc.

What discussed in this paper is not for ITS but for conventional CAI whose tutoring behavior is more or less predetermined in terms of adaptation. If you don't have to discuss adaptive control structure, you can easily discuss reusable content/course of e-Learning systems. However, it is not the case for ITS which requires sophisticate and complex control structure which in turn requires specialized techniques.

What is discussed here are static structure of coursewares and very primitive and elementary exercise of building is-a hierarchy using object-oriented modeling. There is no reason why the authors claim they introduced "ontology".

The authors claim "Those systems record the domain models as ontologies like our system, but still there are some differences.
First, the content they create is specific for a
platform or ITS," on page 3, but it is not correct. Just one counterexample is SCORM (Sharable Content Object Reference Model)[16], which is one of the most well-known standards.

Even worse, the developed system is dependent on Iris shell which is based on Merrill's CDT, which is not a generic.

Tags: