A Survey on Interaction Design with Large Language Models for Ontology Requirements Elicitation with Competency Questions

Tracking #: 4054-5268

Authors: 
Yihang Zhao
Xi Hu
Timothy Neate
Albert Meroño-Peñuela
Elena Simperl

Responsible editor: 
Dagmar Gromann

Submission type: 
Survey Article
Abstract: 
Ontology Engineering (OE) typically begins with ontology requirements elicitation, in which ontology engineers define the scope of concepts and relations an ontology must cover to adequately serve its intended application domain. Large Language Models (LLMs) can generate a large set of Competency Question (CQ) candidates to support this process; however, a target ontology scope is often multi-dimensional, ill-defined, and cannot be fully anticipated in advance, and existing tools provide little support for ontology engineers to subsequently explore and generate new CQ candidates (divergent thinking), evaluate, refine, and eliminate existing ones (convergent thinking), and thus progressively define a well-scoped ontology. We argue that interaction designs from LLM-based systems in arts and creativity domains, where divergent and convergent thinking have been extensively studied, are transferable to ontology requirements elicitation. To identify these designs, we conducted a Systematic Literature Review (SLR) of 50 papers, identifying 7 Interaction Techniques (ITs) and 14 User Interfaces (UIs), each justified with respect to how it supports divergent thinking, convergent thinking, or both. To explore the transferability and applicability of the identified ITs and UIs to LLM-based ontology scoping, we conducted a design thinking workshop (N=7) that produced a conceptual interaction model, a system prototype called OntoScope implementing that model, and a use case demonstration showing how OntoScope can potentially support an OE expert in scoping a university ontology. The identified ITs and UIs can serve as a reference for tool developers working on ontology requirements elicitation, broader OE tasks that require human reasoning and auditing over LLM-generated content, or designing user-friendly OE tools for domain experts and end users without prior OE expertise.
Full PDF Version: 
Tags: 
Reviewed

Decision/Status: 
Accept

Solicited Reviews:
Click to Expand/Collapse
Review #1
Anonymous submitted on 24/Apr/2026
Suggestion:
Minor Revision
Review Comment:

This revised version is a greatly improved version from the previous submission. The authors have made a huge effort in explaining details better, and adding further intuitions behind the work. However, I am still not satisfied by their answers to some of my concerns.

One important question was why they filter by venue. The answer is that this is done to keep the number of analysed papers small. But then this stops being a systematic literature review as they claim in Section 3. The authors are purposefuly excluding some, potentially important, works for the sake of having a small number of papers.

My second issue is with the limitation to art and creativity. My point is that the title and introduction focus on requirements elicitation with competency questions, but the survey is limited to papers in the creativity fields. There is a mismatch there: either the title, abstract and introduction are made less generic, or the survey is generalised to other cases of elicitation.

As a minor remark, my original review did not say that the forward and backward snowballing were applied in a wrong order, but that they were *described* in the wrong direction (calling "forward" what is actually backward and viceversa).

Review #2
Anonymous submitted on 14/Jul/2026
Suggestion:
Accept
Review Comment:

This is a largely improved version of the last submission. The authors have made a substantial effort in addressing the reviewers' comments in this revised submission.

The related work section has also been greatly improved, however, now reads more like a preliminaries chapter. The actual related work should focus on other or similar surveys that are in any way comparable by research question to the one proposed and then explain how the one proposed differs.

The method of venue filtering has been explained considerably better and an attempt of a justification has been included. However, unfortunately, the details from the response did not make it into the revised version, namely the fact that the ranking of the Top 13 papers is based on the ranking provided by the respective search platforms. These are actually not publication venues but just search platforms. Could you please clarify this confusion whether the filtering was based on publication venues, e.g. specific journals, conferences, etc., or on search platforms? The examples in Fig. 9 suggest the same confusion of publication venues and search platforms. However, Table 3 talks about actual publication venues.

For a higher transparency of the provided result corpus, could you please provide short references of the retained papers in Table 4? This is also interesting for the authors of the referenced papers in order to see in which category you classified their publications and for readers to get an overview of the 50 publications retained.

The new section on interaction techniques is very informative and well done. What is a little bit missing is the creation of this category system. If an IRR could be calculated, these categories must have been somehow part of the method. Could you elaborate on that? If so, why not present these classifications/categorizations instead of what is presented explicitly in an overview with citations alongside of or instead of Table 4?

In Section 3.5.5., however, suddenly the line spacing is changed, which needs to be fixed alongside using the correct template.

Given the improved presentation of Section 4, I now better understand and appreciated the workshop and its outcomes. I also better understand the connection of the survey, the workshop, and the prototype.

The limitations should be improved as the three selected databases are rather unclear in terms of the venues discussed beforehand. Please make sure that arguments are consistent and easy to follow.

One major concern that needs to be addressed is that the submitted format does not actually follow the formatting guidelines provided by the Semantic Web journal, as these have changed from IOS one-column to a SAGE two-column format.

I believe that these comments can be easily addressed to prepare a final manuscript, which is why I propose to accept this submission rather than going through another round of reviews.

Minor comments:
I believe the headings for the individual steps in the SLR method section should be capitalized, e.g. Step 1: Identification.
Empathise in the text vs. Empathize in Fig. 10