Review Comment:
The main contribution of this research paper is the proposal of a new method called CEO (Counterfactual Explanations for Ontologies) to explain inconsistent ontologies using counterfactual explanations. The paper also draws a parallel between explanations for machine learning and ontologies, which enables the adaptation of eXplainable AI (XAI) methods to explain ontologies and their entailments. Additionally, the paper highlights the need for explainability methods in the Knowledge Representation and Reasoning (KRR) domain, which is widely used in medicine and the web, as demonstrated by ongoing projects such as the Gene Ontology and DBpedia.
This manuscript was submitted as 'full paper' and is reviewed along the usual dimensions for research contributions which include (1) originality, (2) significance of the results, and (3) quality of writing.
Originality
The CEO method is a novel approach to generating counterfactual explanations for OWL ontologies. While there have been previous efforts to generate explanations for machine learning models, the CEO method is unique in its focus on ontologies and its use of counterfactual reasoning.
Significance of the results
small Sample Size: The CEO method relies on a small sample size of experts to evaluate the quality of the counterfactual explanations generated by the system. This may limit the generalizability of the results and make it difficult to compare the performance of different methods.
limited applicability: The CEO method is tested in a specific domain, which may limit its applicability in other domains or contexts.
Interpretability: While counterfactual explanations can be useful for understanding how an ontology works, they may not always be easy to interpret or understand for non-experts.
Quality of writing
English needs to be checked and improved with a mother-tongue english speaker
What the AI algorithm detected -> What did the AI algorithm detect?
Minors:
the of use -> the use of
Improvements in the organization of sections
Introduction. I think that introductions should be concise and focused. Aim to engage the reader, establish the relevance of your research, and clearly convey the purpose and objectives of your study.
Therefore, I would propose removing the subsections and organizing it as follows:
Start with a general opening: Begin your introduction with a broad statement. You can mention a real-world problem
*Provide background information: Give a brief overview of the background information related to your research topic. Explain the key concepts, theories, or existing research that are relevant to your study. This helps the reader understand the context and significance of your work.
*State the problem or research question: Clearly state the specific problem or research question that your study aims to address. Explain why it is important to investigate this problem and how it relates to existing knowledge or gaps in the field. This will help readers understand the purpose and relevance of your research.
*Summarize related work (you do not need to put here all the related work sections). Review the existing literature and summarize the key findings or approaches related to your research question. Identify the strengths and limitations of previous work and highlight the gaps that your research aims to fill. This demonstrates your familiarity with the field and positions your work within the broader research landscape.
*Present your research objectives: State the specific objectives or goals of your research. Clearly articulate what you intend to achieve through your study and how it will contribute to the existing body of knowledge in computer science. This helps readers understand the expected outcomes and significance of your research.
*Outline the paper's structure (which you already did)
Related Work. The related work section is missing and I think there are enough mentioned works here and there but all of them and how you differ from the state of the art should be in one section named Related Work
Approach. The proposed approach section is a crucial part of a research paper in computer science. This section outlines the methodology, algorithms, models, or techniques that you plan to use in your research. Here are some suggestions for organizing and clarifying the proposed approach:
*Clearly state the objective and provide an overview by explaining the main steps or components involved and how they contribute to achieving the research objective. This provides a roadmap for readers to follow and understand the structure of your approach.
*Describe the fundamental principles, theories, or concepts on which your proposed approach is based. This helps readers understand the theoretical foundations of your work.
*Provide a detailed description of the technical aspects of your approach, including flowcharts where necessary to clarify the steps involved. Use clear and concise language to explain each component and its purpose.
*If there are any assumptions or limitations associated with your proposed approach, explicitly state them. Discuss how these assumptions or limitations may impact the results or the applicability of your approach.
*Explain why you have chosen the specific approach and how it aligns with the research objective. Provide evidence or references to support your decision.
*It is also helpful to connect the proposed approach back to the research problem or objective, emphasizing how your chosen approach is suitable for addressing the specific challenges identified in your study.
Concern about the limitations of conducting experiments on a single dataset and the statistical significance of the number of users. When designing and reporting experiments in a research paper, it is crucial to address these limitations to ensure the credibility and generalizability of your findings. Here are some suggestions for addressing these concerns:
*Acknowledge that your experiment is conducted on a single dataset and explain the reasons behind this choice. For example, you can discuss the dataset's relevance to the research question, its availability, or its uniqueness in terms of characteristics or domain. However, it is important to acknowledge that the results may be specific to that particular dataset and caution readers against making broad generalizations.
*Highlight the potential limitations of generalizing the results beyond the specific dataset used in your experiment. Discuss the dataset's properties, such as size, diversity, and representativeness, and how these factors may impact the generalizability of your findings. Consider suggesting future work that could involve multiple datasets or different data sources to validate and strengthen the conclusions.
*Discuss the potential impact of the sample size on the statistical power and the precision of the results. Consider providing a justification for the chosen sample size, such as resource constraints or the uniqueness of the user population.
*Offer strategies or suggestions for addressing the limitations of a single dataset and limited user sample size. For instance, you can propose cross-validation or bootstrapping techniques to evaluate the robustness of the results. Additionally, you could recommend future research that involves replication studies on different datasets or larger-scale user studies to enhance the reliability and generalizability of your findings.
|