Abstract:
Large Language Models (LLMs) have achieved notable success across diverse domains, yet they remain prone to hallucinations and omissions due to their probabilistic reasoning and black-box nature. These limitations are particularly critical in high-stakes domains, where inaccurate or incomplete responses may lead to severe consequences, for example, failing to instruct a client correctly in an account activation procedure, despite having the acess to the information. Recent efforts to mitigate such issues have focused on integrating structured knowledge into LLM pipelines, predominantly through knowledge graphs. However, the selection of the relevant entities to generate a response may not be intuitive to retrieve from the graph's structure alone. The role of deeper semantic structures has not been systematically explored. Specifically, ontologies with axiomatic foundations could be used for automated inference, leading to a more precise set of entities, and therefore potentially providing a better input to generate more accurate LLM responses.
This paper investigates the impact of deep semantic enrichment on LLM-based knowledge extraction and response generation. We situate our contribution within the Neuro-Symbolic AI paradigm and propose a framework that combines knowledge graphs for contextual storage with ontological reasoning grounded in structural patterns that are ontologically well-founded in UFO. By leveraging the well-founded semantic nature of ontological entities, rather than solely their graph-like structure, the proposed approach constrains interpretation and supports more precise reasoning during prompt construction.
The framework is evaluated by a real company using a domain ontology of the Brazilian financial system, enriched with business rules and processes provided by an industry partner. We compare scenarios with varying strategies for leveraging semantic knowledge from the ontology, ranging from light-semantics approaches that rely only on graph-based properties, as well as a problem-based baseline designed by domain experts. Experimental results demonstrate that the Deep Semantics approach achieves superior performance in both entity extraction and response generation quality, consistently extracting a higher number of relevant entities and producing more complete and semantically aligned responses. These findings highlight the benefits of incorporating ontological depth and axiomatic semantics into LLM workflows.
Overall, this work provides empirical evidence that deeper knowledge representations can significantly enhance LLM reliability and interpreting ability, advancing the development of knowledge-driven large language models.
