AI-Assisted Knowledge Graph Metadata Curation: An Empirical Evaluation

Metadata is essential for making knowledge graphs (KGs) findable and reusable in line with the FAIR principles. However, KG metadata is often incomplete or missing, partly because creating high-quality metadata is a tedious and time-consuming manual task. This paper presents an AI-assisted form for KG metadata curation. The form is structured according to a KG metadata specification and uses a large language model to extract candidate values for metadata fields from KG documentation. The model standardizes the extracted values according to the KG metadata specification and presents them as suggestions that users can review, accept, or reject. We evaluated the AI-assisted form in a within-subjects study by comparing it with two baselines: a version of the same form without AI assistance and manual curation using RDF/Turtle. We assessed curator experience using the System Usability Scale, the Technology Acceptance Model, and open-ended feedback. To complement these subjective measures, we measured curation performance in terms of productivity and metadata quality in terms of coverage and precision. The results indicate that the AI-assisted form enables a more efficient curation workflow, leads to more complete metadata, and is preferred by participants over both baselines.