WikiBias: a Framework to Explore Bias in the Wikimedia Ecosystem through Semantic Modeling and Biographical Event Extraction

The paper presents WikiBias, a framework intended to explore bias in the Wikimedia ecosystem by combining semantic modeling, Wikidata/Wikipedia data, biographical event extraction, and network analysis. The paper focuses on writers and studies how gender and origin interact in the representation and visibility of different groups, especially women and Transnational writers. The topic is important and timely, since Wikimedia resources are widely used and can propagate existing biases.

Overall, the paper is generally well written and addresses a relevant problem. The authors have also carried out an extensive set of experiments in several directions: ontology modeling, biographical event extraction, entity linking, knowledge graph augmentation, representational bias analysis, and network analysis. The results are interesting and relevant, especially those showing that extracted biographical predicates can reveal gendered patterns in the representation of writers. In particular, the paper gives evidence that women writers are more often associated with private-life events such as marriage and parenthood, while men are more often associated with public or career-related events.

The paper presents many components and experiments, but several central methodological details are not sufficiently clear. In particular, the framework itself is not described as a clear reusable framework, the ontology is under-specified, the event detection model is not described in enough detail, and the external resources do not seem sufficient to reproduce the experiments. Please carefully read the following concerns:

1- The title already says a "framework" as the main contribution, but the paper does not clearly describe the framework as a structured system. The components are discussed across different sections, but there is no overall architecture showing the modules, inputs, outputs, intermediate resources, and expected use. A framework paper should make it clear how the different parts fit together (source data, ontology, knowledge graph, extraction pipeline, bias analysis, and network analysis). A diagram and a clearer description of the workflow/achitecture would substantially improve the paper.
2- The ontology description is under-specified. The paper says that the People in the Media Ontology is based on DOLCE and Ontology Design Patterns, but the ontology is introduced mostly through examples around pim:BiographicalSituation. It is not clear how many new classes and properties were created, which axioms are used, and how roles are modeled across different situations. For example, the current explanation suggests that a person has a role, but a person can have different roles in different biographical situations. The authors should clarify the ontology design, provide more details about the classes and properties, and explain the modeling choices more explicitly.
3- The event detection model is not sufficiently described. The paper spends considerable space describing the corpus, training sets, and evaluation results, but the actual model architecture and training setup are not clear enough. The authors should explain which model was used, how it was trained, what the input and output format is, which hyperparameters were used, and how the development set was used. It is also unclear why there is no cross-validation experiment, given the relatively small size of the annotated data used for training, development, and testing.
4- The choice to work only with English Wikipedia pages is a major limitation for a paper about bias and underrepresentation. The paper starts from a much larger set of writers, but then keeps only writers with an English Wikipedia page. This removes a large part of the original set (61%) and may introduce an important language and cultural bias. Many Transnational writers may have pages only in their native language, and these writers are excluded from the extraction pipeline. This limitation should be quantified and discussed in more depth. If possible, discuss whether multilingual Wikipedia extraction could reduce this bias.
5- The definition of "Transnational" and the operationalization of underrepresentation need more analysis. The classification depends on several things (former-colony status, HDI, passport mobility, ethnic minority status, and so on). However, the paper does not analyze how much each parameter affects the final classification. A sensitivity analysis would be useful to show whether the results are stable when these parameters change. The criterion about belonging to an ethnic minority in a Western country also needs clarification: which Wikidata properties are used, and how are missing or noisy ethnicity data handled?
6- The entity linking pipeline should be justified in more detail. The paper says that the pipeline combines neural methods and heuristics, and it describes the main steps of the process, including PERSON detection, Wikipedia API search, string-similarity filtering, and Wikidata ID retrieval. However, the choices behind this pipeline are not sufficiently justified or compared with alternatives. Since the extraction is based on Wikipedia pages, the authors should also justify why they used Wikipedia API search and string similarity instead of comparing with existing entity linking tools such as DBpedia Spotlight, which is built using resources derived from Wikipedia and its entities can often be mapped to Wikidata entities. DBpedia Spotlight could be considered as a baseline or alternative for the entity-linking step, especially for future multilingual extensions, as there are versions trained in different languages (https://demo.dbpedia-spotlight.org/). This would not solve the full multilingual extraction problem, but it could help with the entity-linking part.
7- The analysis of results would benefit from more qualitative validation of the augmented triples and their usefulness. The quantitative increase in triples is large, but it is not always clear how meaningful the new triples are. Although the paper includes some qualitative examples, more systematic validation would be useful. For example, the authors could analyze whether some extracted relations correspond to knowledge that previously existed in Wikidata history, whether they represent genuinely missing knowledge, or whether they are relations outside the current Wikidata taxonomy. This would help assess the practical value of the knowledge graph augmentation.
8- The representational bias analysis needs clearer explanation of normalization. In Table 5, it is not clear whether the most frequent predicates are ranked by raw counts or normalized by the size of each socio-demographic group. Since the groups have very different sizes, raw counts may not reflect the actual representativity of each predicate within each group. Table 6 partially addresses this through Jensen-Shannon Divergence, but the paper should make the methodology clearer.
9- The external resources and code need substantial improvement. The paper presents many experiments and many derived datasets (subsets of writers, training and test configurations, extracted triples, socio-demographic labels, manual evaluation samples, and network analysis data). However, the linked GitHub repository does not appear to contain all the resources, code, intermediate datasets, and outputs needed to reproduce the paper. This is a major reproducibility issue. The authors should make available all resources and scripts used in the different sections of the paper, or clearly explain how they can be regenerated. At minimum, the repository should include the exact data splits, writer lists and labels, the extracted triples, manual evaluation data, scripts for producing the reported tables, configuration files, dependency versions, and clear instructions.
10- The related work section should better position the contribution of the paper. At the moment, it mostly lists related papers one after another. It would be helpful to include a clearer comparison showing what each related work studies, which type of bias it addresses, which dataset it uses, and how the proposed work differs from it. A comparison table would improve readability and make the contribution clearer.
11- The paper would benefit from stronger comparison with existing methods in the literature. Many of the reported results are computed using the authors’ own pipeline, datasets, subsets, and experimental variants. This is understandable for a framework paper, but some components could still be compared with existing solutions or baselines. For example, the entity linking step could be compared with established entity linking systems.

List of minor issues:
- Page 2, line 2: The word "demonstrated" may suggest mathematical proof. A softer wording such as "shown", "reported", or "provided evidence" would be better.
- Page 2, line 24: "from all the English Wikidata pages" -> "from all the English Wikipedia pages".
- Page 3, lines 8–10: The related work discussion does not mention the relevant results of the cited work. Please add the main finding or explain why the cited work is relevant.
- Related work section: Citation style is inconsistent. Sometimes the text uses "Author et al. [34]", while in other places the sentence starts directly with "[34]". Please make this consistent.
- Page 4, lines 36–37: The sentence "They were not born before..." is hard to read. Consider rewriting as "They were born after..." or "They were born in or after...".
- Page 4, lines 49–51: URLs should include the date of last access.
- Section 3.3: At the time of review, the SPARQL endpoint (https://kgccc.di.unito.it/sparql/wikibias) was not available for testing and returned "Service Unavailable".
- Section 4: Please clarify the role of the development set.
- Section 5.2, page 12, line 21: "To do si" -> "To do so".
- Section 5.3: More detail is needed on how the "knowledge graph based on Wikidata relationships" was extracted.
- Page 14, line 50: "connections within a each group" -> "connections within each group".
- Table 8: To facilitate comparison, all numbers should be written using the same scientific notation format.
- Conclusion, page 16, lines 46–49: The sentence beginning "Augmenting the triples about them..." seems to lack a connector, please check it.

Overall, I found the paper very interesting and with promising results. The main problems I see are: (1) the framework as an artifact should be the central part of the paper; (2) the methods, algorithms, and approaches used in each experiment should be better justified, detailed, and compared with existing solutions where possible; and (3) reproducibility should be substantially improved, as the GitHub repository lacks most of the resources and code needed to generate the reported results.

This paper presents WikiBias, a framework that combines Semantic Web technologies and Information Extraction to analyze representational and allocative bias in Wikidata and Wikipedia. The paper addresses a relevant research problem, proposing a multidisciplinary framework that integrates ontology engineering, knowledge graphs, biographical event extraction, and network analysis.

I believe this is a valuable contribution to the Semantic Web and AI communities, particularly given the increasing use of Wikipedia and Wikidata as knowledge sources for AI systems. One aspect that I appreciated is that the paper goes beyond simply identifying bias in existing knowledge bases. Instead, it investigates how automatically enriching a knowledge graph through information extraction affects both the representation of different social groups and the structural properties of the resulting graph.

However, I also identified some issues related to clarity and presentation that should be addressed before possible publication.

One aspect that I found slightly confusing is that the distinction between representational bias and allocative bias is introduced only in the Related Work section. Throughout both the abstract and the introduction, the manuscript instead refers more generally to "different types of bias" or "multiple forms of bias" while discussing underrepresentation and stereotypical representations.

Since representational and allocative bias are precisely the two biases investigated throughout the paper, I believe this terminology should be introduced much earlier, preferably already in the abstract and introduction. Doing so would clarify the scope of the paper from the beginning, improve the conceptual consistency of the manuscript, and better motivate the proposed framework.

There is little inconsistency between the introduction and the methodology regarding the source of the extracted biographical triples. In RQ1, the manuscript states that the triples are extracted from “English Wikidata pages”, but the methodology clearly indicates that the extraction is actually performed from English Wikipedia pages. This wording should be revised throughout the manuscript to accurately reflect the data source and avoid unnecessary ambiguity.

Moreover, I believe the wording of RQ3 could be better aligned with the performed evaluation. The research question asks whether biographical triple extraction facilitates the discovery of minorities, while the evaluation primarily analyzes changes in network connectivity after knowledge augmentation. Although increased connectivity may indeed contribute to discoverability, these concepts are not equivalent, and the manuscript does not explicitly establish this relationship. I suggest clarifying why improved connectivity can reasonably be interpreted as improved discoverability, or rephrasing the research question so that it more accurately reflects the conducted analysis.

Since I mentioned the Related work section, I noted that it covers a broad range of relevant studies and is well organized around the distinction between representational and allocative bias. However, the discussion is predominantly descriptive. Most paragraphs follow a “study–method–result” pattern, summarizing individual papers rather than critically discussing them. I encourage the authors to strengthen this section by explicitly discussing:
- how previous studies relate to one another;
- the limitations that remain;
- why representational and allocative bias have generally been studied separately;
- how the proposed framework addresses this gap.

Currently, the novelty of the framework only becomes fully apparent in the final paragraph of the section. Building this argument progressively would make the contribution considerably stronger. This is particularly important because several cited works are from the authors themselves, and explicitly contrasting the present work with those previous studies would better highlight the actual advancement proposed in this paper.

Going to the proposal, the proposed PiM ontology is one of the interesting aspects of the paper. However, I believe the semantic modeling would benefit from additional clarification.

In the marriage example, the biographical situation is connected through dul:isSettingFor not only to the participating persons but also directly to the corresponding role instances (pim:husband and pim:wife). However, this modeling pattern is not followed in the subsequent migration example, where the situation is connected only to the participating person, while the roles are associated exclusively through dul:hasRole.

It is therefore unclear whether the direct association between a dul:Situation and dul:Role instances is an intentional modeling decision or a simplification of the examples.

Then, I encourage the authors to clarify the intended modeling pattern and explain how it aligns with the semantics of DUL.

Moreover, I believe the ontology presentation could also be strengthened with a simple conceptual diagram illustrating the main classes and relationships. While the Turtle snippets are useful examples, they do not fully communicate the ontology design.

Turning to the evaluation, although the reported differences are interesting, the experimental evaluation remains largely descriptive. I encourage the authors to complement the analysis with appropriate statistical tests or confidence intervals whenever applicable. This would provide stronger evidence that the observed differences between socio-demographic groups are not merely descriptive but statistically meaningful.

Besides, the evaluation convincingly demonstrates a substantial increase in the number of extracted biographical triples. But, the experiments primarily quantify the amount of new knowledge rather than its quality. A discussion of the correctness or usefulness of the newly added relations or a manual validation of a representative sample, would strengthen the conclusions regarding the benefits of the knowledge augmentation.

Another point that worths attention is about the generalization of the proposed solution. Since the framework is presented as a general solution for analyzing bias in Wikimedia projects, it would be interesting to discuss more explicitly the extent to which the observed findings may generalize beyond the domain of writers.

Finally, although the network analysis provides interesting insights into changes in graph connectivity, some interpretations, particularly regarding the discoverability of minority groups, go beyond what is directly measured by the reported network metrics. Discussing these limitations more explicitly would further strengthen the experimental section.

Regarding reproducibility, I appreciate the authors' effort to make the implementation publicly available. But, some of the files are hosted in Google Drive. As a consequence, the repository does not currently provide a completely self-contained package for reproducing the experiments. Although this isn't an issue as long as the files remain available, it would be beneficial to host everything directly on GitHub. Furthemore, I recommend that the authors improve the README with clearer instructions describing the complete reproduction pipeline and provide the scripts necessary to reproduce the reported tables and figures directly from the available data.

Finally, the manuscript is generally well-written and easy to follow. I only noticed a few minor typos:

"In line with with our research aims..." (Page 6) -> remove one with.

"other named entity" (Page 7) -> entities.

"Western Women are negatively affected by the augmentation, loosing..." (Page 10) -> women, losing.

"The average number of triples per writers"(Page 10) -> per writer.

"The gender axis appears to be the one that benefits the less" (Page 11) -> the least.

"within a each group" (Page 14) -> remove a.

This article proposes a framework for exploring biases in the Wikimedia ecosystem. The proposed approach, which combined both semantic web technologies and information extraction techniques, is used to detect and mitigate bias, in terms of gender and origin, via a case study of writers in Wikidata and in English Wikipedia.

The work is most certainly of interest to the Semantic Web community, the paper is well written, and the results are very informative. Nonetheless the paper suffers from several weaknesses in terms of clarity and potential impact.

One major limitation relates to the fact that the framework is more of a framing for the proposed case study and does not constitute a clearly defined template that can serve as the basis for further studies in relation to bias. In order to improve the significance and potential impact of the work, the proposed framework should be introduced in a dedicated section after the related work and should clearly show the benefit of combining semantic web technologies and information extraction techniques when it comes to detecting and mitigating bias in a Wikimedia context.

Additionally, the proposed approaches are not described in sufficient detail thus making it difficult for other researchers to reproduce or built upon the existing study. Both the “People in the Media Ontology” and the “Event Extraction Pipeline” need to be described in detail. In the case of the former, the ontology design methodology should be clarified, the classes and properties should be described in detail (ideally with the help of one or more visualisations), and the modeling choices should be further elaborated. In the case of the latter, it would be helpful to visualise the pipeline and to describe each of the steps in detail and to provide further insights as to the adopted approaches, resources, and design decisions.

The research questions should be updated in order to place the emphasis on “how”, “why”, or to “what extent” as opposed to simple binary questions that limit the scope of the analysis and the value of the proposed framework. The core contributions should also be enumerated in the introduction before the paper summary.

The related work section could be improved by including a gap analysis before introducing the authors own work. A structured analysis of existing approaches with the help of a tabular overview would be particularly helpful. In order to improve readability, It would be better to consistently include the author names when a reference is used at the start of a sentence.

Other comments:
Rather than redefine the meaning of “Global North/Global South” it would be best to give your approach another name such that it can be easily distinguished and reused.

All prefixes should be defined, when the various ontologies and vocabularies are introduced, before they are used in the examples.

It would be helpful to also include some sample results for any queries specified in the paper. See Figure 1.

“bias can be propagated by models behavior” -> bias can be propagated by the models behavior …

“Works on allocative bias are mainly devoted to explore” -> Works on allocative bias are mainly devoted to exploring …

“To study the nature of the representational bias across group induced by the augmentations” -> To study the nature of the representational bias across groups induced by the augmentations,

“To do si, we adopted a pairwise” -> To do so, we adopted a pairwise ??

“if more than one links between the same pair of entities, their relationship is represented by a” -> if more than one link between the same pair of entities, their relationship is represented by a

“Transnational men writers” -> Transnational male writers