An ontology for maintenance activities and its application to data quality

Tracking #: 3209-4423

Authors: 
Caitlin Woods
Matt Selway
Tyler Bikaun
Markus Stumptner
Melinda Hodkiewicz

Responsible editor: 
Guest Editors SW for Industrial Engineering 2022

Submission type: 
Full Paper
Abstract: 
Maintenance of assets is a multi-million dollar cost each year for asset intensive organisations in the defence, manufacturing, resource and infrastructure sectors. These costs are tracked though maintenance work order (MWO) records. MWO records contain structured data for dates, costs, and asset identification and unstructured text describing the work required, for example ‘replace leaking pump’. Our focus in this paper is on data quality for maintenance activity terms in MWO records (e.g. replace, repair, adjust and inspect). We present two contributions in this paper. First, we propose a reference ontology for maintenance activity terms. We use natural language processing and a manual clustering exercise to identify seven core maintenance activity terms and their synonyms from 21,088 MWOs. We provide elucidations for these seven terms. Second, we demonstrate use of the reference ontology in an application-level ontology using an industrial use case. The end-to-end NLP-ontology pipeline identifies data quality issues with 55% of the MWO records for a centrifugal pump over 8 years. For the 33% of records where a verb was not provided in the unstructured text, the ontology can infer a relevant activity class. The selection of the maintenance activity terms is informed by the ISO 14224 and ISO 15926-4 standards and conforms to ISO/IEC 21838-2 Basic Formal Ontology (BFO). The reference and application ontologies presented here provide an example for how industrial organisations can augment their maintenance work management processes with ontological workflows to improve data quality.
Full PDF Version: 
Tags: 
Reviewed

Decision/Status: 
Minor Revision

Solicited Reviews:
Click to Expand/Collapse
Review #1
Anonymous submitted on 19/Sep/2022
Suggestion:
Minor Revision
Review Comment:

I acknowledge the author’s modifications, where most of my comments (mainly the Evaluation section that I find now accomplished) were taken care of.
However, there some few points where I do not completely agree with the author’s replies
• Concerning the manual clustering of terms, the authors claim that giving the full NLP pipeline they have used is out of the scope of the journal. This is maybe true, but this does not avoid that a manual work (even with the help of an expert in the field) remains always subjective.
• Concerning the subtilities between “elucidation” and “definition”, it is needed to retain that natural language is always informal, and that misinterpretations are always possible (Table 7 still indicates “definitions and axioms for maintenance activity terms”, even if formal axioms are not present)
• Finally, I still have concerns regarding the usability of the approach in other industrial context (that was the idea behind Comment 1.10 concerning the real industrial use). You have worked on MWO that are associated, apparently, to a variety of fixed and mobile plant assets, according to you answer to Comment 3.2. Experimentation was done on a very specific case (pumps) and I am still wondering if the reference maintenance activities that you have retained are general enough to cover any other industrial situation.
Concerning the form of the document, there are still two tiny things
• Competency questions have been reformulated as real questions in section 5.1, but not in section 7
• Concerning the styles of section titles and subtitles, sometimes all (important) words appear with a capital first letter, and sometimes all the words are written in lower case :-D

Review #2
By Stefano Borgo submitted on 24/Sep/2022
Suggestion:
Minor Revision
Review Comment:

Thank you for the revision. The paper is much better now.

It is a pity that the editors did not distributed the pdf file with the typos and it is even worse that the authors did not care about asking the editors. As a conclusion, Table 5 has at least 4 errors (even after the “careful” rereading by the authors) and others remains in the paper. New typos below.

At the end of Sect. 2 there are questions (like “Does the activity change the function and/or capability of the item?”) which are ambiguous: it is unclear whether one should answer according to the state of the item in its ideal conditions (a pump) or in its actual conditions (a pump which is not working).

Pg. 10: “Given the inconsistencies in the existing standards presented in Section 2.6, we decided to extract reference-level activity terms from real-world MWO records.”
This means that a specific set of data was used. I suggest to add a note to explain why this choice gives a good coverage of the domain or at least to clarify what turns out to be actually covered.

Other typos (especially in the new section):

p26r5
The presence and magnitude of parts and labour costs are often used to make inference[s] about corrective work.

p.26
parts cost [perhaps "part cost"?]

p.26
filters, belts[,] etc.)

p.26
Each competency question is addressed in turn in the text[next] section below.

p.26
The 5th. column [5th]

p.26
The 6th. column [6th]

Inconsistent capitalisation in fig 1

p. 13 (more than once)
- p is is prescribed [only one is]

Review #3
Anonymous submitted on 04/Oct/2022
Suggestion:
Minor Revision
Review Comment:

I would like to thank the authors for the clarifying answers to the comments and improvements to the paper. If there are any future iterations for this paper, I would highly recommend authors to mark the new content in a different colour in the manuscript, so it makes it easier for the reviewers.

I am satisfied with the answers to my comments. I would recommend the reasoning in answers to Comment 3.3 and Comment 3.4 to be integrated to the paper so that any reader is aware about that reasoning.

Comment 3.9, while I agree with the authors that their argument that this ontology is just a proof of concept, it is claimed as one of the two major contributions of this paper. As such, I believe it won't hurt to have some better documentation and adhere to the ontology publishing best practices and guidelines.

After the first revision, I still have some concerns about the evaluation. With the pitfall scanning, validating competency questions, and confusion matrices, the authors have a reasonable amount of evaluation performed on this work. However, my concern is that it is not presented in a systematic manner. I would recommend starting the evaluation section with a plan or a set of objectives stating (a) what are you doing to evaluate, (b) how are you going to do that, and (c) what are the hypotheses that are being evaluated. In that way, a reader or a reviewer can clearly follow and decide if the evaluation is correct and adequate for the set of contributions you claim in the paper and also confirm that those tasks were performed with a proper evaluation plan in mind rather than in an ad-hoc manner. I believe this is important for a scientific publication.

I would also recommend moving Section 7.1 to the evaluation (Section 6).