Semantic Referee: A Neural-Symbolic Framework for Enhancing Geospatial Semantic Segmentation

Tracking #: 2005-3218

This paper is currently under review
Marjan Alirezaie
Martin Längkvist
Michael Sioutis
Amy Loutfi

Responsible editor: 
Guest Editors Semantic Deep Learning 2018

Submission type: 
Full Paper
Recent machine learning algorithms have shown a considerable success in various computer vision tasks, including semantic segmentation. However, they seldom perform without error. A key aspect of discovering why the algorithm has failed is usually the task of the human who, using domain knowledge and contextual information, can discover systematic shortcomings in either the data or the algorithm. In this paper, we propose a symbolic-based technique, called semantic referee, which is able to extract qualitative features of the errors emerging from the machine learning framework and suggest corrections. The semantic referee relies on a spatial reasoning method applied on ontological knowledge in order to retrieve the features of the errors in terms of their spatial relations with their environment. The reasoner outputs a semantic augmentation for the errors that is then reported back to the learning algorithm to learn from its mistakes. In this paper, the proposed method of the interaction between a neural network classifier and a semantic referee shows how to improve the performance of semantic segmentation for satellite imagery data.
Full PDF Version: 
Under Review