Abstract:
Contextualized or discourse aware commonsense inference [1] is the task of generating commonsense assertions (i.e.,
facts) from a given story, and a sentence from that story. (Here, we think of a story as a sequence of causally-related events and
descriptions of situations.) This task is hard, even for modern contextual language models. Some problems with the task are:
lack of controllability for topics of the inferred assertions; lack of commonsense knowledge during pre-training; and, possibly,
hallucinated or false assertions. The task’s goals are to make sure that (1) the generated assertions are plausible as commonsense;
and (2) to assure that they are appropriate to the particular context of the story.
We utilize a transformer model as a base inference engine to infer commonsense assertions from a sentence within the context
of a story. With our inference engine we address lack of controllability, lack of sufficient commonsense knowledge, and plausibility
of assertions through three techniques. We control the inference by introducing a new technique we call “hinting”. Hinting
is a kind of language model prompting [2], that utilizes both hard prompts (specific words) and soft prompts (virtual learnable
templates). This serves as a control signal to advise the language model “what to talk about”. Next, we establish a methodology
for performing joint inference with multiple commonsense knowledge bases. While in logic, joint inference is just a matter of a
conjunction of assertions, joint inference of commonsense requires more care, because it is imprecise and the level of generality
is more flexible. You want to be sure that the results “still make sense” for the context. To this end, we align the assertions in three
knowledge graphs (ConceptNet [3], ATOMIC2020 [4], and GLUCOSE [5]) with a story and a target sentence, and replace their
symbolic assertions with textual versions of them. This combination allows us to train a single model to perform joint inference
with multiple knowledge graphs.We show experimental results for the three knowledge graphs on joint inference. Our final contribution
is a GAN architecture that generates the contextualized commonsense inference from stories and scores the generated
assertions as to their plausibility through a discriminator. The result is an integrated system for contextual commonsense inference
in stories, that can controllably generate plausible commonsense assertions, and takes advantage of joint inference between
multiple commonsense knowledge bases.
