[CogSci21] Individual vs. Joint Perception: a Pragmatic Model of Pointing as Communicative Smithian Helping

Abstract

The simple gesture of pointing can greatly augment one’s abil-ity to comprehend states of the world based on observations. Ittriggers additional inferences relevant to one’s task at hand. Wemodel an agent’s update to its belief of the world based on in-dividual observations using a partially observable Markov de-cision process (POMDP), a mainstream artificial intelligence(AI) model of how to act rationally according to beliefs formedthroughobservation.Ontopofthat,wemodelpointingasa communicative act between agents who have a mutual un-derstanding that the pointed observation must be relevant andinterpretable. Our model measures “relevance” by defining aSmithianValueofInformation(SVI)astheutilityimprove-ment of the POMDP agent before and after receiving the point-ing. We model that agents calculate SVI by using the cognitivetheory of Smithian helping as a principle of coordinating sep-aratebeliefsforactionpredictionandactionevaluation.Wethen import SVI into rational speech act (RSA) as the utilityfunction of an utterance. These lead us to a pragmatic modelof pointing allowing for contextually flexible interpretations.We demonstrate the power of our Smithian pointing model byextending the Wumpus world, a classic AI task where a hunterhunts a monster with only partial observability of the world.We add another agent as a guide who can only help by markingan observation already perceived by the hunter with a point-ing or not, without providing new observations or offering anyinstrumental help. Our results show that this severely limitedand overloaded communication nevertheless significantly im-proves the hunters’ performance. The advantage of pointing isindeed due to a computation of relevance based on Smithianhelping, as it disappears completely when the task is too diffi-cult or too easy for the guide to help

Publication
In Proceedings of Annual Meeting of the Cognitive Science Society

Related