Communication and Language

Contributors:

• Shuwen Qiu
• Guangyuan Jiang

Reading list

survey/review/perspective paper

Required

• Origins of Human Communication (Chapter 1), MIT Press (2010)
• Iconicity: From sign to system in human communication and language, Pragmatics & Cognition 2015
• Multi-Agent Cooperation and the Emergence of (Natural) Language, ICLR 2017
• Predicting Pragmatic Reasoning in Language Games, Science 2012
• Grounded language acquisition through the eyes and ears of a single child, Science 2024

Optional - Communication

• From communication to language-a psychological perspective, Cognition 1974
• Twelve-month-olds communicate helpfully and appropriately for knowledgeable and ignorant partners, Cognition 2008
• Communication about absent entities in great apes and human infants, Cognition 2015
• The interaction engine: cuteness selection and the evolution of the interactional base for language, Philos. Trans. R. Soc. B 2022
• Animal Communication in Linguistic and Cognitive Perspective, Annual Review of Linguistics 2023
• Social Learning in Animals: The Roots of Culture, Academic Press (1996)

Optional - Pragmatics

• Word Meaning in Minds and Machines, PsychReview 2021
• Meaning in Language: An Introduction to Semantics and Pragmatics, Oxford Handbook in Linguistics (2010)
• The Rational Speech Act Framework, Annual Review of Linguistics 2023
• Pragmatics and Discourse, Speech, language, and communication (1995)
• Pragmatic Language Interpretation as Probabilistic Inference, TiCS 2016
• Colors in Context: A Pragmatic Neural Model for Grounded Language Understanding, TACL 2017

Optional - Implicature, Metaphpor, and Humor

• Implicature
• Formalizing the Pragmatics of Metaphor Understanding, CogSci 2014
• The Funny Thing About Incongruity: A Computational Model of Humor in Puns, CogSci 2013
• Knowledge and Implicature: Modeling Language Understanding as Social Cognition, TopiCS 2013

Optional - Gaze

• Inferring Shared Attention in Social Scene Videos, CVPR 2018
• Understanding Human Gaze Communication by Spatio-Temporal Graph Reasoning, ICCV 2019
• Gaze360: Physically Unconstrained Gaze Estimation in the Wild, ICCV 2019
• The eyes have it: the neuroethology, function and evolution of social gaze, Neuroscience and Biobehavioral Review 2000
• Social Eye Gaze in Human-robot Interaction: A Review, JHRI 2017

Optional - Pointing Perception

• The origin of pointing: Evidence for the touch hypothesis, Science Advances 2019
• Perspective determines the production and interpretation of pointing gestures, Psychonomic Bulletin & Review 2021
• The observer’s perspective determines which cues are used when interpreting pointing gestures, Journal of Experimental Psychology: Human Perception and Performance 2021

Optional - Pointing Meets Cognition

• A New Look at Infant Pointing, Child Development 2007
• Twelve-month-olds point to share attention and interest, Developmental Science 2004
• 12-and 18-month-olds point to provide information for others, Journal of Cognition and Development 2006
• Gesturing makes learning last, Cognition 2007
• Prelinguistic Infants, but Not Chimpanzees, Communicate About Absent Entities, Psychological Science 2009
• Infants’ Use of Shared Experience in Declarative Pointing, Developmental Science 2009
• Twelve-month-olds’ comprehension and production of pointing, British Journal of Developmental Psychology 2012
• Toddlers Prefer Adults as Informants: 2- and 3-Year-Olds’ Use of and Attention to Pointing Gestures From Peer and Adult Partners, Child Development 2021

Optional - Pointing Meets Language

• The relation between pointing and language development: A meta-analysis, Developmental Review 2010
• Gesture Paves the Way for Language Development, Psychological Science 2005
• Pointing: Where language, culture, and cognition meet, Psychology Press (2003)

Optional - Models for Nonverbal Communication

• Pragmatic Inference and Visual Abstraction Enable Contextual Flexibility During Visual Communication, Computational Brain & Behavior 2020
• Visual resemblance and interaction history jointly constrain pictorial meaning, Nature Communications 2023
• Individual vs. Joint Perception: a Pragmatic Model of Pointing as Communicative Smithian Helping, CogSci 2021
• What Is the Point? A Theory of Mind Model of Relevance, CogSci 2022
• Show or Tell? Teaching with language outperforms demonstration but only when context is shared, PsyArXiv 2021
• Learning Triadic Belief Dynamics in Nonverbal Communication from Videos, CVPR 2021
• YouRefIt: Embodied Reference Understanding with Language and Gesture, ICCV 2021
• Emergent Graphical Conventions in a Visual Communication Game, NeurIPS 2022

Optional - Evolution

• Culture: Copying, Compression, and Conventionality, Cognitive Science 2015
• Gesture - Visible Action as Utterance, Cambridge University Press 2015
• Primate Vocalization, Gesture, and the Evolution of Human Language, Current Anthropology 2008
• How Language Began: Gesture and Speech in Human Evolution, Cambridge University Press 2015
• From mouth to hand: Gesture, speech, and the evolution of right-handedness, PloS One 2003
• Cumulative cultural evolution in the laboratory: An experimental approach to the origins of structure in human language, PNAS 2008

Optional - Emergent Languages

• Iterated Learning: A Framework for the Emergence of Language, Artificial Life 2003
• Emergent Multi-Agent Communication in the Deep Learning Era, arXiv preprint arXiv:2006.02419
• Emergence of Grounded Compositional Language in Multi-Agent Populations, AAAI 2018
• Emergent Graphical Conventions in a Visual Communication Game, NeurIPS 2022
• A framework for the emergence and analysis of language in social learning agents, Nature Communications 2024

Optional - Pantomime

• How Pantomime Works: Implications for Theories of Language Origin, Frontiers in Communication 2019
• Pantomime and imitation in great apes, Interaction Studies 2018
• In support of the role of pantomime in language evolution, Journal of Language Evolution 2018
• Pantomime as a foundation for ritual and language, Studia Liturgica 2018
• Defining Pantomime for Language Evolution Research, Topoi 2018
• Demonstration and pantomime in the evolution of teaching and communication, Language & Communication 2021

Optional - Pedagogy

• Natural pedagogy, TiCS 2009
• Natural pedagogy as evolutionary adaptation, Philos. Trans. R. Soc. B 2011
• Preschoolers use pedagogical cues to guide radical reorganization of category knowledge, Cognition 2014
• The effects of information utility and teachers’ knowledge on evaluations of under-informative pedagogy across development, Cognition 2022

Essay

1. There exist multiple language models but for different tasks. Given the above literature and the discussion about the uniqueness of human communication, how to design tasks or evaluations to test language models are doing simple imitation or true communication?
2. If you are designing an embodied chat robot, what kinds of nonverbal expressions will you give to it? How and when will it combine/alternate verbal and nonverbal expressions based on our discussion?