Authors - Shreya S. Partake, Reena S. Satpute Abstract - Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language processing, achieving human-level performance on many semantic and syntactic benchmarks. However, their competence in pragmatics—the study of how context shapes meaning—remains a critical and underexamined frontier. This paper presents a unified analysis of the “pragmatic gap” in LLMs, arguing that it stems from a fundamental distinction between the co-textual statistical patterns LLMs are trained on and the contextual world knowledge humans use for inference. We first establish a theoretical baseline by reviewing foundational linguistic concepts, including Grice’s maxims, implicature, presupposition, speech acts, and deixis. We then systematically evaluate LLM performance, contrasting successes in pattern-rich tasks like coreference resolution with systemic failures in tasks requiring novel inference, such as non-conventionalized indirect speech acts and irony. We analyze the development of new evaluation tools, particularly the Pragmatics Understanding Benchmark (PUB), which quantifies the persistent gap between model and human performance. Subsequently, we synthesize emerging technical solutions, including “thought-based” fine-tuning and the injection of Gricean principles into Retrieval-Augmented Generation (RAG) frameworks. Finally, we dissect the profound cognitive and philosophical implications of this gap, critically examining the debates on the Symbol Grounding Problem and Theory of Mind (ToM). We conclude that while LLMs can pass “literal” ToM tests, they fail “functional” ToM, revealing them to be sophisticated co-text manipulators rather than context-aware agents. We propose that future progress lies in developing a “machine pragmatics” based on probabilistic models rather than flawed anthropomorphic imitation.
Open Zoom