Large Language Models

Given the enormous impact that generative artificial intelligence has had on our lives and on our planet, it is remarkable how little we understand of how these systems do what they do. What do LLMs do when they communicate with humans? On the surface their linguistic capabilities are human-like, but can we find a way of describing more accurately what is going on when they use language? And, importantly, we need also ask the question the other way around: What do humans do when they communicate with AI? How do we anthropomophise them, and how does that affect our relationship with this new technology?

Publications

2026

When Models Decide and When They Bind: A Two-Stage Computation for Multiple-Choice Question-Answering

H. M. Wong, Rick Nouwen, and Albert Gatt

In Findings of the Association for Computational Linguistics (ACL Findings), 2026

Abs arXiv

Multiple-choice question answering (MCQA) is easy to evaluate but adds a meta-task: models must both solve the problem and output the symbol that *represents* the answer, conflating reasoning errors with symbol-binding failures. We study how language models implement MCQA internally using representational analyses (PCA, linear probes) as well as causal interventions. We find that option-boundary (newline) residual states often contain strong linearly decodable signals related to per-option correctness. Winner-identity probing reveals a two-stage progression: the winning *content position* becomes decodable immediately after the final option is processed, while the *output symbol* is represented closer to the answer emission position. Tests under symbol and content permutations support a two-stage mechanism in which models first select a winner in content space and then bind or route that winner to the appropriate symbol to emit.

2025

Can LLMs Detect Ambiguous Plural Reference? An Analysis of Split-Antecedent and Mereological Reference

A. Dang, Rick Nouwen, and Massimo Poesio

In Proceedings of BlackboxNLP, 2025

arXiv
VAQUUM: Are vague quantifiers grounded in visual data?

H. M. Wong, Rick Nouwen, and Albert Gatt

In Findings of the Association for Computational Linguistics (ACL Findings), 2025

arXiv