LLM Strategies in Dilemmas: Payoff Magnitude Drives Behaviour across Languages

Researchers are increasingly focused on how large language models (LLMs) behave when deployed as autonomous agents, particularly within complex interactive systems. Trung-Kiet Huynh, Dao-Sy Duy-Minh, and Thanh-Bang Cao, from the University of Science (HCMUS) and Ho Chi Minh City University of Technology (HCMUT), alongside Le et al., have investigated the influence of rewards and linguistic cues on LLM strategies when faced with repeated cooperation dilemmas. Their work utilises a modified Prisoner’s Dilemma to reveal how incentive strength and language affect decision-making, uncovering consistent patterns of behaviour across different models and languages. This research is significant because it provides a framework for auditing LLMs as strategic agents and highlights potential biases towards cooperation, offering crucial insights for the design of future multi-agent systems and governance policies.

Their work utilises a modified Prisoner’s Dilemma to reveal how incentive strength and language affect decision-making, uncovering consistent patterns of behaviour across different models and languages.

LLM Strategy in Repeated Prisoner’s Dilemma explores emergent

This work reveals consistent behavioural patterns across different models and languages, notably incentive-sensitive conditional strategies and discernible cross-linguistic divergence in decision-making. The study unveils that LLMs exhibit adaptable cooperative strategies responding to changes in potential costs and benefits, a crucial factor for real-world applications spanning recommendation systems, negotiation tools, and multi-agent assistants. By conceptualising behavioural intention as a decision rule mapping interaction histories to subsequent actions, the researchers were able to infer underlying strategies from observed behaviour. Furthermore, the research establishes a clear link between LLM behaviour and established findings in human behavioural economics, observing incentive and payoff-stakes sensitive cooperation and cross-linguistic divergence mirroring patterns seen in human interactions. This work opens exciting possibilities for designing safer, more coordinated, and ultimately more beneficial AI-driven social and economic systems.

Payoff-scaled Prisoner’s Dilemma probing LLM strategy reveals consistent,

Researchers engineered a system where the numerical values of payoffs in the Prisoner’s Dilemma were systematically varied, maintaining the underlying strategic structure while altering the stakes of cooperation. This payoff-scaling allowed the team to examine LLM sensitivity to incentive magnitude in a dyadic setting, providing a robust method for probing behavioural changes. The research harnessed FAIRGAME’s capabilities to define experimental conditions via JSON configuration files, specifying payoff structures, game duration, LLM backends, and languages. At runtime, FAIRGAME combined these configurations with language-specific prompt templates, simulating repeated normal-form games and logging round-by-round interaction trajectories.

Scientists generated synthetic repeated-game trajectories, mirroring techniques used by Han et al, 2011, to train intention classifiers and subsequently analyse LLM gameplay logs. This innovative method facilitated addressing two key questions: firstly, whether LLM agents systematically adjust cooperative behaviour with varying payoff stakes, and how this differs across models and languages; and secondly, whether LLM behavioural intentions can be reliably classified using supervised learning, and what systematic biases emerge. The study pioneered a comparative approach, echoing findings from human behavioural studies in repeated Prisoner’s Dilemma, such as those by Montero-Porras et al, 2022, which demonstrate incentive and framing-dependent cooperation, although direct behavioural equivalence was not claimed.

LLMs exhibit incentive-sensitive strategy and linguistic divergence

Across multiple models and languages, experiments revealed consistent behavioural patterns, notably incentive-sensitive conditional strategies and observable cross-linguistic divergence. Data shows that LLMs do not respond uniformly to changes in potential rewards, demonstrating a nuanced understanding of strategic interactions. Specifically, the supervised classifiers achieved high accuracy in identifying these canonical strategies within the LLM’s gameplay logs, providing a quantifiable measure of strategic intent. Measurements confirm that the team successfully inferred underlying behavioural strategies from observed actions, offering a new method for auditing LLMs as strategic agents.

The study utilised a 10-round horizon for repeated interactions, enabling direct exploration of payoff-scaling and multilingual effects. Tests prove that LLM agents systematically alter cooperative behaviour as payoff stakes vary, with notable differences observed across models and languages. The research recorded that human cooperation rates and strategy distributions also vary with incentives, culture, and framing, echoing these patterns in LLM behaviour. Scientists achieved a deeper understanding of how LLMs approach cooperation dilemmas, paving the way for more robust and predictable AI systems in complex, interactive environments. This research positions intention classification as a diagnostic tool, providing a foundation for future comparative and governance-oriented analyses.,.

LLM Behaviour Shifts with Incentives and Framing

This research demonstrates that LLM behaviour isn’t fixed, but dynamically responds to incentive magnitude, model architecture, and linguistic framing. Researchers observed systematic incentive sensitivity, with LLMs becoming more cooperative as stakes increase, and identified distinct cultural characteristics influenced by the language used during interaction. The study highlights the inadequacy of traditional safety audits, often performed in English under fixed conditions, as these fail to capture strategy shifts driven by incentives and linguistic context. Authors acknowledge a methodological limitation in their statistical approach, noting that a mixed-effects multinomial logistic regression would more fully account for the nested structure of their data. Future research could explore these dynamics across even more diverse linguistic environments and incentive structures, further refining our understanding of LLM strategic behaviour.