Emergent Reciprocity Through Temporal Credit Assignment in Reinforcement Learning Agents

Le Thuy Duong Nguyen1,2, Dane Malenfant1,2, Blake Aaron Richards1-4

1McGill University, 2Mila - Quebec Artificial Intelligence Institute, 3The Neuro. Montreal Neurological Institute-Hospital

Manitokan Harvesting Environment

Manitokan Harvesting Environment - 8x8 grid reinforcement learning demonstration showing emergent reciprocal behavior in multi-agent systems

8 x 8

Manitokan Harvesting Environment - 15x15 grid reinforcement learning demonstration showing temporal credit assignment in multi-agent systems

15 x 15

Manitokan Harvesting Environment - 30x30 grid reinforcement learning demonstration showing emergent reciprocity through temporal credit assignment in multi-agent systems

30 x 30

Poster References

[1] R. S. Sutton, "Temporal credit assignment in reinforcement learning," University of Massachusetts Amherst, 1984.

[2] L. Barkwell, "Manitokanac," Gabriel Dumont Institute of Native Studies and Applied Research Virtual Métis Museum. [Online]. https://www.metismuseum.ca/resource.php/148154. [Accessed: Apr. 7, 2025].

[3] D. Malenfant and B. A. Richards, "The challenge of hidden gifts in multi-agent reinforcement learning," arXiv preprint arXiv:2505.20579, 2025.

[4] M. Chevalier-Boisvert, B. Dai, M. Towers, R. Perez-Vicente, L. Willems, S. Lahlou, S. Pal, P. S. Castro, and J. Terry, "Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks," in Advances in Neural Information Processing Systems, vol. 36, 2023, pp. 73383-73394.

[5] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, "Proximal policy optimization algorithms," arXiv preprint arXiv:1707.06347, 2017.

[6] C. Yu, A. Velu, E. Vinitsky, J. Gao, Y. Wang, A. Bayen, and Y. Wu, "The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games," in Advances in Neural Information Processing Systems, vol. 35, 2022, pp. 24611-24624.

Connect:

Le Thuy Duong Nguyen - NeuroAI Researcher at McGill University and Mila Quebec AI Institute

Le Thuy Duong Nguyen

(Thuy)