• Can machine learning be used to help with the development of effective economic policy?
  • Can we understand economic behavior through granular, economic data sets?
  • Can we automate economic transactions for individuals?
  • Can we build rich and faithful simulations of economic systems with strategic agents?


UPCOMING: Noon EST, 9 am PST. March 9, 2021

Thore Graepel


Automatic Curricula in Deep Multi-Agent Reinforcement Learning

Multi-agent systems are emerging as a crucial element in our pursuit of designing and building intelligent systems. In order to succeed in the real world artificial agents must be able to cooperate, communicate, and reason about other agents’ beliefs, intentions and behaviours. Furthermore, as system designers we need to think about composing intelligent systems from intelligent subsystems, a multi-agent approach inspired by the observation that intelligent agents like organisations or governments are composed of other agents. Last but not least, as a product of evolution intelligence did not emerge in isolation, but as a group phenomenon. Hence, it seems plausible that learning agents require interaction with other agents to develop intelligence. In this talk, I will discuss the exciting role that deep multi-agent reinforcement learning can play in the design and training of intelligent agents. In particular, training RL agents in interaction with each other can lead to the emergence of an automatic learning curriculum: From the perspective of each learning agent, the evolving behaviours of the other learning agents constitute a challenging environment dynamics and pose ever evolving tasks. I will present three case studies of deep multi-agent RL with auto-curricula: i) Learning to play board games at master level with AlphaZero, ii) Learning to play the game of Capture-The-Flag in 3d environments, and iii) Learning to cooperate in social dilemmas.

Thore Graepel works as a research group lead at Google DeepMind and holds a part-time position as Chair of Machine Learning at University College London. In support of responsible innovation in artificial intelligence, Thore also serves as a Member of the Board of Directors at Partnership on AI. Thore studied physics at the University of Hamburg, Imperial College London, and Technical University of Berlin, where he also obtained his PhD in machine learning in 2001. After holding post-doctoral positions at ETH Zurich and Royal Holloway College, University of London, Thore joined Microsoft Research in Cambridge in 2003, where he co-founded the Online Services and Advertising group. Major applications of Thore’s work include Xbox Live’s TrueSkill system for ranking and matchmaking and the AdPredictor framework for click-through rate prediction in Bing. Furthermore, Thore’s work on the predictability of private attributes from digital records of human behaviour has been the subject of intense discussion among privacy experts and the general public. At DeepMind, Thore has returned to his original passion of understanding and creating intelligent systems, and recently contributed to creating AlphaGo, the first computer program to defeat a human professional player in the full-sized game of Go.


Date Speaker Title
Oct 6, 2020 Doyne Farmer
Global Microeconomics
Nov 10, 2020 Amy Greenwald
Learning Equilibria in Simulation-Based Games ... and the Ensuing Empirical Design of Mechanisms
Dec 15, 2020 Lihong Li
Estimating Long-term Rewards by Off-policy Reinforcement Learning
Feb 2, 2021 Fei Fang
Game Theory and Machine Learning for Multiagent Communication and Coordination
Video  -  Notes (thanks Yaman Habip!)
March 9, 2021 Thore Graepel
(DeepMind, UCL)
Automatic Curricula in Deep Multi-Agent Reinforcement Learning


Machine learning offers enormous potential to transform our understanding of economics, economic decision making, and public policy. Yet its adoption by economists, social scientists, and policymakers remains nascent.

This seminar series will highlight both the opportunities as well as the barriers to the adoption of ML in economics. In particular, we aim to accelerate the use of machine learning to rapidly develop, test, and deploy effective economic policies that are grounded in representative data.

This seminar series will expose some of the critical socio-economic issues that stand to benefit from applying machine learning, expose underexplored economic datasets and simulations, and identify machine learning research directions that would have a significant positive socio-economic impact. This includes policies and mechanisms that target socio-economic issues such as diversity and fair representation in economic outcomes, economic equality, and improving economic opportunity.


  • Inequality and social mobility
  • Sustainability
  • Innovation + entrepreneurship
  • Market design (e.g., labor, capital, consumer-facing)
  • Taxation
  • Behavioral economics
  • Game theory
  • Data-driven policy-making, and collecting representative and robust economic datasets


  • Reinforcement learning: multi-agent RL, cooperation, social dilemmas, principal-agent problems, equilibria and solution concepts.
  • Inverse reinforcement learning
  • Transfer from simulation to the real world
  • Multi-objective and constrained optimization
  • Causal inference
  • Explainability
  • Ethical issues: addressing bias in economic data, learning equitable policies, privacy-preserving learning.


David C. Parkes


Alex Trott


Stephan Zheng