A Multi-Agent Reinforcement Learning Approach to Promote Cooperation in Evolutionary Games on Networks with Environmental Feedback

A prominent feature of biological organization in many species of social animals is the ability to achieve cooperation. However, despite its predominance in natural evolution, cooperative behaviors come at a cost, typically in the form of do ut des mechanisms (e.g., reciprocal altruism in vampire bats) with given thresholds for sharing resources or communication efforts. In this paper, we investigate the conditions of cooperation through the evolutionary dynamics of the prisoner's dilemma (PD) game as well as the learning dynamics resulting from the corresponding multi-agent reinforcement learning (MARL) model. In both cases, the interactions in the population are captured by a regular network and the impact of the players' actions is reflected through the evolution of an environmental resource, which also acts as a feedback on the dynamics. The following is a list of contributions: i) we provide a full characterization of the stability properties of the networked feedback-evolving PD game; ii) we determine a set of threshold values below which cooperation is promoted; iii) we develop the corresponding cross-learning model, which is a stateless MARL model, and we show that this model is equivalent to the networked PD game with environmental feedback.