Official Review · ICLR 2026

This paper introduces MAFE (Multi-Agent Fair Environments), a benchmarking suite designed to evaluate fairness-aware policies in multi-agent reinforcement learning settings. It provides three domain-specific environments: MAFE-Loan (financial lending), MAFE-Health (healthcare), and MAFE-Edu (education), and demonstrates their utility through experiments using an evolutionary strategy adapted for fairness-aware optimization. 1. I acknowledge that there is currently a gap in the community for more sophisticated benchmarks for measuring fairness in dynamic environments, and this paper addresses that need. 2. The modeling details, particularly those based on real data, represent substantial extensions over existing benchmarks such as D’Amour et al. 1. The proposed algorithm is a straightforward adaptation of the cross-entropy method, and the environments are not benchmarked against existing fairness-aware RL algorithms cited by the authors. While algorithmic novelty is not essential for a benchmark paper, I believe the environments should still be evaluated using state-of-the-art fairness-aware RL algorithms to better demonstrate their utility. However, such comparison is completely missing from the paper and the environments are benchmarks on two standard MARL algorithms. 2. From my understanding, all environments assume cooperation among agents. This is a major limitation, as such full cooperation is rarely the case in reality. Since the paper’s goal is to provide realistic benchmark environments, I see this as a significant issue. In addition, it would be valuable for these environments to include a single-agent variant, where other agents are controlled by the environment through logic or heuristics. This would increase the impact of the work by allowing the benchmark to be used for both single-agent and multi-agent research. 3. The reported runs using an MLP, 100 episodes with 400 steps each (a total of 40k steps), take 2–3 days to complete. This seems surprisingly long, given that the simulations are based on relatively simple mathematical equations (with no rendering or complex physics, unlike many existing benchmarks). This could significantly limit the benchmark’s practicality and adoption. Most RL algorithms typically require 500k–1M steps for training, which would take more than a month at this rate!! The authors should optimize the runtime or provide simplified versions that can be completed in less than a day using an MLP model, if their goal is to have their benchmark be used by the community. 4. The benchmark imposes certain unnecessary constraints on the algorithm and policy. For example, it requires the policy to be permutation equivariant. Such requirements add unnecessary complexity and reduce the benchmark’s practical usability. A good benchmark should be model agnostic and should not impose specific requirements like this on the algorithm. **Minor:** Many of the citations show the authors twice, like Atwood et al. Atwood e