Beyond Reward Scoring: Online Multi-Agent Preference Learning for Multimodal Response Generation

Multimodal preference alignment often follows a score-based RLHF pipeline: a reward model is first trained to learn human preferences, then used to score on-policy responses and optimize the response model's generation. This turns preference supervision into a scalar signal for generation: a model m