MoralBench: A Multi-Faceted Benchmark for Ethical and Safety Alignment in LLMs

We introduce MoralBench, a consolidated benchmark dataset for evaluating moral alignment, ethical reasoning, and socially sensitive dialogue generation in large language models (LLMs). MoralBench combines three datasets generated via a large language model: (1) a 1K high-fidelity safety-alignment prompt set, (2) a 10K abstracted variation set for domain transfer and low-specificity alignment testing, and (3) a 19,132-record rich metadata preference dataset with structured ethical annotations. This work provides both the dataset and an initial analysis of its coverage, metadata distribution, and cost of creation. We also discuss its unique potential for non-traditional applications such as multi-agent system (MAS) training and as a test suite for evaluating privacy protocols (e.g., rLDP). The MoralBench-1K dataset is particularly suited for verifying how MAS agents with different roles collaborate to refute harmful prompts, while the rich metadata of the MoralBench-19K dataset is key for measuring the efficacy of privacy protocols and for fine-tuning models via RLHF.