argilla/OpenHermesPreferences

OpenHermesPreferences v0.1 🧙 Using LLMs to improve other LLMs, at scale! OpenHermesPreferences is a dataset of ~1 million AI preferences derived from teknium/OpenHermes-2.5. It combines responses from the source dataset with those from two other models, Mixtral-8x7B-Instruct-v0.1 and Nous-Hermes-2-Yi-34B, and uses PairRM as the preference model to score and rank the generations. The dataset can be used for training preference models or aligning language models through…