OrionMoE-v1.0: A Mixture-of-Experts Foundation Model for Tabular In-Context Learning

Tabular foundation models enable in-context learning by predicting unseen observations from a labeled support table without task-specific parameter updates. Despite recent advances, most existing approaches rely on dense architectures that apply the same transformations across features and datasets.