DFM Mimir v1: An Open HRM Delivering Frontier Performance at 1B Parameters Using Only Permissible Post-Training Data

DFM Mimir v1: An Open HRM Delivering Frontier Performance at 1B Parameters Using Only Permissible Post-Training Data Peter Schneider-Kamp1,2,∗ Jacob Nielsen1,2 , Gianluca Barmina1 , Kenneth Enevoldsen3 , Lukas Galke Poech1 ,

arXiv:2608.13517v1 [cs.CL] 13 Aug 2026

1

University of Southern Denmark

2

Ordbogen A/S

3

Aarhus University

Current large language model development relies on massive, often non-permissible datasets, creating a high barrier for researchers committed to open-source and ethically sourced data. We introduce Mimir v1, a 1-billion-parameter language model based on the Hierarchical Reasoning Model (HRM) architecture, that is trained from scratch and delivers highly competitive performance for English and sets a new state of the art for Danish using only permissible post-training data. Trained on a mixture of 161 datasets, Mimir v1 outperforms the original HRM-Text 1B and competes with larger frontier models like Qwen 3.5 4B and Gemma 4 E2B, tested across 20 benchmarks for English, Math & Code and Danish. The model is available on the Hugging Face Hub: https://huggingface.co/danish-foundation-models/DFM-Mimir

Date: 14 August 2026 Correspondence: Peter Schneider-Kamp via petersk@imada.sdu.dk

Figure 1: Aggregate results comparing DFM Mimir 1B against the HRM-Text 1B, Qwen 3.5 2B and Gemma 4 E2B, displaying highly competitive performance across 20 benchmarks.

1

1 Introduction In recent years, Large Language Models (LLMs) have catalyzed a paradigm shift in artificial intelligence, characterized by rapid iterations and significant advancements in emergent capabilities. However, as noted by Wang et al. [2026], current development is largely driven by a “monolithic recipe” consisting of massive, multi-stage pipelines and training on exorbitant volumes of data. This approach not only necessitates vast computational resources for pre-training but also creates a prohibitive entry barrier for the broader community of researchers and practitioners. For national initiatives such as the Danish Foundation Models project1 [Enevoldsen et al., 2023], which adheres to a philosophy of using exclusively permissible and, whenever possible, openly licensed data, training a capable LLM from scratch is often infeasible given the limited pool of high-quality data for a language such as Danish. Consequently, it has been challenging to provide fully permissible base models on LLM platforms to serve as foundations for post-training objectives. To address these constraints, we employ the HRM-Text framework, which enables focusing on post-training data during the initial training phase, thereby facilitating the creation of a viable base model for the wider community. In this technical report, we present Mimir v1, a 1-billionparameter hierarchical reasoning model (HRM) trained from scratch, utilizing the architecture proposed by Wang et al. [2026]. Mimir v1 is optimized for Danish and English tasks and has undergone instruction-tuning on a curated mixture of 161 datasets, comprising approximately 70.5 billion tokens per epoch. Furthermore, as certain datasets used in [Wang et al., 2026] do not align with DFM’s permissibility standards, we demonstrate the efficacy of synthetically generating “transplant datasets”, replacing non-permissible data with synthetically generated permissible variants. Our results indicate that these synthetic alternatives achieve comparable or superior performance without compromising data rights, further underscoring the relevance of the HRM approach for low-resource linguistic domains, empowering communities of practitioners and researchers with small, capable and fully permissible models, with both low training and inference requirements.

2 Datasets For training HRM-Mimir v1, we curated an extensive mix of data with different objectives, ranging from English and Danish instruction & knowledge to mathematics and agentic-style post-training data. Our mix draws from 161 datasets with almost all of them being freely available on the HuggingFace Hub. The corpus amounts to 70,479,308,606 tokens per epoch. The full list of datasets and their sources is listed in Appendix A. 2.1 Category Distribution We classify each dataset into one of eight functional categories based on its content and intended use. Table 1 provides the distribution. The three largest categories: Danish instruction & knowledge (22.07%), English instruction (19.26%), and selected datasets from the Sapient mixed collection (17.02%)2 , together account for over 58% of the corpus. Math & reasoning contributes a further 14.8%, bringing the combined share of the top four categories to 73%. The Danish instruction & knowledge category is the largest by token volume, driven primarily by lærebogen, a 1 2

https://www.foundationmodels.dk sapientinc/HRM-Text-data-io-cleaned-20260515

2

Table 1: Token share and dataset count by functional category. Category

Tokens/epoch

Share

Datasets

Avg. tokens/dataset

Danish instruction & knowledge English instruction Sapient mixed (Flan/Platypus) Math & reasoning Mimir synthetic Agentic & tool use Machine translation Science & summarization

15.56B 13.58B 12.00B 10.40B 7.05B 6.66B 3.50B 1.74B

22.07% 19.26% 17.02% 14.76% 10.00% 9.46% 4.96% 2.47%

30 16 71 10 16 8 5 5

518.6M 848.5M 169.0M 1,040.2M 440.4M 832.5M 699.0M 348.5M

Total

70.48B

100%

161

Danish instruction-following dataset contributing 8.32B tokens (11.8% of the corpus) at 4× repetition, alongside dfm-dyna-instruct (3.54B) and synquid wiki-instruct-da (0.99B). English instruction is anchored in Dolci [Team Olmo, 2025] (7.71B combined), Tulu 3 [Lambert et al., 2024] variants (1.57B), and Nemotron instruction-following (1.60B). The Sapient mixed category is dominated by a single large repository (11.92B, 16.9%), which bundles 107 sub-collections from Flan, Platypus, and tasksource. An additional 70 Sapient-synth transplant datasets contribute 75M tokens. These are synthetic recreations of English instruction tasks (Flan NIV2, Flan Dialog, Platypus, Tasksource) in generated-and-audited form, replacing original Sapient data that was non-compliant with the DFM philosophy. Math & reasoning is led by OpenMathInstruct-2 [Toshniwal et al., 2024] (6.60B, 9.4%), the third-largest single dataset overall, followed by AceReason1.1-SFT [Liu et al., 2025] (1.95B) and verifiable reasoning traces (0.68B). The remaining four categories – synthetic, agentic & tool use, machine translation, and science & summarization – together contribute 19.0B tokens (27%). 2.2 Language Distribution Table 2 shows the distribution of tokens by language. The corpus is predominantly English (68.5%), with Danish contributing 24.7% (6 out of 8 categories are entirely English) and bilingual Danish–English data a further 6.4%. A small fraction (0.2%) contains other bi-lingual translation data. The bilingual da+en category includes machine translation data and synthetic transformation datasets that pair Danish and English. Table 2: Token share by language. Language

Tokens/epoch

Share

48.36B 17.44B 4.61B 0.14B

68.62% 24.74% 6.54% 0.20%

English (en) Danish (da) Bilingual da+en other

3

Table 3: Token share by data form. Tokens/epoch Share

Reformatted

Curated + reformatted

Synthetic + audited

Tool-call formatted

Translated + audited

Agreement-supplied

Derived task

46.49B 65.96%

11.92B 16.91%

7.81B 11.08%

1.87B 2.65%

1.59B 2.26%

0.67B 0.95%

0.13B 0.18%

2.3 Data Processing Datasets enter the corpus in seven distinct forms, reflecting the pipeline’s processing stages. We report the forms in Table 3. Reformatted datasets are existing Hugging Face repositories simply converted into the training format – this represents the default pathway for public data. Curated + reformatted applies to the Sapient mega-repository, whose 107 sub-collections were selected as a curated subset before reformatting. Synthetic + audited data is LLM-generated using Gemma4 31B and quality-audited before inclusion, with acceptance rates ranging from single digit percentages to high nineties for different categories. This includes transformations from high-quality English and Danish text corpora into span-filling, denoising, reordering, and continuation tasks. Here, we employ Common Pile [Kandpal et al., 2025] for English text corpora and Danish Dynaword [Enevoldsen et al., 2025] for Danish ones. We similarly generated and audited the dfm8-synthetic-* instruction datasets and Sapient-synth transplants. Tool-call formatted data incorporates native tool-calling structure for agentic training. Translated + audited covers the OpenHermes-based data and DA/EN translations that were repaired and audited. A small corpus of agreement-supplied data comes from Danish Foundation Model agreements (DBC, Lex.dk), where licensing does not permit public sharing. Derived task data is derived from an existing dataset to create a new task formulation. The original Sapient training data3 consists mainly of Flan, Platypus, and tasksource subcollections, which are themselves dominated by multiple-choice classification tasks — pick the correct option from A/B/C/D. This is a natural consequence of their source benchmarks: Flan NIV2, Platypus (Reclor, SciBench), and tasksource (PragmEval, Reclor) are structured around categorical selection. In this work, we shift the balance away from multiple-choice toward free-form generation, presenting a substantially harder task for the model to achieve exact match accuracy. The majority of the corpus (83% of tokens) comes from outside the Sapient collection, and the dominant non-Sapient categories are generally and inherently more generative: • English instruction (13.58B, 19.3%): Dolci, Tulu 3, and Nemotron SFT mixtures are predominantly free-form instruction following. • Math & reasoning (10.40B, 14.8%): OpenMathInstruct-2, AceReason, and reasoning traces all require the model to produce a free-form numerical or symbolic answer that is scored by exact match, not by selecting from options. • Danish instruction & knowledge (15.56B, 22.1%): Laerebogen, dfm-dyna-instruct, and Danish QA/summarization datasets are open-ended generation tasks. Together these three categories account for 39.54B tokens (56.1% of the corpus), nearly all scored on exact accuracy or free-form generation rather than multiple-choice selection. Even 3

sapientinc/HRM-Text-data-io-cleaned-20260515

4

Table 4: Top 10 datasets by sampled tokens per epoch. Dataset sapientinc/HRM-Text-data-io-cleaned-20260515 danish-foundation-models/laerebogen nvidia/OpenMathInstruct-2 nvidia/Nemotron-SFT-Agentic-v2 danish-foundation-models/dfm-dyna-instruct allenai/Dolci-Instruct-SFT-No-Tools schneiderkamplab/opus-da-en-permissive allenai/Dolci-Instruct-SFT nvidia/AceReason-1.1-SFT allenai/big-reasoning-traces

Tokens/epoch

Share

11.92B 8.32B 6.60B 4.27B 3.54B 3.49B 2.90B 2.24B 1.95B 1.66B

16.91% 11.81% 9.37% 6.06% 5.03% 4.95% 4.12% 3.17% 2.76% 2.35%

within the Sapient-synth transplant datasets (70 datasets, 75M tokens), many original multiplechoice classification tasks were regenerated as open-ended generation or answer-generation formulation for example, ‘task590-amazonfood-summary-correction-classification‘ and ‘task870msmarco-answer-generation‘ ask the model to produce a free-form answer rather than select from candidates. This change of the training data composition means Mimir v1 is trained to generate answers rather than discriminate among options, aligning with evaluation suites that prioritise exact-match scoring (GSM8k, MATH, DROP) over multiple-choice accuracy (ARC-C, MMLU, Hellaswag). 2.4 Data Concentration and Repetition Table 4 shows the top 10 datasets by sampled tokens per epoch. The corpus is highly concentrated: these ten datasets account for 66.5% of all tokens, and the top three alone for 38.1%, while the remaining 151 datasets contribute 33.5%. Two single sources exceed 10% each: the Sapient mega-repository (16.9%) and lærebogen (11.8%). The Dolci family contributes 5.73B tokens across two instruction datasets, the largest English instruction contribution after the cleaned Sapient corpus, and NVIDIA sources account for 12.82B tokens across the reasoning, math, and agentic datasets in the top 10. This concentration is partly by construction: Several datasets are sampled more than once per epoch. Lærebogen is repeated 4×, inflating 2.08B base tokens to 8.32B and making it the second-largest entry in the corpus. Eight small Danish datasets are repeated 10× to ensure sufficient coverage despite limited source data, and Dolci-Instruct-SFTNo-Tools is doubled to increase its English instruction contribution. The most heavily repeated dataset, kaenguruen (20×), is negligible in volume at 638K tokens.

3 Architecture Mimir uses the Hierarchical Reasoning Model Text (HRM-Text) architecture with a hidden size of 1,536. The model has 12 attention heads per layer and a feed-forward expansion factor of 4. Hierarchical reasoning is configured with 2 H-cycles and 3 L-cycles, with truncated backpropagation limited to 5 steps and a warmup ratio of 0.2. Positional encoding uses Rotary Position Embedding (RoPE) with θ = 10,000, and the model applies pre-norm layer normalisation with ϵ = 10−6 . Table 5

5

Table 5: Model hyperparameters. Hidden size 1,536

Layers

Half layers

Attn. heads

Expansion

H cycles

L cycles

BP max steps

BP warmup ratio

Pos. emb.

RoPE θ

Norm

Norm ϵ

32

true

12

4

2

3

5

0.2

RoPE

10,000

pre-norm

10−6

summarises the model configuration.

4 Training The Mimir model is trained from scratch using the Gemma-4 tokenizer [Gemma Team, 2026], whereas HRM-Text employs a custom one. Through the application of a chat template, the model learns the structural conventions and behavioural patterns characteristic of modern conversational AI. Mimir was trained with Fully Sharded Data Parallelism (FSDP) using bfloat16 as dType for computation with fp32 used as gathering precision (conventional setup). We employ the AdamW optimizer [Loshchilov and Hutter, 2017] with a peak learning rate of 3 × 10−4 , 2,000-step linear warm-up, and a constant schedule thereafter (min ratio 1.0). We use a global batch size of 262,144 tokens with a gradient accumulation of 2 on 8 accelerators for a per-accelerator batch size of 16384. fitting 4 contexts of length 4096 each. Table 6 provides an overview of our hyperparameter values. Our openly available framework4 builds upon Sapient’s code for HRM-Text [Wang et al., 2026]. Table 6: Training hyperparameters. LR 3 × 10−4

LR warmup

LR min ratio

Adam β1 /β2

Weight decay

EMA decay

Global batch

Grad. accum. steps

Seed

2,000

1.0

0.9 / 0.95

0.1

0.9999

262,144

2

0

We trained the model for 1.65M steps on 8 NVIDIA B200 GPUs with 180 GB HBMe3 in just under 3 weeks with an average step time of just under 1.1 seconds.

5 Results Tables 7, 8, and 9 show evaluation results across a broad range of English, Math & Code, and Danish benchmarks. We compare our Mimir model with other 1B-parameter models: HRM-Text, Qwen 3.5, Gemma 3 and OLMo2. Furthermore, we compare Mimir against models in the ranges 2-3B and 4-5B including Gemma 4 E2B with 5B total parameters (effective 2.3B). In Figure 1, we report the average score for English, Math & Code, and Danish benchmark suites. On the English benchmarks, Mimir outperforms all considered competitors on BoolQ, Winogrande, and DROP. On Math & Code, Mimir leads across its weight-class for GSM8K and HumanEval, with Mimir being second overall on GSM8K and better than Qwen3.5 2B on HumanEval. On the Danish benchmarks, Mimir outperforms all competitors on grammatical tasks (DaLA, GEC), question-answering tasks (WikiQA), and is close to the best on Nordjylland News (N.News; summarization). On average, Mimir displays superior performance on the Danish benchmarks, is only 0.3 points behind Qwen 3.5 4B on English tasks, and only 3.8% behind SmolLM3 3B on Math & Code, which was the best tested conventional LLM of the 2–3B weight class. On Math & Code, Mimir yields a 36.7% improvement compared to HRM-Text (64.1 Mimir vs. 46.9 HRM-Text). 4

https://github.com/schneiderkamplab/HRM-Text

6

Table 7: English benchmark results. Best scores in bold. Model

BoolQ (Acc)

Winogrande (Acc)

Hellaswag (Acc)

MMLU (Acc)

ARC-C (Acc)

DROP (F1)

GovRep. Avg. (R1)

∼1B models Mimir 1B HRM-Text 1B Qwen 3.5 0.8B Gemma 3 1B OLMo 2 1B

87.8 87.5 69.8 62.4 67.2

73.5 70.4 48.9 49.1 51.0

67.3 60.4 37.0 30.6 42.4

57.5 58.7 51.5 37.5 41.6

81.6 82.2 68.4 43.5 48.1

83.1 78.1 45.2 7.0 12.4

32.0 25.4 32.5 29.5 37.7

69.0 66.1 50.5 37.1 42.9

2–3B models Qwen 3.5 2B SmolLM3 3B

80.8 84.3

53.4 60.3

64.6 65.1

62.8 60.2

82.7 79.5

31.3 54.0

31.5 38.1

58.2 63.1

4–5B models Qwen 3.5 4B Gemma 4 E2B Gemma 4 E2B (think)

87.0 64.1 83.4

70.0 56.7 63.0

83.2 55.6 55.8

75.8 59.3 72.0

92.9 69.8 86.8

48.0 57.3 70.8

27.9 33.6 34.7

69.3 56.6 66.6

Table 8: Math & Code benchmark results. Best scores in bold. Model

GSM8K (Acc)

MATH (Acc)

∼1B models Mimir 1B HRM-Text 1B Qwen 3.5 0.8B Gemma 3 1B OLMo 2 1B

89.9 84.8 49.1 49.7 59.4

45.8 56.0 36.1 37.2 18.8

56.7 0.0 30.5 42.7 15.9

64.1 46.9 38.6 43.2 31.4

2–3B models Qwen 3.5 2B SmolLM3 3B

73.7 80.0

55.7 62.2

47.6 61.6

59.0 67.9

4–5B models Qwen 3.5 4B Gemma 4 E2B Gemma 4 E2B (think)

60.5 88.3 90.3

56.5 64.2 49.1

78.0 73.8 72.0

65.0 75.4 70.5

7

HumanEval Avg. (Acc)

Table 9: Danish benchmark results. Best scores in bold. Model

Angry Tweets (Acc)

DaLA

GEC

PIQA

Daisy

WikiQA

WMT

N.News

IFEval

Hellaswag- Avg. DA (Acc)

(F1)

(EM)

(Acc)

(EM)

(EM)

(chrF)

(chrF)

(Acc)

∼1B models Mimir 1B HRM-Text 1B Qwen 3.5 0.8B Gemma 3 1B OLMo 2 1B

67.4 42.4 53.8 54.4 33.6

96.1 26.7 51.0 41.0 48.7

85.6 0.5 0.7 3.3 0.2

53.7 13.0 56.5 72.2 75.0

9.6 0.0 0.7 1.4 0.0

66.8 34.9 41.6 42.6 8.4

53.9 25.4 37.8 45.1 30.0

35.87 26.76 35.30 35.56 33.77

63.9 18.5 39.6 47.2 32.5

35.3 28.8 25.0 24.8 26.7

56.8 21.7 34.2 36.8 28.9

2–3B models Qwen 3.5 2B SmolLM3 3B

61.6 63.2

36.4 33.5

8.0 3.3

25.0 51.9

2.5 2.2

49.4 0.3

45.6 37.3

34.85 35.98

56.1 49.8

24.7 40.1

34.4 31.7

4–5B models Qwen 3.5 4B Gemma 4 E2B Gemma 4 E2B (think)

69.1 64.6 67.7

50.1 56.7 66.8

42.6 36.9 23.4

70.4 46.3 63.9

4.7 5.6 5.1

57.1 44.1 59.3

52.1 55.2 56.0

37.03 35.67 36.30

73.7 75.5 81.2

34.7 25.6 39.0

49.2 44.6 49.9

8–9B models Munin-Apertus 8B Munin-Mistral 8B Munin-Qwen 9B

60.6 61.3 69.1

46.1 48.8 60.6

42.1 26.4 11.4

81.5 76.9 38.9

12.5 8.4 5.4

49.9 48.4 55.7

55.8 51.8 56.1

30.30 32.92 35.89

53.0 67.8 71.8

24.5 33.6 34.3

45.6 45.6 43.9

Evaluation Setup All benchmarks were evaluated at temperature 0 (greedy decoding) with shuffle seed 4242 on full datasets. All models used vLLM-served [Kwon et al., 2023] endpoints with FlashInfer, with the exception of Mimir, which requires FlashAttention to correctly capture the PrefixLM and Gemma 4 chat template. We ran both vLLM with FlashAttention4 [Zadouri et al., 2026] and Hugging Face Transformers, obtaining comparable results up to numerical stability. For the ease of reproduction, we report the results from Hugging Face Transformers [Wolf et al., 2019]. Some English benchmarks use few-shot prompting, with the number of shots following Wang et al. [2026]’s evaluation config. All Danish tasks are 0-shot. MCQ tasks use max tokens=1. Details in Appendix B. Gemma 4 was evaluated in two modes: non-thinking and thinking with vLLM flag --reasoning-parser gemma4 to strip thinking tokens before scoring. Thinking requires ∼500–650 tokens. All non-MCQ tasks (or whenever reasoning is enabled) use max tokens=2048. All baseline evaluations were conducted via the Inspect AI Framework of the AI Security Institute [2024].

6 Conclusion, Limitations and Future Directions In this technical report, we presented Mimir v1, a 1-billion-parameter language model that leverages the HRM-Text architecture to provide frontier-level performance using only permissible data, excluding data containing personal information or copyright infringement and including data that is either openly licensed, made available by agreement, or allowed by the European Union’s text and data mining exception for research institutions. By curating a diverse, large corpus, consisting of 70.5B tokens per epoch, including synthetic ‘transplant’ datasets, we have demonstrated that high-quality instruction following and reasoning capabilities can be achieved without relying on large-scale pre-training corpora or prohibited data sources. Mimir shows strong improvements over the baseline HRM-Text and remains competitive, or superior, to much larger models in several English and Danish benchmarks. Despite the generally highly competitive performance, Mimir v1 8

still lags behind Gemma 4 (5B, effective 2.3B) on the Math & Code domains, making room for improvement in future iterations. Future work will focus on investigating scaling behavior of HRM models like Mimir. Moreover, even though Mimir is trained with a Gemma 4 chat template from scratch, the capabilities as an assistant are still limited compared to the state of the art. This calls for future work in this capacity, including reinforcement learning, which is yet unexplored for this architecture. Lastly, we will continue developing the dataset to achieve full openness regarding licensing and further improved model performance.

7 Contributors We list contributions according to the Contributor Roles Taxonomy (CRediT)5 : Peter Schneider-Kamp Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing Jacob Nielsen Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – original draft, Writing – review & editing Lukas Galke Poech Formal analysis, Investigation, Methodology, Software, Supervision, Validation, Visualization, Writing – review & editing Gianluca Barmina Data curation, Investigation, Writing – review & editing Kenneth Enevoldsen Data curation, Resources, Writing – review & editing Additional Contributors Mogens Henrik From Investigation, Project administration, Software Andrea Blasi Núñez Formal analysis, Software, Visualization Annemette Brok Pirchert Formal analysis, Software, Visualization Stine Lyngsø Beltoft Data curation, Resources Torben Blach Project administration, Funding acquisition Sofie Helene Bruun Data curation, Resources Oliver Kinch Data curation, Resources Rasmus Larsen Data curation, Resources Dan Saattrup Smart Data curation, Resources Kristoffer Laigaard Nielbo Funding acquisition 5

https://credit.niso.org

9

8 Acknowledgements This work originated within OdenseNLP6 and was supported by the Danish Foundation Models project – a collaboration between the University of Southern Denmark, Aarhus University, the Alexandra Institute, and the University of Copenhagen – funded by the Ministry of Science, Higher Education and Digital Affairs. We thank all members of the Danish Foundation Models project and the OdenseNLP research group for many fruitful discussions and their continued support.

References UK AI Security Institute. Inspect AI: Framework for Large Language Model Evaluations, 2024. URL https://github.com/UKGovernmentBEIS/inspect_ai. Kenneth Enevoldsen, Lasse Hansen, Dan S. Nielsen, Rasmus A. F. Egebæk, Søren V. Holm, Martin C. Nielsen, Martin Bernstorff, Rasmus Larsen, Peter B. Jørgensen, Malte Højmark-Bertelsen, Peter B. Vahlstrup, Per Møldrup-Dalum, and Kristoffer Nielbo. Danish foundation models, 2023. URL https://arxiv.org/abs/2311.07264. Kenneth Enevoldsen, Kristian Nørgaard Jensen, Jan Kostkan, Balázs Szabó, Márton Kardos, Kirsten Vad, Johan Heinsen, Andrea Blasi Núñez, Gianluca Barmina, Jacob Nielsen, Rasmus Larsen, Peter Vahlstrup, Per Møldrup Dalum, Desmond Elliott, Lukas Galke, Peter SchneiderKamp, and Kristoffer Nielbo. Dynaword: From one-shot to continuously developed datasets, 2025. Gemma Team. Gemma 4 technical report, 2026. URL https://arxiv.org/abs/2607.02770. Nikhil Kandpal, Brian Lester, Colin Raffel, Sebastian Majstorovic, Stella Biderman, Baber Abbasi, Luca Soldaini, Enrico Shippole, A. Feder Cooper, Aviya Skowron, John Kirchenbauer, Shayne Longpre, Lintang Sutawika, Alon Albalak, Zhenlin Xu, Guilherme Penedo, Loubna Ben Allal, Elie Bakouch, John David Pressman, Honglu Fan, Dashiell Stander, Guangyu Song, Aaron Gokaslan, Tom Goldstein, Brian R. Bartoldson, Bhavya Kailkhura, and Tyler Murray. The common pile v0.1: An 8tb dataset of public domain and openly licensed text, 2025. URL https://arxiv.org/abs/2506.05209. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023. Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, et al. Tulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:2411.15124, 2024. Zihan Liu, Zhuolin Yang, Yang Chen, Chankyu Lee, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. Acereason-nemotron 1.1: Advancing math and code reasoning through sft and rl synergy. arXiv preprint arXiv:2506.13284, 2025. 6

https://odensenlp.github.io

10

Ilya Loshchilov and Frank Hutter. arXiv:1711.05101, 2017.

Decoupled weight decay regularization.

arXiv preprint

Team Olmo. Olmo 3, 2025. URL https://arxiv.org/abs/2512.13961. Shubham Toshniwal, Wei Du, Ivan Moshkov, Branislav Kisacanin, Alexan Ayrapetyan, and Igor Gitman. Openmathinstruct-2: Accelerating ai for math with massive open-source instruction data. arXiv preprint arXiv:2410.01560, 2024. Guan Wang, Changling Liu, Chenyu Wang, Cai Zhou, Yuhao Sun, Yifei Wu, Shuai Zhen, Luca Scimeca, and Yasin Abbasi Yadkori. Hrm-text: Efficient pretraining beyond scaling. arXiv preprint arXiv:2605.20613, 2026. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771, 2019. Ted Zadouri, Markus Hoehnerbach, Jay Shah, Vijay Thakkar, and Tri Dao. Flashattention-4: Algorithm and kernel pipelining co-design for asymmetric hardware scaling. Proceedings of Machine Learning and Systems, 8:912–926, 2026.

11

A List of Training Datasets Table 10 lists all the datasets with their corresponding HuggingFace identifier, the form type, how many tokens they represent, and the share in percentages. Table 10: All 161 datasets in the Mimir corpus, sorted by sampled tokens per epoch. #

Dataset

Form

1.

sapientinc/HRM-Text-data-io-cleaned20260515 danish-foundation-models/laerebogen nvidia/OpenMathInstruct-2 nvidia/Nemotron-SFT-Agentic-v2 danish-foundation-models/dfm-dynainstruct allenai/Dolci-Instruct-SFT-No-Tools schneiderkamplab/opus-da-enpermissive allenai/Dolci-Instruct-SFT nvidia/AceReason-1.1-SFT allenai/big-reasoning-traces allenai/Dolci-Instruct-SFT-Tool-Use

2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24.

nvidia/Nemotron-SFT-InstructionFollowing-Chat-v2 allenai/tulu-3-sft-mixture laion/Scientific-Summaries schneiderkamplab/common-pile-prefixcontinuation synquid/wiki-instruct-da schneiderkamplab/dfm8-openhermes-da schneiderkamplab/common-piledenoising allenai/tulu-v2-sft-mixture schneiderkamplab/common-pile-spanfilling schneiderkamplab/dfm8-openhermes-en allenai/tulu-v2-sft-long-mixture allenai/verifiable-reasoning-filtered-gpt41 open-thoughts/OpenThoughts2-1M

12

Tokens/epoch

Share

Curated + reformatted Reformatted Reformatted Reformatted Reformatted

11.92B

16.91%

8.32B 6.60B 4.27B 3.54B

11.81% 9.37% 6.06% 5.03%

Reformatted Reformatted

3.49B 2.90B

4.95% 4.12%

Reformatted Reformatted Reformatted Tool-call formatted Reformatted

2.24B 1.95B 1.66B 1.61B

3.17% 2.76% 2.35% 2.29%

1.60B

2.27%

Reformatted Reformatted Synthetic + audited Reformatted Translated + audited Synthetic + audited Reformatted Synthetic + audited Translated + audited Reformatted Reformatted

1.57B 1.27B 1.15B

2.23% 1.80% 1.64%

988M 922M

1.40% 1.31%

884M

1.25%

840M 822M

1.19% 1.17%

672M

0.95366%

605M 605M

0.85902% 0.85798%

Reformatted

534M

0.75834%

continued from previous page # 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44. 45. 46. 47. 48. 49.

Dataset

Form

schneiderkamplab/dfm8-syntheticnative-tool-calling schneiderkamplab/transformationsdanish-danish schneiderkamplab/transformationsenglish-english schneiderkamplab/transformationsdanish-english schneiderkamplab/transformationsenglish-danish nvidia/Nemotron-SFT-Multilingual-v1 schneiderkamplab/dfm8-syntheticdanish-summarization-rewrite-controls MegaScience/TextbookReasoning schneiderkamplab/dfm8-syntheticmultiturn-danish-english-chat DBC (agreement-supplied)

Synthetic + dited Synthetic + dited Synthetic + dited Synthetic + dited Synthetic + dited Reformatted Synthetic + dited Reformatted Synthetic + dited Agreementsupplied Synthetic + dited Agreementsupplied Reformatted Synthetic + dited Synthetic + dited Synthetic + dited Reformatted Reformatted Reformatted Reformatted Synthetic + dited Synthetic + dited Reformatted Synthetic + dited Synthetic + dited

schneiderkamplab/danish-dynaworddenoising Lex.dk articles oliverkinch/machine-translation-da-en schneiderkamplab/danish-dynawordprefix-continuation schneiderkamplab/danish-dynawordspan-filling schneiderkamplab/dfm8-synthetic-codedebugging GEM/wiki cat sum allenai/tulu-3-sft-personas-math giannor/gec dala tv2r it synquid/wildchat-100k-qwen-messages schneiderkamplab/dfm8-synthetic-strictmath-answer-contract schneiderkamplab/dfm8-syntheticconstrained-format-following oliverkinch/danish-summarization schneiderkamplab/common-pileparagraph-reordering schneiderkamplab/danish-dynawordparagraph-reordering 13

Tokens/epoch

Share

au-

513M

0.728%

au-

496M

0.70403%

au-

471M

0.66842%

au-

469M

0.66529%

au-

418M

0.59274%

au-

413M 396M

0.58565% 0.56181%

au-

374M 366M

0.5311% 0.51997%

356M

0.50529%

323M

0.45861%

313M

0.44467%

au-

284M 252M

0.40348% 0.35728%

au-

251M

0.35593%

au-

247M

0.34996%

au-

205M 204M 193M 190M 173M

0.29117% 0.28922% 0.27377% 0.2694% 0.24605%

au-

171M

0.24275%

au-

168M 167M

0.23888% 0.23653%

au-

160M

0.22769%

au-

continued from previous page #

Dataset

Form

50.

glaiveai/glaive-function-calling-v2

51. 52. 53. 54. 55. 56. 57.

oliverkinch/machine-translation-da-ar allenai/SciRIFF-train-mix common-pile/arxiv papers filtered oliverkinch/da-instruct-dynaword synquid/translation-100k kobprof/skolegpt-instruct allenai/verifiable-reasoning-filtered-o4mini oliverkinch/tidsskrift-dk-bt allenai/open math 2 50k r1-original oliverkinch/machine-translation-da-uk giannor/dala tv2r it Salesforce/xlam-function-calling-60k

Tool-call formatted Reformatted Reformatted Derived task Reformatted Reformatted Reformatted Reformatted

58. 59. 60. 61. 62. 63. 64. 65. 66. 67. 68. 69. 70. 71. 72. 73. 74. 75. 76. 77. 78. 79. 80. 81. 82.

oliverkinch/danish-qa oliverkinch/dst-table-prompts-bt danish-foundationmodels/ai arena udtraek oliverkinch/multi-wiki-qa-high-qualitysubset oliverkinch/dynaword-bt HuggingFaceH4/no robots oliverkinch/da-instruct-dynawordcontemporary-hq Team-ACE/ToolACE oliverkinch/da-instruct-dynaword-hq oliverkinch/danish-university-portals-bt allenai/tulu-3-sft-personas-algebra allenai/IF sft data verified oliverkinch/autodata-da-sft oliverkinch/danmarks-statistik-bt synquid/danish-verifiable-reasoning oliverkinch/da-instruct-dynawordcontemporary oliverkinch/eur-lex-bt allenai/tulu-3-sft-personas-code oliverkinch/instruct-bt schneiderkamplab/sapient-synth-flandialog-fsopt-data-qrecc

Share

156M

0.22066%

140M 132M 130M 103M 98.2M 86.4M 80.2M

0.19802% 0.18771% 0.18386% 0.14592% 0.13938% 0.12259% 0.11378%

Reformatted Reformatted Reformatted Reformatted Tool-call formatted Reformatted Reformatted Reformatted

76.7M 71.8M 69.5M 68.5M 67.0M

0.10878% 0.1019% 0.09859% 0.09723% 0.09502%

57.5M 51.1M 45.7M

0.08153% 0.07248% 0.06483%

Reformatted

41.9M

0.05947%

Reformatted Reformatted Reformatted

34.3M 31.4M 25.6M

0.04867% 0.04459% 0.03626%

Tool-call formatted Reformatted Reformatted Reformatted Reformatted Reformatted Reformatted Reformatted Reformatted

25.2M

0.03579%

24.1M 21.8M 21.0M 19.9M 18.9M 18.9M 18.1M 17.6M

0.03414% 0.03088% 0.02982% 0.02829% 0.02677% 0.02676% 0.02573% 0.02491%

16.7M 14.6M 13.5M 13.1M

0.02369% 0.02077% 0.0191% 0.0186%

Reformatted Reformatted Reformatted Synthetic + audited 14

Tokens/epoch

continued from previous page # 83. 84. 85. 86. 87. 88. 89. 90. 91. 92.

93. 94.

95. 96.

97.

98.

99. 100.

101. 102.

Dataset

Form

synquid/ifbench-train allenai/tulu-3-sft-personas-instructionfollowing schneiderkamplab/sapient-synth-flanflan-fsopt-data-aeslc-1.0.0 synquid/mt-da-deepseek allenai/RLVR-MATH allenai/RLVR-GSM allenai/Dolci-Instruct-SFT-Tool-Use-SA

Reformatted Reformatted

12.7M 11.5M

0.018% 0.01634%

Synthetic + audited Reformatted Reformatted Reformatted Tool-call formatted Synthetic + audited Reformatted Synthetic + audited

10.2M

0.01441%

8.9M 8.1M 6.8M 5.8M

0.01261% 0.01145% 0.00968% 0.00825%

4.8M

0.0068%

4.4M 4.3M

0.00626% 0.00612%

Synthetic + audited Synthetic + audited

2.9M

0.00413%

2.8M

0.00399%

Reformatted Synthetic + audited

2.6M 2.5M

0.00373% 0.0036%

Synthetic + audited

2.2M

0.00309%

Synthetic + audited

1.9M

0.00276%

Synthetic + audited Synthetic + audited

1.9M

0.00271%

1.7M

0.00247%

Synthetic + audited Synthetic + audited

1.7M

0.00245%

1.7M

0.00245%

schneiderkamplab/sapient-synth-flanflan-fsnoopt-data-aeslc-1.0.0 ccdv/govreport-summarization schneiderkamplab/sapient-synth-flanflan-fsopt-data-opinion-abstracts-rottentomatoes schneiderkamplab/sapient-synth-flandialog-fsopt-data-qrecc-ii schneiderkamplab/sapient-synth-flanniv2-fsopt-data-task589-amazonfoodsummary-text-generation oliverkinch/eur-lex-sum-instruct schneiderkamplab/sapient-synth-flanniv2-fsopt-data-task590-amazonfoodsummary-correction-classification schneiderkamplab/sapient-synth-flanniv2-fsopt-data-task618-amazonreviewsummary-text-generation schneiderkamplab/sapient-synthflan-niv2-fsopt-data-task1309amazonreview-summary-classification schneiderkamplab/sapient-synth-flanflan-zsnoopt-data-aeslc-1.0.0 schneiderkamplab/sapient-synth-flanflan-fsnoopt-data-opinion-abstractsrotten-tomatoes schneiderkamplab/sapient-synth-flanflan-zsopt-data-aeslc-1.0.0 schneiderkamplab/sapient-synth-flandialog-zsopt-data-qrecc

15

Tokens/epoch

Share

continued from previous page # 103.

104.

105.

106. 107. 108.

109.

110.

111.

112.

113. 114.

115.

116.

117.

Dataset

Form

schneiderkamplab/sapient-synth-flanniv2-fsopt-data-task1375-newscommtranslation schneiderkamplab/sapient-synth-flanniv2-fsopt-data-task1373-newscommtranslation schneiderkamplab/sapient-synth-flanniv2-fsopt-data-task1376-newscommtranslation schneiderkamplab/sapient-synthplatypus-reclor schneiderkamplab/sapient-synthtasksource-reclor schneiderkamplab/sapient-synth-flanniv2-fsopt-data-task902-deceptiveopinion-spam-classification schneiderkamplab/sapient-synth-flanniv2-fsopt-data-task634-allegro-reviewsclassification schneiderkamplab/sapient-synth-flanniv2-fsopt-data-task635-allegro-reviewsanswer-generation schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task590-amazonfoodsummary-correction-classification schneiderkamplab/sapient-synthflan-niv2-zsopt-data-task1309amazonreview-summary-classification schneiderkamplab/sapient-synth-flanflan-fsopt-data-opinion-abstracts-idebate schneiderkamplab/sapient-synth-flanniv2-fsopt-data-task1370-newscommclassification schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task589-amazonfoodsummary-text-generation schneiderkamplab/sapient-synth-flanniv2-fsopt-data-task1371-newscommtranslation oliverkinch/doab-da-bt

Synthetic + audited

1.2M

0.00166%

Synthetic + audited

1.1M

0.0015%

Synthetic + audited

974K

0.00138%

Synthetic + audited Synthetic + audited Synthetic + audited

922K

0.00131%

901K

0.00128%

878K

0.00125%

Synthetic + audited

816K

0.00116%

Synthetic + audited

816K

0.00116%

Synthetic + audited

811K

0.00115%

Synthetic + audited

795K

0.00113%

Synthetic + audited Synthetic + audited

746K

0.00106%

721K

0.00102%

Synthetic + audited

705K

0.001%

Synthetic + audited

703K

0.001%

Reformatted

694K

0.00098%

16

Tokens/epoch

Share

continued from previous page # 118.

119. 120.

121.

122.

123.

124. 125.

126.

127.

128. 129.

130.

131.

132.

Dataset

Form

schneiderkamplab/sapient-synth-flanniv2-fsopt-data-task1377-newscommtranslation danish-foundation-models/kaenguruen schneiderkamplab/sapient-synth-flanflan-zsopt-data-opinion-abstracts-rottentomatoes schneiderkamplab/sapient-synth-flanflan-zsnoopt-data-opinion-abstractsrotten-tomatoes schneiderkamplab/sapient-synth-flanniv2-fsopt-data-task870-msmarcoanswer-generation schneiderkamplab/sapient-synth-flanniv2-fsopt-data-task1374-newscommtranslation schneiderkamplab/sapient-synthtasksource-pragmeval-sarcasm schneiderkamplab/sapient-synth-flanniv2-fsopt-data-task903-deceptiveopinion-spam-classification schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task618-amazonreviewsummary-text-generation schneiderkamplab/sapient-synth-flanflan-fsnoopt-data-opinion-abstractsidebate schneiderkamplab/sapient-synth-flandialog-zsopt-data-qrecc-ii schneiderkamplab/sapient-synth-flanniv2-fsopt-data-task265-paper-reviewslanguage-identification schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task635-allegro-reviewsanswer-generation schneiderkamplab/sapient-synthflan-niv2-fsopt-data-task909-dialogreprevalent-speakers schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task1376-newscommtranslation

Synthetic + audited

679K

0.00096%

Reformatted Synthetic + audited

638K 622K

0.0009% 0.00088%

Synthetic + audited

616K

0.00087%

Synthetic + audited

599K

0.00085%

Synthetic + audited

599K

0.00085%

Synthetic + audited Synthetic + audited

542K

0.00077%

535K

0.00076%

Synthetic + audited

532K

0.00076%

Synthetic + audited

527K

0.00075%

Synthetic + audited Synthetic + audited

513K

0.00073%

395K

0.00056%

Synthetic + audited

320K

0.00045%

Synthetic + audited

296K

0.00042%

Synthetic + audited

274K

0.00039%

17

Tokens/epoch

Share

continued from previous page # 133.

134.

135.

136.

137.

138.

139.

140.

141.

142.

143.

144.

145.

146.

Dataset

Form

schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task1373-newscommtranslation schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task1375-newscommtranslation schneiderkamplab/sapient-synth-flanniv2-fsopt-data-task266-paper-reviewsreviewer-perspective schneiderkamplab/sapient-synthflan-niv2-fsopt-data-task906-dialogreidentify-names schneiderkamplab/sapient-synth-flanflan-zsnoopt-data-opinion-abstractsidebate schneiderkamplab/sapient-synth-flanflan-zsopt-data-opinion-abstractsidebate schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task634-allegro-reviewsclassification schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task870-msmarcoanswer-generation schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task902-deceptiveopinion-spam-classification schneiderkamplab/sapient-synth-flanniv2-fsopt-data-task672-amazon-yelpsummarization schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task1377-newscommtranslation schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task1374-newscommtranslation schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task1371-newscommtranslation schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task265-paper-reviewslanguage-identification

Synthetic + audited

268K

0.00038%

Synthetic + audited

265K

0.00038%

Synthetic + audited

236K

0.00033%

Synthetic + audited

235K

0.00033%

Synthetic + audited

223K

0.00032%

Synthetic + audited

223K

0.00032%

Synthetic + audited

215K

0.0003%

Synthetic + audited

212K

0.0003%

Synthetic + audited

198K

0.00028%

Synthetic + audited

198K

0.00028%

Synthetic + audited

182K

0.00026%

Synthetic + audited

174K

0.00025%

Synthetic + audited

171K

0.00024%

Synthetic + audited

154K

0.00022%

18

Tokens/epoch

Share

continued from previous page # 147.

148.

149. 150.

151.

152.

153.

154.

155.

156.

157.

158.

159.

160.

Dataset

Form

schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task903-deceptiveopinion-spam-classification schneiderkamplab/sapient-synth-flanniv2-fsopt-data-task264-paper-reviewsaccept-reject schneiderkamplab/sapient-synthplatypus-scibench schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task672-amazon-yelpsummarization schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task909-dialogreprevalent-speakers schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task266-paper-reviewsreviewer-perspective schneiderkamplab/sapient-synth-flanniv2-fsopt-data-task871-msmarcoquestion-generation schneiderkamplab/sapient-synthflan-niv2-fsopt-data-task908-dialogreidentify-familial-relationships schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task906-dialogreidentify-names schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task264-paper-reviewsaccept-reject schneiderkamplab/sapient-synthflan-niv2-fsopt-data-task907-dialogreidentify-relationships schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task871-msmarcoquestion-generation schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task908-dialogreidentify-familial-relationships schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task907-dialogreidentify-relationships

Synthetic + audited

149K

0.00021%

Synthetic + audited

140K

0.0002%

Synthetic + audited Synthetic + audited

104K

0.00015%

85.3K

0.00012%

Synthetic + audited

82.0K

0.00012%

Synthetic + audited

71.9K

0.0001%

Synthetic + audited

71.1K

0.0001%

Synthetic + audited

68.4K

0.0001%

Synthetic + audited

57.5K

0.00008%

Synthetic + audited

49.3K

0.00007%

Synthetic + audited

46.0K

0.00007%

Synthetic + audited

35.6K

0.00005%

Synthetic + audited

17.3K

0.00002%

Synthetic + audited

13.4K

0.00002%

19

Tokens/epoch

Share

continued from previous page # 161.

Dataset

Form

schneiderkamplab/sapient-synth-flanniv2-zsopt-data-task1370-newscommclassification

Synthetic + audited

Tokens/epoch 1.9K

Share <0.00001%

B List of Evaluation Datasets Table 11 lists all considered evaluation datasets with their precise source as HuggingFace identifier, and the number of in-context shots provided to the models during evaluation. Table 11: Overview of benchmarks and their configuration. All benchmarks are evaluated with temperature 0 and a fixed seed. Task

Dataset (split)

English BoolQ Winogrande Hellaswag MMLU ARC-C DROP GovReport

google/boolq allenai/winogrande Rowan/hellaswag cais/mmlu allenai/ai2 arc EleutherAI/drop ccdv/govreport

5 5 10 5 25 3 0

Math&Code GSM8K MATH HumanEval

openai/gsm8k EleutherAI/hendrycks math human eval (164 problems)

0 0 0

Danish Angry Tweets DaLA GEC-DaLA PIQA-da Daisy Multi Wiki QA WMT24++ EN-DA Nordj. News Summ. IFEval-Da Hellaswag-da

DDSC/angry-tweets giannor/dala giannor/dala gen v3 local JSON schneiderkamplab/SDU-Daisy oliverkinch/multi-wiki-qa synquid/wmt24pp alexandrainst/nordj-news danish-foundation-models/ifeval-da EuroEval

0 0 0 0 0 0 0 0 0 0

20

N-shots

Related documents

• DFM Mimir v1: An Open HRM Delivering Frontier Performance at 1B Parameters Using Only Permissible Post-Training Data#652456
• DFM Mimir v1: An Open HRM Delivering Frontier Performance at 1B Parameters Using Only Permissible Post-Training Data#651003
• What Does Frontier Access Buy? Benchmarking LLM Access Tiers and the Reproducibility Case for Open Weights in Survey Coding#428545
• Ethics and HRM Education#774709
• The future of work: implications for the frontline manager's role in HR implementation#185035