Overview of MiniCPM series

Author

Updated

Sep, 11, 2026

Category

LLM

MiniCPM-5

ModelparametersAttentionContext Length
MiniCPM5-1B1.08BGQA (16/2)128K
MiniCPM5-2B2.5BGQA (16/2)128K

MiniCPM5-1B

MiniCPM5-1B Training Pipeline
MiniCPM5-1B Training Pipeline. Sourced from https://huggingface.co/openbmb/MiniCPM5-1B

pre-training data: Ultra Fineweb, Ultra-fineweb-l3, ultradata-math

post-training: multi-teacher OPD

SFT: 200B deep-thinking SFT, 200B hybrid thinking SFT (UltraData-SFT-2605)

RL:

performance

MiniCPM5-1B Score Gain from RL+OPD
MiniCPM5-1B Score Gain from RL+OPD. Sourced from https://huggingface.co/openbmb/MiniCPM5-1B
MiniCPM5-1B overlong response ratio drop from RL+OPD
MiniCPM5-1B overlong response ratio drop from RL+OPD. Sourced from https://huggingface.co/openbmb/MiniCPM5-1B

MiniCPM5-2B

MiniCPM5-2B 的数据包括：

  1. UltraX, web pre-training dataset
  2. UltraData-Code, coding data
  3. UltraData-SFT-agent-2609, 500K agent training examples
  4. UltraData-RL-2609, 80K RL training examples

作者主要强调了 post-training 过程中的 RL + OPD 训练策略，RL 作者使用了 JustRL II 算法

MiniCPM5-2B Training Pipeline
MiniCPM5-2B Training Pipeline. Sourced from https://huggingface.co/openbmb/MiniCPM5-2B