MTP
Nemotron 3 Super (NVIDIA et al., 2026) 提出了在不同的 head 之间共享参数来解决 distribution shift 的问题
- NVIDIA, :, Chandiramani, A., Blakeman, A., Olaoye, A., Gupta, A., Somasamudramath, A., Khattar, A., Adesoba, A., Renduchintala, A., Asif, A., Agrawal, A., Vavre, A., Kiswani, A., Padmakumar, A., Hotchandani, A., Shukla, A., Bercovich, A., Ficek, A., … Ahmed, Z. (2026). Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning. https://arxiv.org/abs/2604.12374