Articles
From foundational knowledge to independent analysis. Choose a reading path.
What do these statuses mean?
Status describes reading readiness within the current scope. Ready to Read articles may still receive updates.
Ready to Read A coherent treatment of the current scope, ready for end-to-end reading.
In Progress Substantial content is available; sections, derivations, or validation are still being developed.
Outline An outline, source collection, or early notes.
Deep Dives 2
Explore specific questions through derivations, comparisons, and independent analysis. View Collection →
- Deep Dives In ProgressTraining Inference Mismatch
分析训练与推理不一致如何影响策略优化，梳理误差来源与修正思路。
- Deep Dives In ProgressKL divergence from machine learning to reinforcement learning
围绕 KL 散度的估计与优化展开推导，辨析不同形式的含义和适用条件。
Learning Notes & Tutorials 9
Build the mathematical, algorithmic, and systems foundations for understanding LLMs. View Collection →
- Learning Notes Ready to ReadTokenizer
从文本到 token：理解分词方法，以及词表与编码的基本取舍。
- Learning Notes Ready to ReadLLM Resource Computation: Parameters, FLOPs and Memory
理解大模型参数量、计算量与显存开销，建立训练资源估算的基础。
- Learning Notes Ready to ReadReinforcement Learning for Large Language Models: An Overview
从强化学习基础到大模型训练，串起价值估计、策略优化与关键推导。
- Learning Notes In ProgressMath Foundations
- Learning Notes In ProgressHands on LLM(2) Transformer
基于Qwen3讲解transformer的架构以及核心代码
- Learning Notes In ProgressOverview of Parallelism
- Learning Notes In ProgressTheory of scalability
- Learning Notes In ProgressGPU vs TPU: A Comprehensive Guide to Specialized Hardware Accelerators
An in-depth exploration of Graphics Processing Units and Tensor Processing Units, comparing their computational capabilities, memory architectures, and applications in machine learning, deep learning, and scientific computing.
- Learning Notes OutlineDeep Learning Techniques
Topic Overviews 17
Follow how problems, methods, and trade-offs evolve within a research area. View Collection →
- Topic Overviews In ProgressOverview of Agentic RL
从 Agent 定义、POMDP 形式化与核心挑战出发，分别讨论 Data、Infra 与 Methods（奖励、算法、上下文、评测），并覆盖近期进展与案例。
- Topic Overviews In ProgressOverview of Attention Mechanism
A hands-on guide to understanding and implementing the Attention mechanism in deep learning models.
- Topic Overviews In ProgressOverview of Multimodal Large Language Models
- Topic Overviews In ProgressLong Context：问题、解决方法、主流方案与现状
- Topic Overviews In ProgressOverview of LLM Inference
梳理大语言模型推理中的采样策略、Prefill 与 Decode 性能分析、KV cache、Continuous batching、并行推理、Speculative decoding 与 Multi-token prediction，并讨论生成结果的可复现性。
- Topic Overviews In ProgressOverview of Visual Foundation Models
- Topic Overviews In ProgressOverview of Policy Optimization Methods
- Topic Overviews In ProgressOverview of position encoding
- Topic Overviews In ProgressScaling law
Publish‑ready workflow that lets you focus on ideas, not infrastructure
- Topic Overviews In ProgressMoE tutorial
本 blog 详细介绍了 MoE 模型的一些关键设计与相关实验结果，为 MoE 模型的学习提供基础。
- Topic Overviews In ProgressOverview of optimizers for LLMs
Overview of optimizers for LLMs
- Topic Overviews In ProgressOverview of Flash Attention series
作者提出了 flashattention, 一个通过降低 multi head attention 内存访问开销来提高 attention 计算效率的方法
- Topic Overviews In ProgressOverview of OPD
- Topic Overviews In ProgressOverview of RLHF
Overview of RLHF
- Topic Overviews OutlineOverview of unified MLLM
Overview of unified MLLM
- Topic Overviews OutlineOverview of RLVR
Overview of RLVR
- Topic Overviews OutlineLow Precision
Technical Reports 19
Reading notes on model architectures, training recipes, and experimental evidence. View Collection →
- Technical Reports In ProgressOverview of MiniMax series
- Technical Reports In ProgressNotes on Technical Reports
Reading notes on LLM technical reports that have not been organized into dedicated series.
- Technical Reports In ProgressOverview of LLaMA series
- Technical Reports In ProgressOverview of Nemotron series
- Technical Reports In ProgressOverview of Gemma series
Overview of Gemma series
- Technical Reports In ProgressOverview of Qwen series
系统性回顾Qwen系列模型架构的演变与关键技术
- Technical Reports In ProgressOverview of Qwen-VL series
Overview of Qwen-VL series
- Technical Reports In ProgressOverview of MiMo series
- Technical Reports In ProgressOverview of MiniCPM series
- Technical Reports In ProgressOverview of OLM series
- Technical Reports In ProgressOverview of Composer series
- Technical Reports In ProgressOverview of DeepSeek series
- Technical Reports In ProgressOverview of Gemini Series
Gemini 3.0 是是 Google 新一代最强模型，model card 介绍了 Gemini 3.0 系列的评估结果以及基本能力
- Technical Reports In ProgressOverview of GPT series
openAI 发布了 gpt-oss 大语言模型，包含 120B-A5.1B 以及 20.9B-A3.6B 两个 size, 作者强调了模型的 instruction following, tool use, 以及 adaptive thinking 能力
- Technical Reports In ProgressOverview of InternVL series
- Technical Reports In ProgressOverview of Keye-VL series
- Technical Reports In ProgressOverview of Kimi series
- Technical Reports In ProgressOverview of GLM series
- Technical Reports OutlineDeepSeek-R1: the first open sourced reasoning models