Skip to main content

Mao Song(毛松)'s Homepage

Delving into the Latent Unknown.

Home
Archives
Search
Tags
About
1. Dark Mode

Search

Categories

LLM LeetCode MLLM Infra Tutorial Machine Learning Reasoning Math NLP Terminal Deep Learning RAG RL Unified MLLM 随笔

Tags

MoE Qwen Medium Attention Reasoning Google Deepseek DFS Position Encoding Matrix Array DP RL String Transformer Tree BFS Bit Manipulation Hard Distributed Training

Archives

Tags

13 pages

Attention

Notes on QK-Norm

August 13, 2025

Notes on GQA

Notes on MQA

1 2 3

© 2020 - 2026 Mao Song(毛松)'s Homepage

Built with Hugo
Theme Stack designed by Jimmy