所属模块：
M4 · 序列与 Transformer (Sequences & Transformers)| 专题分类：KV Cache 与推理优化 (KV Cache & Inference Optimizations)| 难度等级：Easy
一、核心一句话结论 (One-Sentence Summary)
prefill 并行处理输入（compute-bound、算力密集）；decode 逐 token 生成（memory-bound、读 KV）；两者需不同优化。
The prefill phase processes the prompt in parallel and is compute-bound, whereas the decode phase generates tokens autoregressively one by one and is memory-bandwidth bound.
二、核心考点要义 (Key Insights)
- 📌 prefill：整段输入并行，大矩阵乘，算力受限
- 📌 decode：每步 1 token，读整个 KV cache，带宽受限
- 📌 TTFT 由 prefill 决定；TPOT 由 decode 决定
English Insights:
– Prefill (Time-to-First-Token / TTFT): computes $Q, K, V$ for all prompt tokens simultaneously in parallel; high arithmetic intensity, saturating GPU Tensor Cores (compute-bound)
– Decode (Time-Per-Output-Token / TPOT): generates one token per step ($Q$ length is 1); reads the entire historical KV cache for a tiny matrix-vector multiplication, severely constrained by memory bandwidth (memory-bound)
– Different optimization objectives: Prefill prioritizes FlashAttention, tensor parallelism, and prompt caching; Decode prioritizes batching concurrency, GQA, quantization, and speculative decoding
三、核心数学原理与机理推导 (Mathematical Principles & Derivation)
$$text{prefill}: L text{queries parallel}, text{compute-bound};qquad text{decode}: 1 text{query}, text{memory-bound}$$
数学机理：prefill（预填充）——处理用户输入的整段 prompt（长度 L），所有位置的 Q/K/V 可一次性并行计算（类似训练的前向），注意力是 L×L 的大矩阵运算，FFN 是 L×d×d 的大矩阵乘；故 prefill 是 compute-bound（受算力限制），且能高效利用张量核心。decode（解码）——逐 token 生成，每步只有 1 个 query，需读取整个 KV cache（长度 S）做注意力；每步的 FLOPs 很小但访存量 ∝S，故是 memory-bound（受带宽限制），且并行度低（batch×heads）。关键指标：(a) TTFT（Time To First Token）——首 token 延迟，主要由 prefill 决定（∝ prompt 长度）；(b) TPOT（Time Per Output Token）——每输出 token 的延迟，由 decode 决定（∝ 读取 KV 的量）；(c) 吞吐——由两阶段的效率与批大小共同决定。为什么需不同优化——prefill 适合大 batch 的矩阵乘（算力密集，batch 越大越好）；decode 适合大 batch 以摊薄 KV 读取（带宽密集，batch 越大 KV 复用越高）。故推理引擎需分别调优，甚至分离部署（PD 分离）。两阶段的显存需求也不同——prefill 需大量激活显存（大矩阵中间结果），decode 需大量 KV cache 显存。
📖 查看英文严格数学推导 (English Mathematical Derivation)
Mathematical Mechanism: Prefill Phase: Given prompt length $L_{text{in}}$, the input is $X in mathbb{R}^{L_{text{in}} times d}$. $Q, K, V in mathbb{R}^{L_{text{in}} times d}$ are computed in a single forward pass. The attention computation involves GEMM (General Matrix Multiply): $Q K^T in mathbb{R}^{L_{text{in}} times L_{text{in}}}$. The arithmetic intensity is approximately $frac{2 L_{text{in}} d + 2 L_{text{in}}^2 d}{2 d^2 + 2 L_{text{in}} d}$, which scales with $L_{text{in}}$. For moderate to large $L_{text{in}}$ (e.g., $>512$), operational intensity surpasses the GPU hardware balance point (e.g., ~150 FLOPs/byte on A100), placing prefill squarely in the compute-bound regime. Decode Phase: At generation step $t$, the query is a single vector $q_t in mathbb{R}^{1 times d}$. The model must load the model weights ($W in mathbb{R}^{d times d}$) and all cached keys and values $K_{1:t}, V_{1:t} in mathbb{R}^{t times d}$ from HBM. The operations are GEMV (General Matrix-Vector Multiply). FLOPs executed are $O(d^2 + t d)$, while bytes transferred are $O(d^2 + t d)$. The arithmetic intensity is $approx 1$ FLOP/byte, orders of magnitude below hardware compute capability (~312 TFLOPs vs 2 TB/s HBM bandwidth), making decode strictly memory bandwidth-bound.
四、工业级落地权衡与工程考量 (Industrial Trade-offs)
深度剖析与工程权衡：① ‘chunked prefill’——把长 prompt 的 prefill 切成多个 chunk 逐步处理，与 decode 请求混批，使 (a) 长 prompt 不阻塞短请求（改善延迟）、(b) 计算与带宽资源互补（prefill 算力密集、decode 带宽密集，混批可同时利用）；这是现代推理引擎（vLLM、TensorRT-LLM）的关键调度技术。② PD 分离（disaggregation）——把 prefill 与 decode 部署在不同实例（各自用最适合的硬件/并行策略），通过高速网络传递 KV cache；优点是各阶段可独立扩缩容、避免相互干扰，代价是 KV 传输开销。③ batch 策略的差异——prefill 用’大 batch + 长序列’填满算力；decode 用’尽量大的 batch’摊薄 KV 读取；两者的最优 batch 大小不同，故混批需调度算法（如 Sarathi-Serve 的 chunked prefill + piggybacking）。④ 与投机解码的关系——投机解码在 decode 阶段一次验证多个 token，把’每步 1 个 query’变成’每步 k 个 query’，从而提升 decode 的算术强度（更接近 prefill 的效率）；这是’把 decode 变 compute-bound’的思路。⑤ 与 KV 压缩的关系——decode 的瓶颈是读 KV，故 MQA/GQA/MLA/KV 量化直接改善 TPOT。⑥ 面试要点——被问’prefill 与 decode 的区别’，应给出’并行度（L vs 1）+ 瓶颈（算力 vs 带宽）+ 指标（TTFT vs TPOT）+ 优化方向‘四维对比，并说明’chunked prefill / PD 分离’的工程动机；这是推理系统类问题的核心考点。
⚙️ 查看英文落地权衡分析 (English Systems & Trade-offs)
Deep Dive & Engineering Trade-offs: ① Interference in Co-located Serving: When prefill requests (bursty, high compute, monopolizing GPU cores) and decode requests (latency-sensitive, low compute, high bandwidth) share the same GPU, prefill execution causes massive latency spikes (TPOT jitter) for ongoing decode requests. ② Mitigation: Chunked Prefill: Splitting long prompt prefills into smaller chunks (e.g., 512 tokens) and co-scheduling them alongside decode batches to smooth out GPU execution. ③ Architectural Evolution: PD Disaggregation: Physically separating prefill instances and decode instances onto dedicated GPU clusters, transferring KV caches over high-speed networks (InfiniBand/PCIe Gen5) to achieve optimal hardware utilization. ④ Batching Impact: Increasing batch size during decode amortizes weight loading across multiple requests, converting GEMV toward GEMM and boosting throughput at the cost of higher latency. ⑤ Interview Strategy: Frame the answer around the Roofline Model, contrasting TTFT vs. TPOT metrics and highlighting system-level scheduling solutions like vLLM and TensorRT-LLM.
五、常见面试避坑陷阱 (Common Pitfalls & Traps)
- ⚠️ 把 prefill 与 decode 的瓶颈当作同类
- ⚠️ 忽略 TTFT 与 TPOT 的区分
English Pitfalls:
– Assuming both prefill and decode phases are compute-bound or that adding more compute always speeds up decoding
– Failing to distinguish TTFT from TPOT when discussing latency SLAs
– Ignoring the severe GPU resource contention caused by running prefill and decode concurrently on the same device
六、高频深度面试追问与预测 (Follow-Up Questions)
- 为什么 prefill 与 decode 需要不同的 batch 策略？
- How does Chunked Prefill improve decode latency jitter?
- PD 分离的动机是什么？
- Under what conditions does PD Disaggregation justify the network overhead of KV cache transfer?
七、知识图谱对齐 (Knowledge Graph Anchor)
- 🔗 关联底层卡片：
KV Cache 显存占用公式、Prefill/Decode 阶段与 PagedAttention(KV Cache Memory, Prefill/Decode & PagedAttention) - 🗺️ 知识图谱模块：
AI 基础设施工程导图
🔬 算法科学家与机器学习深度考察全量题库 (Science Depth)
本题收录于 TalentMe 算法科学家深度考察真题库 (Science Depth)。全库共 856 道硬核考点，深度覆盖数学统计、经典ML、深度学习、Transformer、大语言模型、多模态、推荐系统与 MLOps。支持 Jev 面经智能匹配、一键离线单文件 HTML 手册导出并直连 Obsidian 本地记忆。