所属模块：
M8 · 系统架构、MLOps 与工程实战 (ML Systems, Engineering & Research)| 专题分类：可靠性与降级 (Reliability & Graceful Degradation)| 难度等级：Medium
一、核心一句话结论 (One-Sentence Summary)
由峰值 QPS × 单请求资源 × 冗余系数估算容量，用负载/压力/浸泡测试验证，并保留 headroom 应对突发与故障切换。
Capacity planning calculates required cluster resources from peak traffic projections, per-request resource footprints, and N+1 redundancy buffers, validating stability via load, stress, and long-duration soak testing to operate strictly below the queuing latency knee-of-curve.
二、核心考点要义 (Key Insights)
- 📌 需求估算——峰值 QPS（含促销/突发）、每请求资源（CPU/GPU/显存/带宽）
- 📌 冗余系数——为突发、故障切换（N+1）、滚动发布留余量（常见 20%-50%）
- 📌 压测类型——负载测试（预期峰值）、压力测试（找拐点）、浸泡测试（长时间稳定性）、尖峰测试
- 📌 拐点与排队——吞吐随并发上升到拐点后延迟剧增，容量应设在拐点前
- 📌 成本-可靠性权衡——headroom 越大越可靠但越贵，需按 SLO 与故障模型定量
English Insights:
– Demand & supply estimation: Peak QPS projections (organic + promotional surges) $times$ Per-request resource cost (GPU FLOPs, VRAM, CPU, bandwidth) $times$ Redundancy multiplier ($N+1$, multi-AZ).
– Queuing theory & knee-of-curve: Little’s Law ($L = lambda W$) and M/M/1 queuing curves dictate that latency approaches infinity as utilization nears 100%; systems must operate below the throughput saturation knee ($pprox 70%$ utilization).
– Testing taxonomy: Load testing (validating SLAs at expected peak), Stress testing (finding the breaking inflection point), Soak testing (detecting memory leaks over 24-48h), and Spike testing.
三、核心数学原理与机理推导 (Mathematical Principles & Derivation)
$$text{capacity}=frac{text{peak QPS}cdot r_{text{req}}cdot(1+m)}{r_{text{node}}},qquad text{headroom}=1-frac{text{peak}}{text{provisioned}}$$
数学机理：容量规划——(1) 需求侧——(a) 峰值 QPS——含日常峰、促销峰、突发（用历史分位数 + 业务预测）；(b) 每请求资源 r_req——CPU/GPU 时间、显存、内存、带宽；(c) 请求分布——不同接口/模型资源差异大，需分类。(2) 供给侧——(a) 单节点容量 r_node——由压测测定（给定 SLO 下的最大 QPS）；(b) 节点数——capacity = peak·r_req·(1+m) / r_node。(3) 冗余系数 m——(a) 突发余量——应对超预期流量；(b) 故障切换——N+1（一个节点挂了其余承担）；多可用区（一个 AZ 挂了另一 AZ 承担）；(c) 滚动发布——发布期间部分节点不可用；(d) 常见取值——20%-50%，关键系统更高。(4) 排队论视角——(a) 利用率与延迟——排队延迟随利用率非线性上升（接近 1 时爆炸）；(b) 利特尔法则（Little’s Law）——L = λW（在制请求数 = 到达率 × 停留时间）；(c) 结论——容量不应按 100% 利用率设计，而应留 headroom（如 70% 目标利用率）。(5) 压测类型——(a) 负载测试——在预期峰值下验证 SLO；(b) 压力测试——逐步加压找拐点（吞吐不再上升、延迟剧增）；(c) 浸泡测试（soak）——长时间运行检测内存泄漏、连接泄漏、缓存增长；(d) 尖峰测试（spike）——瞬时高压测试弹性与限流；(e) 故障注入——杀掉节点验证降级与切换。(6) 拐点（knee）——(a) 吞吐随并发上升到某点后饱和，继续加压延迟剧增；(b) 容量设在拐点前（如拐点的 70%）；(c) 拐点由资源瓶颈（CPU/GPU/带宽/连接池）决定。(7) 瓶颈定位——(a) 用资源利用率找瓶颈（哪个资源先到 100%）；(b) 常见瓶颈——GPU 显存/算力、CPU、连接池、DB、网络带宽。(8) 成本权衡——(a) headroom 越大越可靠但越贵（固定成本）；(b) 定量——按 SLO 与故障模型（允许的不可用时间）算所需冗余；(c) 弹性伸缩——云上用自动扩缩容按需调整（但冷启动有延迟，需预热/预留）。(9) 持续——(a) 流量增长与代码变更需重新评估容量；(b) 定期压测（而非一次性）。与其他问题的关系——(a) 与限流熔断（保护容量）；(b) 与延迟分解（拐点由瓶颈决定）；(c) 与成本优化（headroom 即成本）；(d) 与可靠性（冗余）。度量——(a) 峰值利用率与 headroom；(b) 拐点位置；(c) SLO 达标率；(d) 单位容量成本。
📖 查看英文严格数学推导 (English Mathematical Derivation)
Mathematical Foundations & Queuing Dynamics:
(1) Capacity Sizing Equation:
Let $Q_{text{peak}}$ be projected peak requests/sec, $r_{text{req}}$ be compute duration per request on standard hardware, and $C_{text{node}}$ be single-node request capacity under latency SLAs. Total required node count is:
$$N_{text{nodes}} = leftlceil frac{Q_{text{peak}} times r_{text{req}}}{C_{text{node}}} times (1 + m_{text{surge}}) times (1 + m_{text{AZ}}) rightrceil + N_{text{buffer}}$$
– $m_{text{surge}}$: Traffic surge margin (typically $20% – 40%$).
– $m_{text{AZ}}$: Multi-Availability Zone loss tolerance (e.g., $1.5times$ provisioned across 3 AZs so losing 1 AZ retains $100%$ capacity).
– $N_{text{buffer}}$: Headroom for rolling deployments ($N+1$ or $N+2$).
(2) Queuing Theory & The Latency Knee-of-Curve:
– Little’s Law: $L = lambda cdot W$ (Concurrency = Arrival Rate $times$ Residence Time).
– M/M/1 Queuing Delay Model:
For arrival rate $lambda$ and service rate $mu$, utilization is $rho = frac{lambda}{mu}$. Expected total response latency is:
$$W = frac{1}{mu(1 – rho)}$$
– The Saturation Knee: As utilization $rho to 1.0$, latency $W to infty$ non-linearly.
– At $rho = 0.5$, normalized delay is $2.0times$.
– At $rho = 0.8$, normalized delay is $5.0times$.
– At $rho = 0.95$, normalized delay is $20.0times$.
– Engineering Golden Rule: Size cluster capacity so that peak traffic corresponds to $rho le 0.70$, safely positioned before the exponential hockey-stick latency explosion.
(3) Stress Testing Taxonomy:
– Load Testing: Runs constant traffic at $1.0times Q_{text{peak}}$ to verify P99 latency and error budgets.
– Stress / Step-up Testing: Linearly ramps traffic ($1.0times to 2.5times Q_{text{peak}}$) until error rates spike to locate the precise system breaking point (the knee).
– Soak Testing: Sustains $0.8times Q_{text{peak}}$ continuously for 24 to 72 hours; uncovers slow memory leaks, connection pool exhaustion, file descriptor leaks, and JVM/CUDA fragmentation.
– Spike Testing: Instantaneously steps traffic from $0.1times$ to $2.0times$ in 5 seconds to validate autoscaling latency, rate limiters, and queue buffering.
四、工业级落地权衡与工程考量 (Industrial Trade-offs)
深度剖析与工程权衡：① 容量应设在拐点前——面试中能解释’吞吐拐点’是深度理解的标志。② 排队延迟随利用率非线性上升——不能按 100% 利用率设计。③ 冗余需覆盖故障切换——N+1 与多 AZ 都消耗容量。④ 浸泡测试查泄漏——短时压测查不出内存/连接泄漏。⑤ headroom 是成本与可靠性的权衡——需定量而非拍脑袋。⑥ 弹性伸缩有冷启动——需预热或预留容量。⑦ 面试要点——被问怎么定容量，应给出’需求（峰值 QPS × 每请求资源）+ 冗余（突发/N+1/多 AZ/发布）+ 压测（负载/压力/浸泡/尖峰）找拐点 + headroom 定量 + 弹性与预热‘；能指出拐点与排队非线性是深度理解的标志。
⚙️ 查看英文落地权衡分析 (English Systems & Trade-offs)
In-Depth Analysis & Engineering Trade-offs: ① Designing for 100% utilization guarantees production outages—queuing theory proves that pushing server utilization to near 100% causes infinite queuing delay; targeting $65-70%$ peak utilization provides the necessary buffer to absorb natural Poisson traffic clustering. ② Soak testing catches the deadliest bugs—short 10-minute load tests never reveal slow PyTorch CUDA memory leaks, growing Redis connection pools, or disk filling with un-rotated logs; long-duration soak testing is mandatory before major launches. ③ Multi-AZ redundancy math—hosting across 2 AZs requires $100%$ excess capacity per AZ ($2.0times$ total) to survive an AZ outage; hosting across 3 AZs requires only $50%$ excess capacity per AZ ($1.5times$ total), delivering both higher resilience and superior cost efficiency. ④ Synthetic traffic vs. Replayed production traffic—synthetic load testing generated by simplistic scripts often fails to mimic realistic multi-modal token length distributions (e.g., testing LLMs with uniform 50-token inputs); stress testing must replay sanitized production traffic traces to exercise true memory bandwidth and prefill boundaries. ⑤ Autoscaling is not a substitute for baseline headroom—deep learning container cold starts take minutes; autoscaling cannot save a cluster hit by an instant $3times$ surge; static baseline capacity must absorb the initial spike while autoscalers spin up auxiliary pods. ⑥ Interview takeaway—formalize the sizing equation, derive the M/M/1 queuing latency explosion $W = 1/(mu(1-rho))$, explain why systems must operate below the $rho=0.70$ knee, and contrast Load, Stress, and Soak testing.
五、常见面试避坑陷阱 (Common Pitfalls & Traps)
- ⚠️ 按 100% 利用率设计容量（无 headroom）
- ⚠️ 只做短时压测（漏掉内存/连接泄漏）
English Pitfalls:
– Sizing cluster capacity based on 100% utilization, ensuring that any minor traffic variation triggers infinite queuing latency explosions.
– Conducting only short 15-minute load tests, missing insidious memory leaks and connection leaks that trigger outages after 24 hours of operation.
– Relying entirely on reactive autoscaling to absorb spiky flash sales without pre-provisioning headroom, causing widespread timeouts during pod cold starts.
六、高频深度面试追问与预测 (Follow-Up Questions)
- 为什么容量要设在吞吐拐点之前？
- How does Little’s Law ($L = lambda W$) calculate the required concurrency worker pool depth for a model serving cluster?
- N+1 冗余与多可用区如何影响容量估算？
- Why does a 3-Availability-Zone deployment architecture reduce cloud redundancy expenditure compared to a 2-AZ deployment?
七、知识图谱对齐 (Knowledge Graph Anchor)
- 🔗 关联底层卡片：
工业级可靠性保障：熔断限流 (Circuit Breaker)、自适应退避与分级降级兜底(Production Reliability: Circuit Breakers, Fallbacks & Shedding) - 🗺️ 知识图谱模块：
AI 基础设施工程导图
🔬 算法科学家与机器学习深度考察全量题库 (Science Depth)
本题收录于 TalentMe 算法科学家深度考察真题库 (Science Depth)。全库共 856 道硬核考点，深度覆盖数学统计、经典ML、深度学习、Transformer、大语言模型、多模态、推荐系统与 MLOps。支持 Jev 面经智能匹配、一键离线单文件 HTML 手册导出并直连 Obsidian 本地记忆。