所属模块：
M8 · 系统架构、MLOps 与工程实战 (ML Systems, Engineering & Research)| 专题分类：推理服务与部署 (Inference Serving & Deployment)| 难度等级：Easy
一、核心一句话结论 (One-Sentence Summary)
在线（低延迟同步）、批处理（高吞吐异步）、流式（持续输入）、边缘（本地）；以及’级联/多模型’的组合。
Model serving spans four primary paradigms—Online synchronous, Offline batch, Streaming event-driven, and Edge on-device—often combined into multi-stage cascades (such as offline candidate pre-computation paired with online real-time re-ranking) to balance latency, throughput, cost, and privacy.
二、核心考点要义 (Key Insights)
- 📌 在线：同步请求-响应（低延迟、高可用）
- 📌 批处理：离线大批量（高吞吐、可等待）
- 📌 流式：持续输入（事件驱动）；边缘：本地推理（低延迟/隐私）
English Insights:
– Four core deployment paradigms: Online/Real-time (synchronous, sub-second latency SLAs), Offline Batch (asynchronous high-throughput scoring), Streaming/Nearline (continuous event-driven inference), and Edge/On-device (local low-latency and privacy-preserving).
– Hybrid & cascade topologies: Offline batch pre-generation of recommendation candidates combined with online real-time fine-ranking; edge coarse filtering coupled with cloud deep inference.
– Selection criteria: Latency tolerance (milliseconds vs. hours), data security/privacy boundaries, throughput scale, compute resource constraints, and deployment cost.
三、核心数学原理与机理推导 (Mathematical Principles & Derivation)
$$text{forms}: text{online}, text{batch}, text{streaming}, text{edge};qquad text{choose by latency/throughput}$$
数学机理：四种部署形态——(1) 在线（online / real-time）——(a) 模式——同步请求-响应（用户等待结果）；(b) 要求——低延迟（毫秒到秒）、高可用（99.9%+）、自动扩缩容；(c) 场景——推荐/搜索/风控/对话；(d) 实现——模型服务（Triton/TorchServe/vLLM）+ 负载均衡 + 自动扩缩容。(2) 批处理（batch / offline）——(a) 模式——离线处理大批数据（无需等待）；(b) 特点——高吞吐（可用全部资源）、可等待（小时级）、成本低（按需启动）；(c) 场景——离线推荐（预计算）、批量评分、数据标注；(d) 实现——Spark/分布式推理（GPU 批处理）。(3) 流式（streaming）——(a) 模式——持续消费事件流，逐条/微批推理；(b) 场景——实时风控（交易流）、实时监控、IoT；(c) 实现——Flink/Kafka + 模型服务（低延迟）。(4) 边缘（edge / on-device）——(a) 模式——模型部署在本地设备（手机/IoT/车机）；(b) 优点——(i) 极低延迟（无网络）；(ii) 隐私（数据不出设备）；(iii) 离线可用；(c) 约束——(i) 算力/内存/功耗受限（需小模型 + 量化 + 剪枝）；(ii) 更新难（需 OTA）；(iii) 碎片化（设备异构）；(d) 场景——键盘输入预测、语音唤醒、图像处理。(5) 混合/级联——(a) 多级——’边缘粗筛 + 云端精算’；(b) 模型级联（小模型快速筛选 + 大模型精算）；(c) 批 + 在线——’离线预计算候选 + 在线精排’（推荐系统的常见架构）。选择依据——(a) 延迟需求（毫秒→边缘/在线；小时→批处理）；(b) 吞吐需求（高吞吐→批处理）；(c) 隐私（敏感数据→边缘）；(d) 成本（批处理最便宜）；(e) 更新频率（高频更新→云端）；(f) 设备约束（端侧→小模型）。与其他问题的关系——(a) 与’推理引擎’（在线服务的实现）；(b) 与’自动扩缩容’（在线服务的弹性）；(c) 与’成本优化’（批处理最省）。实践建议——(a) 按延迟与吞吐需求选形态；(b) 推荐系统用’离线预计算 + 在线精排’（成本与延迟的平衡）；(c) 边缘用’小模型 + 量化’；(d) 批处理优先（若可等待——成本最低）；(e) 级联（成本-质量平衡）。度量——(a) 延迟/吞吐；(b) 成本；(c) 可用性；(d) 资源利用率。
📖 查看英文严格数学推导 (English Mathematical Derivation)
Taxonomy of Deployment Paradigms:
(1) Online Synchronous Serving:
– Execution Model: Client issues synchronous RPC (gRPC / HTTP REST) and blocks waiting for immediate response.
– SLA Constraints: Strict latency budgets (e.g., $10 – 50text{ ms}$ for recommendation ranking; $< 2text{ s}$ TTFT for interactive LLMs); 99.9%+ availability.
– Infrastructure: Model serving frameworks (NVIDIA Triton, vLLM, TorchServe) hosted on auto-scaling GPU clusters with load balancers and horizontal pod autoscalers (HPA).
(2) Offline Batch Serving:
– Execution Model: Scheduled jobs (Airflow DAGs, Spark jobs) processing millions of records asynchronously from distributed object stores.
– Characteristics: Latency-insensitive (hours/days turnaround); maximizes vectorized GPU throughput via massive batch sizes; compute instances spin up on demand and terminate upon completion.
– Use Cases: Daily user embedding generation, batch churn scoring, catalog text classification, offline dataset labeling.
(3) Streaming / Nearline Serving:
– Execution Model: Event-driven processors (Apache Flink, Kafka Streams) consuming continuous message streams, performing micro-batched or per-event inference.
– Characteristics: Medium latency budget ($100text{ ms} – 5text{ s}$); decouple user-facing HTTP request paths from heavy model scoring.
– Use Cases: Real-time fraud detection, financial transaction scoring, live content moderation, IoT sensor telemetry.
(4) Edge / On-Device Serving:
– Execution Model: Executing inference locally on client hardware (smartphones, IoT microcontrollers, automotive ECUs).
– Benefits: Zero network latency, zero cloud hosting server costs, offline availability, and absolute privacy (user data never leaves the device).
– Hardware Constraints: Stringent thermal, power, and memory limits; requires aggressive quantization (INT4/INT8 via CoreML, ONNX Runtime, llama.cpp, TensorRT-Edge).
四、工业级落地权衡与工程考量 (Industrial Trade-offs)
深度剖析与工程权衡：① ‘推荐系统用离线预计算 + 在线精排’——这是工业界的经典架构；面试中能指出是深度理解的标志。② ‘批处理成本最低’——若业务可等待，优先批处理。③ ‘边缘的约束’——算力/内存/功耗 + 更新难 + 碎片化。④ ‘级联（小模型+大模型）’——成本-质量的平衡。⑤ ‘隐私驱动边缘’——敏感数据不出设备。⑥ 面试要点——被问’模型怎么部署’，应给出’四形态（在线/批处理/流式/边缘）+ 选择依据（延迟/吞吐/隐私/成本）+ 混合与级联‘；能指出’离线预计算+在线精排’是深度理解的标志。
⚙️ 查看英文落地权衡分析 (English Systems & Trade-offs)
In-Depth Analysis & Engineering Trade-offs: ① The industrial hybrid pattern: Offline pre-computation + Online fine-ranking—calculating full deep recommendations across 100 million items in real-time is computationally impossible within 30ms; systems pre-compute candidate vectors in daily batch jobs, store them in Redis/ANN indexes, and invoke deep neural networks online only for the top 500 candidates. ② Batch serving is exponentially cheaper—if business requirements can tolerate a 4-hour delay, deploying an online service wastes massive continuous cluster idle costs; batch jobs running on spot instances reduce serving costs by 80%. ③ Edge serving operational hurdles—edge models cannot be instantly rolled back or hot-swapped (requiring app store releases or OTA firmware pushes); fragmented client hardware platforms demand extensive cross-compilation testing. ④ Model cascades (Small model filter -> Big model compute)—routing 100% of user queries to a 70B parameter LLM is cost-prohibitive; systems deploy a lightweight 1B model on CPU/edge to filter out 70% of benign or simple queries, routing only complex requests to cloud GPU clusters. ⑤ Streaming nearline as a protective buffer—placing heavy risk/fraud inference on asynchronous Kafka topics protects front-end checkout services from crashing when model clusters experience temporary latency spikes. ⑥ Interview takeaway—systematically compare the four paradigms across latency, cost, and complexity, detail the hybrid ‘offline pre-compute + online rank’ pattern, and articulate model cascade filtering.
五、常见面试避坑陷阱 (Common Pitfalls & Traps)
- ⚠️ 为可等待的场景用在线服务（成本高）
- ⚠️ 边缘部署不考虑算力约束
English Pitfalls:
– Deploying high-cost online synchronous serving infrastructure for analytical workloads that users inspect only once daily.
– Ignoring thermal, battery, and memory constraints when porting server-side deep neural networks to client edge devices.
– Tightly coupling synchronous microservices to heavy deep learning inference without timeout fallbacks, causing cascading user-facing 504 gateway timeouts.
六、高频深度面试追问与预测 (Follow-Up Questions)
- 什么场景用批处理？
- How do enterprise recommendation systems partition work between offline batch candidate generation and online real-time re-ranking?
- 边缘部署的约束？
- What quantization and operator compilation techniques enable running 7B parameter LLMs locally on mobile hardware?
七、知识图谱对齐 (Knowledge Graph Anchor)
- 🔗 关联底层卡片：
大模型高并发推理服务架构：连续批处理 (Continuous Batching) 与 Triton(High-Concurrency Serving: Continuous Batching, vLLM & Triton) - 🗺️ 知识图谱模块：
AI 基础设施工程导图
🔬 算法科学家与机器学习深度考察全量题库 (Science Depth)
本题收录于 TalentMe 算法科学家深度考察真题库 (Science Depth)。全库共 856 道硬核考点，深度覆盖数学统计、经典ML、深度学习、Transformer、大语言模型、多模态、推荐系统与 MLOps。支持 Jev 面经智能匹配、一键离线单文件 HTML 手册导出并直连 Obsidian 本地记忆。