所属模块：
M7 · 检索、排序与推荐系统 (Retrieval, Ranking & RecSys)| 专题分类：离线评估与 OPE (Offline Evaluation & OPE)| 难度等级：Easy
一、核心一句话结论 (One-Sentence Summary)
IPS 用’1/倾向’加权历史奖励，得到无偏估计；但倾向很小时权重爆炸 → 方差极大（需截断/自归一化）。
The Inverse Propensity Scoring (IPS) estimator uses importance weights (pi_new / pi_old) to achieve unbiased offline policy evaluation, but suffers from extreme variance when logging probabilities are tiny; clipping and self-normalization stabilize estimation.
二、核心考点要义 (Key Insights)
- 📌 用 π_new/π_old 的比值加权历史奖励
- 📌 无偏（若 π_old 有正性）
- 📌 方差大：π_old 小时权重爆炸
English Insights:
– Importance sampling foundation: Reweights logged rewards by the likelihood ratio between candidate policy pi_new and historical logging policy pi_old.
– Proof of unbiasedness: Re-weighting cancels out the sampling distribution of the logging policy in expectation.
– Variance explosion pathology: When the old policy assigns tiny probability to an action that the new policy favors, weights explode, destabilizing the estimator.
– Mitigation techniques: Weight clipping (Clipped-IPS) and Self-Normalized IPS (SNIPS).
三、核心数学原理与机理推导 (Mathematical Principles & Derivation)
$$hat V_{text{IPS}}=frac1nsum_{i=1}^nfrac{pi_{text{new}}(a_i|x_i)}{pi_{text{old}}(a_i|x_i)}r_i$$
数学机理：IPS（Inverse Propensity Scoring） 的推导——(1) 目标——估计新策略 πnew 的期望奖励 V(π_new)=E{x∼p(x), a∼πnew}[r(x,a)]。(2) 关键恒等式——利用’重要性采样’：E{a∼πnew}[r] = E{a∼πold}[(π_new(a|x)/π_old(a|x))·r]；为什么——因为 E{a∼πold}[w(a)·r] = Σ_a π_old(a|x)·(π_new(a|x)/π_old(a|x))·r(a) = Σ_a π_new(a|x)·r(a) = E{a∼π_new}[r]。(3) 估计式——V̂_IPS=(1/n)Σ_i [π_new(a_i|x_i)/π_old(a_i|x_i)]·r_i，其中 (x_i, a_i, r_i) 是历史数据（a_i 由 π_old 选择）。(4) 性质——(a) 无偏（若 π_old(a|x)>0 对所有 π_new 可能选的 a）；(b) 只需 π_old 已知（不需模型）。(5) 方差问题（核心缺陷）——(a) 权重 w=π_new/π_old；若 π_old(a|x) 很小（该物品很少被旧策略展示），则 w 很大；(b) 后果——(i) 单个样本可能主导估计（方差极大）；(ii) 极端情况（π_old→0）权重爆炸（估计不可用）；(c) 量化——IPS 的方差 ∝ E[w²]，而 w² 在 π_old 小时极大；故’倾向小的样本’是方差的主要来源；(d) 有效样本量——ESS=(Σw)²/Σw²；若 ESS ≪ n，说明’少数样本主导’（估计不可靠）。缓解手段——(1) 权重截断（clipping）——把 w 截断到 [0, M]（如 M=10）；代价——引入偏差（但方差降低）；权衡——M 小则偏差大、M 大则方差大（经典的偏差-方差权衡）。(2) 自归一化（self-normalized IPS，SNIPS）——V̂_SNIPS=Σ(w_i·r_i)/Σw_i；优点——(a) 有界（在 [min r, max r] 内）；(b) 对’权重整体缩放’不敏感；(c) 实践中方差更小；缺点——轻微有偏（但’一致性’仍成立）。(3) 混合（shrinkage）——把 w 向 1 收缩（w’=α·w+(1−α)·1）；权衡——降低方差但引入偏差。(4) 更好的倾向估计——若 π_old 未知，用模型估计（’倾向模型’）；注意——倾向估计的误差会传播。(5) DR（Doubly Robust）——用模型作为基线（见下一题）；优点——(a) 方差更低（因为模型解释了大部分变异）；(b) 双重稳健（模型或倾向之一正确即可无偏）。(6) 限制支持集——只在’π_new 与 π_old 都有正概率’的动作集上评估（避免’外推’）。与其他问题的关系——(a) 与’IPS 在 LTR 中的去偏’（位置偏置）；(b) 与’DR’（下一题）；(c) 与’权重截断’（下一题）。实践建议——(a) 用 SNIPS（比原始 IPS 更稳）；(b) 权重截断（M=5~20）；(c) 监控 ESS（有效样本量）；(d) 检查正性（π_old 是否有 0）；(e) 优先用 DR（方差更低）；(f) 离线只做粗筛。度量——(a) OPE 估计 vs 在线 A/B（偏差）；(b) ESS；(c) 权重的分布（最大权重）；(d) 截断阈值的影响。
📖 查看英文严格数学推导 (English Mathematical Derivation)
Mathematical Formulation & Proof: IPS Mechanics & Variance Analysis.
(1) The IPS Estimator Formulation:
Let $mathcal{D} = {(x_i, a_i, r_i, p_i)}_{i=1}^N$ be logged under policy $pi_0$, where $p_i = pi_0(a_i mid x_i)$. The IPS estimator for new policy $pi_1$ is:
$$hat{V}_{text{IPS}}(pi_1) = frac{1}{N} sum_{i=1}^N frac{pi_1(a_i mid x_i)}{pi_0(a_i mid x_i)} cdot r_i = frac{1}{N} sum_{i=1}^N w_i cdot r_i$$
where $w_i = frac{pi_1(a_i mid x_i)}{pi_0(a_i mid x_i)}$ is the importance weight.
(2) Proof of Unbiasedness:
Assuming Common Support ($pi_1(a mid x) > 0 implies pi_0(a mid x) > 0$):
$$mathbb{E}_{mathcal{D}}[hat{V}_{text{IPS}}(pi_1)] = mathbb{E}_{x sim P(x)} mathbb{E}_{a sim pi_0(a mid x)} left[ frac{pi_1(a mid x)}{pi_0(a mid x)} r(x, a) right]$$$$= int P(x) left( sum_{a in mathcal{A}} pi_0(a mid x) frac{pi_1(a mid x)}{pi_0(a mid x)} r(x, a) right) dx = int P(x) left( sum_{a in mathcal{A}} pi_1(a mid x) r(x, a) right) dx = V(pi_1)$$
The estimator is mathematically unbiased: $mathbb{E}[hat{V}_{text{IPS}}(pi_1)] = V(pi_1)$.
(3) The Variance Explosion Problem:
The variance of the IPS estimator depends quadratically on importance weights:
$$text{Var}(hat{V}_{text{IPS}}) = frac{1}{N} mathbb{E}_{pi_0}left[ left( frac{pi_1(a mid x)}{pi_0(a mid x)} right)^2 r^2(x, a) right] – frac{V(pi_1)^2}{N}$$
If for some sample $i$, the old policy rarely took action $a_i$ (e.g., $pi_0 = 0.001$) while the new policy takes it frequently ($pi_1 = 0.50$), the weight explodes: $w_i = frac{0.50}{0.001} = 500$. A single observation dominates the entire summation, driving variance to infinity.
(4) Mitigation Variants:
– Clipped IPS: Truncates weights at ceiling $M$ ($w_i^{text{clip}} = min(w_i, M)$). Bounded variance at the expense of introducing slight asymptotic bias.
– Self-Normalized IPS (SNIPS / Hájek Estimator):
$$hat{V}_{text{SNIPS}}(pi_1) = frac{sum_{i=1}^N w_i cdot r_i}{sum_{i=1}^N w_i}$$
Normalizing by empirical weight sum guarantees the estimate remains strictly bounded within the valid reward range $[0, 1]$.
四、工业级落地权衡与工程考量 (Industrial Trade-offs)
深度剖析与工程权衡：① ‘IPS 无偏但方差大’是核心权衡——故需截断/自归一化；面试中能写出 IPS 公式并解释方差来源是深度理解的标志。② ‘ESS 是实用诊断——ESS ≪ n 说明估计不可靠。③ ‘SNIPS 有界且更稳’——实践中常用；虽有轻微偏差但方差优势明显。④ ‘权重截断是偏差-方差权衡’——M 小则偏差大、M 大则方差大。⑤ ‘正性假设’是 IPS 的前提——π_old 为 0 的动作无法估计（外推失败）。⑥ 面试要点——被问’IPS 是什么’，应写出估计式 + 无偏性推导（重要性采样）+ 方差问题（权重爆炸）+ 缓解（截断/SNIPS/DR）；能指出’ESS 诊断’是深度理解的标志。
⚙️ 查看英文落地权衡分析 (English Systems & Trade-offs)
In-Depth Analysis & Engineering Trade-offs: ① The Bias-Variance Trade-Off in Clipped IPS—as clipping threshold $M$ decreases, variance drops dramatically, but bias increases; setting $M in [10, 50]$ is standard in industrial counterfactual evaluation. ② SNIPS vs. Standard IPS—standard IPS is unbiased for finite samples, but can output absurd values (e.g., predicting an average CTR of $140%$ if an explosive weight hits a clicked sample); SNIPS is slightly biased for finite $N$ but asymptotically consistent ($N to infty$), strictly bounded in $[0, 1]$, and empirically exhibits substantially lower Mean Squared Error (MSE). ③ Logging policy design—if the production logging policy $pi_0$ is purely uniform random, $p_i = 1/K$ is constant, eliminating weight variance at the cost of severely degrading user experience during data collection; systems use $epsilon$-greedy logging or softmax logging around the production model to balance user experience with offline evaluation fidelity. ④ Slate recommendation variance explosion—for a slate of 10 items, the importance weight is the product of marginal weights: $W = prod_{k=1}^{10} w_k$; joint weights easily reach $10^{15}$, making naive slate-level IPS completely useless; systems apply independent slate IPS or top-k prefix importance sampling. ⑤ Effective Sample Size (ESS)—monitoring $text{ESS} = frac{(sum w_i)^2}{sum w_i^2}$ alerts engineers when an evaluation is dominated by a few outlier weights. ⑥ Interview takeaway—write out the IPS formula, prove its unbiasedness via the cancellation of $pi_0(a mid x)$, explain the quadratic variance term $w_i^2$, and present Clipped IPS and SNIPS as the standard solutions.
五、常见面试避坑陷阱 (Common Pitfalls & Traps)
- ⚠️ 用原始 IPS 不截断（方差爆炸）
- ⚠️ 不检查正性（π_old 为 0 时外推失败）
English Pitfalls:
– Deploying unclipped IPS on datasets where logging probabilities pi_0 are small, allowing a single weight spike to produce nonsensical evaluation estimates.
– Applying single-action IPS directly to multi-item recommendation slates without factoring slate independence, triggering exponential product variance explosion.
– Evaluating IPS on historical logs where the logging policy pi_0 assigned exact zero probability to certain candidate actions, violating the common support condition.
六、高频深度面试追问与预测 (Follow-Up Questions)
- 为什么 IPS 是无偏的？
- Why is the Self-Normalized IPS (SNIPS) estimator guaranteed to output values bounded within the true reward interval [0, 1]?
- 权重截断的偏差-方差权衡？
- How does Effective Sample Size (ESS) quantify the statistical reliability of an off-policy importance sampling evaluation?
七、知识图谱对齐 (Knowledge Graph Anchor)
- 🔗 关联底层卡片：
推荐系统离线评估与离线策略评估 (OPE)：逆倾向得分 (IPS) 与重要性采样(Off-Policy Evaluation (OPE): IPS, Doubly Robust & Calibration) - 🗺️ 知识图谱模块：
机器学习工程师高频考点导图
🔬 算法科学家与机器学习深度考察全量题库 (Science Depth)
本题收录于 TalentMe 算法科学家深度考察真题库 (Science Depth)。全库共 856 道硬核考点，深度覆盖数学统计、经典ML、深度学习、Transformer、大语言模型、多模态、推荐系统与 MLOps。支持 Jev 面经智能匹配、一键离线单文件 HTML 手册导出并直连 Obsidian 本地记忆。