所属模块:
M8 · 系统架构、MLOps 与工程实战 (ML Systems, Engineering & Research)| 专题分类:模型治理与风险 (Model Governance & Risk Management)| 难度等级:Easy
一、核心一句话结论 (One-Sentence Summary)
常见定义有人口统计均等(DP)、均等机会(EO)、均等赔率;多个定义通常不可同时满足(不可能定理),需按场景与法规选主指标并明确权衡。
Mathematical definitions of fairness—Demographic Parity, Equal Opportunity, Equalized Odds, and Predictive Rate Parity—cannot be simultaneously satisfied in the presence of unequal base rates (the Impossibility Theorem), requiring domain-specific objective selection and explicit trade-off governance.
二、核心考点要义 (Key Insights)
- 📌 人口统计均等(demographic parity)——各群体正例率相同,不看真实标签
- 📌 均等机会(equal opportunity)——各群体真阳性率(TPR)相同
- 📌 均等赔率(equalized odds)——各群体 TPR 与 FPR 都相同(更强)
- 📌 不可能定理——除退化情形外,多个公平定义不可同时满足(尤其基率不同时)
- 📌 选择依据——场景(谁受影响)、法规(如美国 EEOC 的 80% 规则)、以及明确记录的权衡
English Insights:
– Statistical definitions: Demographic Parity ($P(hat{Y}=1 mid A=0) = P(hat{Y}=1 mid A=1)$), Equal Opportunity (equal true positive rates across groups), Equalized Odds (equal TPR and FPR across groups), and Calibration within Groups (equal positive predictive value).
– The Impossibility Theorem (Kleinberg, Chouldechova): If the base rates $P(Y=1 mid A=0) ne P(Y=1 mid A=1)$, then a non-trivial predictor cannot simultaneously achieve equalized odds and calibration within groups.
– Regulatory & domain alignment: Selecting metrics based on legal doctrine (e.g., EEOC 80% four-fifths rule for disparate impact vs. Equal Credit Opportunity Act) and documenting irreducible Pareto trade-offs.
三、核心数学原理与机理推导 (Mathematical Principles & Derivation)
$$text{DP}: Pr[hat Y{=}1mid A{=}0]=Pr[hat Y{=}1mid A{=}1];quad text{EO}: Pr[hat Y{=}1mid Y,A]=Pr[hat Y{=}1mid Y,A’]$$
数学机理:主要公平性定义——设 A 为受保护属性,Y 为真实标签,Ŷ 为预测。(1) 人口统计均等(demographic parity / statistical parity)——(a) 定义——Pr[Ŷ=1 | A=0] = Pr[Ŷ=1 | A=1);(b) 含义——各群体被预测为正的比例相同;(c) 不看真实标签——故若真实基率不同,则必然导致某群体被错误对待(’公平’但可能牺牲精度);(d) 适用——正例率本身就是关注目标时。(2) 均等机会(equal opportunity)——(a) 定义——Pr[Ŷ=1 | Y=1, A=0] = Pr[Ŷ=1 | Y=1, A=1);(b) 含义——真阳性率(TPR/召回)在各群体相同;(c) 关注——’该被选中的都该被选中’(如招聘中合格者应被录取);(d) 允许 FPR 不同。(3) 均等赔率(equalized odds)——(a) 定义——Pr[Ŷ=1 | Y=y, A=0] = Pr[Ŷ=1 | Y=y, A=1) 对 y ∈ {0,1} 都成立;(b) 含义——TPR 与 FPR 都相同(比 EO 更强);(c) 即预测在给定真实标签下与受保护属性独立。(4) 预测率均等(predictive rate parity)——Pr[Y=1 | Ŷ=1, A] 相同(精确率相同);与 EO/均等赔率冲突(Chouldechova 的结果)。(5) 不可能定理(impossibility)——(a) Chouldechova / Kleinberg 结果——若各群体的基率(base rate)不同,则校准(calibration)+ 均等赔率不可同时满足(除非完美预测或随机);(b) 直觉——基率不同时,’校准’要求不同群体的风险分数含义一致,而’均等赔率’要求错误率一致,二者数学上冲突;(c) 结论——公平是多目标权衡,不存在’对所有定义都公平’的模型。(6) 其他度量——(a) 平等化赔率;(b) 条件使用准确率;(c) 个人公平(相似个体相似对待);(d) 反事实公平(改变受保护属性不改变预测)。(7) 缓解方法——(a) 预处理——重加权、数据增强、去偏表示;(b) 处理中——正则项(在损失中加公平约束)、对抗训练;(c) 后处理——按群体调整阈值(满足 EO/均等赔率)。(8) 选择与治理——(a) 按场景选主指标——招聘看 EO、信贷看预测率均等(与法规相关);(b) 法规——如美国 EEOC 的 four-fifths(80%)规则(通过率比 ≥ 0.8);(c) 记录权衡——明确选了什么、牺牲了什么、为什么。与其他问题的关系——(a) 与偏见来源与缓解;(b) 与高风险场景验证;(c) 与模型卡(披露分群指标);(d) 与合规(法规要求)。度量——(a) 各定义下的群体差异(ΔDP、ΔEO、ΔTPR、ΔFPR);(b) 精度-公平权衡曲线;(c) 与法规阈值(80% 规则)的比较。
📖 查看英文严格数学推导 (English Mathematical Derivation)
Mathematical Formalisms & The Impossibility Proof:
(1) Core Fairness Formulations:
Let $A in {0, 1}$ be a binary protected attribute, $Y in {0, 1}$ the ground-truth label, and $hat{Y} in {0, 1}$ the model prediction ($R in [0, 1]$ the predicted probability).
– 1. Demographic Parity (Statistical Parity):
$$P(hat{Y}=1 mid A=0) = P(hat{Y}=1 mid A=1)$$
Acceptance rate is invariant to group membership. Completely ignores ground-truth $Y$.
– 2. Equal Opportunity (Hardt et al.):
$$P(hat{Y}=1 mid Y=1, A=0) = P(hat{Y}=1 mid Y=1, A=1) quad (text{Equal TPR / Recall})$$
– 3. Equalized Odds:
$$forall y in {0, 1}: quad P(hat{Y}=1 mid Y=y, A=0) = P(hat{Y}=1 mid Y=y, A=1) quad (text{Equal TPR and FPR})$$
Equivalently, prediction $hat{Y}$ is conditionally independent of protected attribute $A$ given true label $Y$: $hat{Y} perp A mid Y$.
– 4. Predictive Rate Parity (Calibration by Group):
$$P(Y=1 mid R=r, A=0) = P(Y=1 mid R=r, A=1) = r quad forall r in [0, 1]$$
(2) The Impossibility Theorem (Kleinberg et al., Chouldechova 2017):
– Theorem: If the underlying base rates differ between groups, i.e., $p_0 = P(Y=1 mid A=0) ne P(Y=1 mid A=1) = p_1$, and predictions are imperfect (AUC $< 1.0$), then Calibration within Groups and Equalized Odds are mathematically incompatible.
– Algebraic Mechanism: By Bayes’ rule, the False Positive Rate can be expressed via positive predictive value (PPV $v$) and base rate $p$:
$$text{FPR} = frac{p}{1-p} cdot frac{1-v}{v} cdot text{TPR}$$
If PPV ($v$) is calibrated across groups and TPR is equalized, then whenever $p_0 ne p_1$, the term $frac{p}{1-p}$ forces $text{FPR}_0 ne text{FPR}_1$, directly violating Equalized Odds.
四、工业级落地权衡与工程考量 (Industrial Trade-offs)
深度剖析与工程权衡:① 不可能定理是核心——基率不同时公平定义冲突,不存在全公平的模型;面试中能指出这点是深度理解的标志。② DP 不看真实标签——基率不同时会导致错误对待。③ EO 关注 TPR、均等赔率关注 TPR+FPR——强度递增。④ 校准与均等赔率冲突——需明确取舍。⑤ 后处理调阈值可满足 EO——但会牺牲整体精度。⑥ 选主指标需结合场景与法规——并记录权衡。⑦ 面试要点——被问怎么保证公平,应给出’明确主定义(DP/EO/均等赔率)+ 承认不可能定理 + 按场景与法规选择 + 记录权衡 + 分群评估与缓解‘;能指出基率不同导致冲突是深度理解的标志。
⚙️ 查看英文落地权衡分析 (English Systems & Trade-offs)
In-Depth Analysis & Engineering Trade-offs: ① The Impossibility Theorem proves there is no universally fair model—attempting to satisfy all definitions simultaneously is mathematically impossible whenever underlying historical base rates diverge; teams must make an explicit, documented choice. ② Demographic Parity penalizes qualification-based selection when base rates differ—if group base rates differ, forcing equal positive rates forces either accepting lower-qualified candidates from one group or rejecting qualified candidates from another. ③ Calibration is essential for risk scoring, Equal Opportunity is essential for opportunity access—underwriting requires well-calibrated probabilities for pricing interest rates; employment screening prioritizes equal chance of selection for qualified applicants (Equal Opportunity). ④ Post-processing thresholds trade off global utility—setting group-specific thresholds $tau_0 ne tau_1$ to enforce Equal Opportunity reduces global classification accuracy and may face legal restrictions under anti-discrimination statutes against explicit race/gender scoring. ⑤ Fairness interventions span three tiers—Pre-processing (re-weighting/re-sampling data), In-processing (adversarial debiasing loss penalties), and Post-processing (threshold tuning). ⑥ Interview takeaway—write down the mathematical definitions of DP, EO, and Calibration, sketch the algebraic proof of why $p_0 ne p_1$ forces incompatibility, and explain how domain context dictates the optimal Pareto compromise.
五、常见面试避坑陷阱 (Common Pitfalls & Traps)
- ⚠️ 声称同时满足多个公平定义(忽略不可能定理)
- ⚠️ 用人口统计均等却不看真实基率
English Pitfalls:
– Claiming a model satisfies all algorithmic fairness metrics simultaneously, displaying complete ignorance of Kleinberg’s Impossibility Theorem.
– Enforcing Demographic Parity indiscriminately in high-stakes risk prediction where true base rates differ, distorting probability calibration and downstream utility.
– Applying group-specific post-processing thresholds in production environments without verifying whether disparate treatment statutes prohibit explicit group-conditioned decision rules.
六、高频深度面试追问与预测 (Follow-Up Questions)
- 为什么基率不同时公平定义会冲突?
- Why does enforcing Equalized Odds via group-specific thresholding often reduce global overall accuracy?
- 均等机会与均等赔率的区别是什么?
- How does the U.S. EEOC 80% (four-fifths) rule translate into an empirical Demographic Parity threshold?
七、知识图谱对齐 (Knowledge Graph Anchor)
- 🔗 关联底层卡片:
企业级模型治理体系:公平性偏差审计、可解释性 (SHAP) 与风险合规防线(Model Governance: Fairness Audit, Explainability (SHAP) & Risk) - 🗺️ 知识图谱模块:
AI 基础设施工程导图
🔬 算法科学家与机器学习深度考察全量题库 (Science Depth)
本题收录于 TalentMe 算法科学家深度考察真题库 (Science Depth)。全库共 856 道硬核考点,深度覆盖数学统计、经典ML、深度学习、Transformer、大语言模型、多模态、推荐系统与 MLOps。支持 Jev 面经智能匹配、一键离线单文件 HTML 手册导出并直连 Obsidian 本地记忆。