所属模块:
M1 · 数学与统计基础 (Mathematics & Statistics Fundamentals)| 专题分类:假设检验 (Hypothesis Testing)| 难度等级:Medium
一、核心一句话结论 (One-Sentence Summary)
① 拟合优度(分布是否匹配)② 独立性(两分类变量是否相关)③ 方差/比例的比较。
The Chi-Square test evaluates (1) Goodness-of-Fit (matching empirical counts to expected theoretical distributions), (2) Test of Independence (evaluating association between two categorical variables in contingency tables), and (3) Test of Homogeneity (comparing categorical proportions across multiple independent sub-populations).
二、核心考点要义 (Key Insights)
- 📌 拟合优度:自由度 = 类别数 − 1 − 估计参数数
- 📌 独立性:自由度 = (r−1)(c−1)
English Insights:
– Unified Test Statistic: $chi^2 = sum_{i} frac{(O_i – E_i)^2}{E_i} sim chi^2_nu$.
– Goodness-of-Fit: Tests if observed counts match a known distribution (e.g. SRM check in A/B testing has $text{df} = k – 1$).
– Contingency Table Independence: Evaluates $rtimes c$ matrix independence: $E_{ij} = frac{R_i C_j}{N}$ with $text{df} = (r-1)(c-1)$.
– Precondition: Expected cell count $E_i ge 5$ in at least 80% of cells (otherwise Fisher’s Exact Test must be used).
三、核心数学原理与机理推导 (Mathematical Principles & Derivation)
$$chi^2=sum_ifrac{(O_i-E_i)^2}{E_i}$$
三种用途:① 拟合优度检验——检验观测频数 Oᵢ 是否与某理论分布(如均匀、泊松)的期望频数 Eᵢ 一致;自由度 = 类别数 −1(若分布参数也从数据估计,还需减去估计的参数个数)。② 独立性检验(列联表)——检验两个分类变量是否独立:把观测频数与’独立假设下的期望频数’ Eᵢⱼ=(行和×列和)/总数 比较,自由度 =(r−1)(c−1)。这是 A/B 测试中 SRM(样本比例失配)检测的标准方法。③ 方差/比例的检验——对单个方差(相对理论值)或比例的比较(如’两组转化率是否相同’可用卡方或比例 z 检验,两者等价)。统计量构造:χ²=Σ(Oᵢ−Eᵢ)²/Eᵢ,在 H₀ 成立且期望频数足够大时渐近服从 χ² 分布。
📖 查看英文严格数学推导 (English Mathematical Derivation)
Derivation of the Chi-Square statistic: Let observed counts in $k$ categories follow a Multinomial distribution $text{Multinomial}(N, p_1, dots, p_k)$. By the Multivariate Central Limit Theorem, the normalized vector $frac{O_i – N p_i}{sqrt{N p_i}}$ converges to a singular multivariate normal distribution with rank $k-1$ (due to constraint $sum O_i = N$). The sum of squares of these standardized normal components is $sum_{i=1}^k left(frac{O_i – N p_i}{sqrt{N p_i}}right)^2 = sum_{i=1}^k frac{(O_i – E_i)^2}{E_i} sim chi^2_{k-1}$. In a contingency table with $r$ rows and $c$ columns, estimating expected frequencies $E_{ij} = N hat{p}_{icdot} hat{p}_{cdot j}$ consumes $(r-1) + (c-1)$ parameters, leaving degrees of freedom $text{df} = (rc – 1) – (r-1) – (c-1) = (r-1)(c-1)$.
四、工业级落地权衡与工程考量 (Industrial Trade-offs)
实践要点:① 期望频数要求——卡方检验是渐近检验,要求每个单元格的期望频数 ≥5(保守做法);若有不满足的单元格,应改用 Fisher 精确检验(小样本列联表)或合并类别。② 只能说明’是否相关’,不能说明’强度’——χ² 随样本量增大而增大,故显著不等于关联强;度量关联强度应用 Cramér’s V(=√(χ²/(n·min(r−1,c−1))))、优势比(OR) 或 phi 系数。③ 不能给出方向——独立性检验是双侧的,若需方向(如’处理组比例更高’)应用比例检验(单侧 z 检验)或看残差符号(标准化残差 (O−E)/√E)。④ 连续数据需先分箱——卡方用于分类数据;对连续数据需先分箱(会损失信息),或改用 KS 检验(分布比较)、t 检验(均值比较)。⑤ A/B 测试中的 SRM 检测——用卡方检验分流比例是否偏离预期,阈值通常取 p<0.001(比常规更严,因为 SRM 会持续影响所有实验);发现 SRM 应废弃实验而非’校正’数据。⑥ 与 G 检验的关系——G 检验(似然比卡方)在理论上更优(可分解),小样本下表现更好。
⚙️ 查看英文落地权衡分析 (English Systems & Trade-offs)
Industrial uses: (1) SRM Sanity Check: Flagging sample ratio mismatches in A/B testing uses goodness-of-fit. (2) Feature Selection: $chi^2$ feature selection tests whether categorical features (e.g. Device type, search query intent) are independent of class label $Y$, ranking features by test statistic magnitude.
五、常见面试避坑陷阱 (Common Pitfalls & Traps)
- ⚠️ 在期望频数 <5 时仍用卡方(应用 Fisher 精确检验)
- ⚠️ 用卡方 p 值说明关联强度(应用 Cramér’s V)
English Pitfalls:
– Applying the Chi-Square test when expected cell counts are tiny ($E_i < 5$), inflating Type I error.
– Confusing the Test of Independence (single sample cross-classified) with Test of Homogeneity (pre-determined fixed row totals across independent samples).
六、高频深度面试追问与预测 (Follow-Up Questions)
- 卡方检验对期望频数有什么要求?
- Why does Fisher’s Exact Test supersede Chi-Square when cell counts are small?
- 为什么它不能说明关联强度?
- How does Cramér’s V standardize the Chi-Square statistic into an effect size in $[0, 1]$?
七、知识图谱对齐 (Knowledge Graph Anchor)
- 🔗 关联底层卡片:
数理统计假说检验、P 值、I/II 类错误与统计功效(Hypothesis Testing, P-Values, Power & Type I/II Error) - 🗺️ 知识图谱模块:
数理基础思维导图
🔬 算法科学家与机器学习深度考察全量题库 (Science Depth)
本题收录于 TalentMe 算法科学家深度考察真题库 (Science Depth)。全库共 856 道硬核考点,深度覆盖数学统计、经典ML、深度学习、Transformer、大语言模型、多模态、推荐系统与 MLOps。支持 Jev 面经智能匹配、一键离线单文件 HTML 手册导出并直连 Obsidian 本地记忆。