所属模块：
M2 · 经典机器学习 (Classical Machine Learning)| 专题分类：特征工程 (Feature Engineering)| 难度等级：Easy
一、核心一句话结论 (One-Sentence Summary)
one-hot（低基数）、目标编码（高基数）、频率编码、哈希、嵌入（深度学习）。
One-hot for low cardinality, target encoding for high cardinality, frequency encoding for prevalence, hashing for unbounded streams, and embeddings for deep architectures.
二、核心考点要义 (Key Insights)
- 📌 高基数 one-hot 导致维度爆炸与稀疏
- 📌 目标编码需交叉拟合防泄漏
English Insights:
– One-hot: low cardinality (< 20 levels), linear models and neural networks
– Target encoding: high cardinality, requires out-of-fold smoothing to prevent leakage
– Embeddings: dense representation capable of capturing semantic similarities in deep learning
三、核心数学原理与机理推导 (Mathematical Principles & Derivation)
$$text{target enc}: bar y_{text{cat}} text{with smoothing}$$
五种编码的适用与权衡：① One-hot——每个类别一列 0/1；适合低基数（<10–20 类）；优点是简单无信息泄漏；缺点是高基数时维度爆炸、矩阵稀疏、且对树模型低效（每个 one-hot 列只能做是/否分裂）。② 目标编码（Target/Mean Encoding）——用类别对应的目标均值替代；适合高基数（如城市、商品 ID）；关键是必须交叉拟合（在 K 折内用其他折的均值编码本折）与平滑（向全局均值收缩：编码 = (n·mean_cat + m·global_mean)/(n+m)，n 为类别样本数、m 为平滑强度），否则低频类别的编码会严重过拟合。③ 频率/计数编码——用类别出现频次；简单、无泄漏，但信息量有限（只反映流行度）；常与其他编码组合。④ 哈希编码（Hashing Trick）——用哈希函数把类别映射到固定维度；优点是无需维护词表、支持流式与未知类别；缺点是哈希冲突；用符号哈希（signed hash）可部分抵消冲突。⑤ 嵌入（Embedding）——为每个类别学一个稠密向量；适合深度学习与极高基数（推荐系统中的商品/用户 ID）；能捕捉类别间的相似性，但需足够数据训练。
📖 查看英文严格数学推导 (English Mathematical Derivation)
Trade-offs across categorical encodings: ① One-Hot Encoding: Generates an indicator vector per category. Suitable for low cardinality ($< 20$). In tree models, it degrades efficiency as splits become sparse and unbalanced. ② Target Encoding: Encodes categories as the smoothed expectation of the target $hat{S}_c = lambda(n_c) bar{y}_c + (1 – lambda(n_c)) bar{y}$, where $lambda(n_c) = frac{n_c}{n_c + m}$. Crucial: Must use out-of-fold (OOF) cross-fitting and additive noise to prevent target leakage. ③ Frequency/Count Encoding: Maps categories to their frequency $n_c / N$. Effective when prevalence correlates with the target. ④ Feature Hashing (Hashing Trick): Uses hash functions to project categories into a fixed-size space $h(x) pmod B$. Handles open vocabularies and streaming inputs at the cost of hash collisions. ⑤ Entity Embeddings: Maps high-cardinality discrete IDs to continuous vectors via lookup tables, capturing non-linear relationships.
四、工业级落地权衡与工程考量 (Industrial Trade-offs)
实践要点：① 基数与编码的对应——低基数（<20）→ one-hot；中高基数（20–10⁴）→ 目标编码（带交叉拟合）或频率编码；极高基数（>10⁴）→ 哈希或嵌入。② 目标编码的泄漏机制——若用全量数据算类别均值，则每个样本的编码都包含了自己的标签（尤其当类别样本少时），模型会过度依赖该特征；交叉拟合模拟了’预测时只有其他样本’的情形。③ 树模型的原生支持——LightGBM 与 CatBoost 可直接处理类别特征（CatBoost 的 ordered target statistics 从原理上避免泄漏）；若用这两个库，可不手动编码。④ 未知类别处理——线上会出现训练时未见过的类别，需预留’未知’类别或用哈希（哈希天然支持）。⑤ 编码与模型的交互——线性模型需 one-hot 或目标编码（要显式数值）；树模型对编码方式较宽容但目标编码的泄漏风险仍需注意；神经网络宜用嵌入。
⚙️ 查看英文落地权衡分析 (English Systems & Trade-offs)
Engineering trade-offs: ① Tree Models vs Linear Models: Gradient boosted trees (LightGBM/CatBoost) handle integer/categorical splits natively without one-hot expansion. Linear models require one-hot or target encoding. ② High-Cardinality Cold Start: Target encoding and frequency encoding require a robust fallback (global prior) for unseen categories. ③ Dimension vs Collision: In hashing, balance memory footprint against collision rates.
五、常见面试避坑陷阱 (Common Pitfalls & Traps)
- ⚠️ 对高基数类别用 one-hot（维度爆炸）
- ⚠️ 目标编码不做交叉拟合与平滑（严重泄漏）
English Pitfalls:
– Using global target statistics without out-of-fold cross-validation, causing severe target leakage
– Applying one-hot encoding to features with thousands of levels, causing dimensionality explosion and memory exhaustion
六、高频深度面试追问与预测 (Follow-Up Questions)
- 为什么高基数不适合 one-hot？
- How does CatBoost prevent target leakage in its ordered target encoding implementation?
- 目标编码如何做平滑？
- When would you choose frequency encoding over target encoding?
七、知识图谱对齐 (Knowledge Graph Anchor)
- 🔗 关联底层卡片：
特征工程实战：Target Encoding、组合特征与特征离散化(Feature Engineering: Target Encoding & Feature Stores) - 🗺️ 知识图谱模块：
机器学习工程师高频考点导图
🔬 算法科学家与机器学习深度考察全量题库 (Science Depth)
本题收录于 TalentMe 算法科学家深度考察真题库 (Science Depth)。全库共 856 道硬核考点，深度覆盖数学统计、经典ML、深度学习、Transformer、大语言模型、多模态、推荐系统与 MLOps。支持 Jev 面经智能匹配、一键离线单文件 HTML 手册导出并直连 Obsidian 本地记忆。