如何从Sklearn管道中提取特征重要性

Question

我在Scikit-Learn中构建了一个流水线，其中包括两个步骤：第一个步骤是构造特征，第二个步骤是一个RandomForestClassifier。

虽然我可以保存这个流水线，查看各个步骤和步骤中设置的各种参数，但我想能够检查结果模型中的特征重要性。

这可行吗？

- elksie5000

Answer 1

啊，是的，没错。

您需要确定要检查估计器的步骤：

例如：

pipeline.steps[1]

返回：

('predictor',
 RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',
             max_depth=None, max_features='auto', max_leaf_nodes=None,
             min_samples_leaf=1, min_samples_split=2,
             min_weight_fraction_leaf=0.0, n_estimators=50, n_jobs=2,
             oob_score=False, random_state=None, verbose=0,
             warm_start=False))

您可以直接访问模型的一步：

pipeline.steps[1][1].feature_importances_

- elksie5000

---

要获取特征名称，您需查看pipe.steps[0][1].get_feature_names()。 - Devon

这是一个不完整的答案。预处理和特征工程通常是管道的一部分。因此，您需要考虑到这一点。 - ben26941

如果有多个步骤，一种方法是使用步骤名称检索估计器。对于提问者的情况，这可以是 pipeline.named_steps['predictor'].feature_importances_。 - edesz

你如何更改特征重要性类型？ - Maths12

Answer 2

我写了一篇关于如何在通用情况下实现这一点的文章，你可以在这里找到。

通常来说，在管道中您可以访问named_steps参数。 这将为您提供管道中的每个转换器。例如，对于此管道：

model = Pipeline(
[
    ("vectorizer", CountVectorizer()),
    ("transformer", TfidfTransformer()),
    ("classifier", classifier),
])

我们可以通过model.named_steps["transformer"].get_feature_names()访问各个特征步骤。这将返回来自TfidfTransformer的特征名称列表。这很好，但并不真正涵盖许多用例，因为我们通常想要组合几个特征。以这个模型为例：

model = Pipeline([
("union", FeatureUnion(transformer_list=[
    ("h1", TfidfVectorizer(vocabulary={"worst": 0})),
    ("h2", TfidfVectorizer(vocabulary={"best": 0})),
    ("h3", TfidfVectorizer(vocabulary={"awful": 0})),
    ("tfidf_cls", Pipeline([
        ("vectorizer", CountVectorizer()),
        ("transformer", TfidfTransformer())
    ]
    ))
])
 ),
("classifier", classifier)])

这里我们使用特征联合和子管道组合了一些特征。要访问这些特征，我们需要按顺序显式调用每个命名步骤。例如，如果要从内部流水线中获取TF-IDF特征，我们需要执行以下操作：

model.named_steps["union"].tranformer_list[3][1].named_steps["transformer"].get_feature_names()

这有点麻烦，但是可行。通常我使用以下代码段的变体来获取它。下面的代码将管道/特征联合集合视为一棵树，并执行DFS组合feature_names。

from sklearn.pipeline import FeatureUnion, Pipeline

def get_feature_names(model, names: List[str], name: str) -> List[str]:
    """Thie method extracts the feature names in order from a Sklearn Pipeline
    
    This method only works with composed Pipelines and FeatureUnions.  It will
    pull out all names using DFS from a model.

    Args:
        model: The model we are interested in
        names: The list of names of final featurizaiton steps
        name: The current name of the step we want to evaluate.

    Returns:
        feature_names: The list of feature names extracted from the pipeline.
    """
    
    # Check if the name is one of our feature steps.  This is the base case.
    if name in names:
        # If it has the named_steps atribute it's a pipeline and we need to access the features
        if hasattr(model, "named_steps"):
            return extract_feature_names(model.named_steps[name], name)
        # Otherwise get the feature directly
        else:
            return extract_feature_names(model, name)
    elif type(model) is Pipeline:
        feature_names = []
        for name in model.named_steps.keys():
            feature_names += get_feature_names(model.named_steps[name], names, name)
        return feature_names
    elif type(model) is FeatureUnion:
        feature_names= []
        for name, new_model in model.transformer_list:
            feature_names += get_feature_names(new_model, names, name)
        return feature_names
    # If it is none of the above do not add it.
    else:
        return []

您还需要这个方法。它作用于单个转换，例如TfidfVectorizer，以获取名称。在SciKit-Learn中没有通用的get_feature_names，因此您必须为每种不同情况进行调整。这是我尝试为大多数用例做出合理处理的方法。

def extract_feature_names(model, name) -> List[str]:
  """Extracts the feature names from arbitrary sklearn models
  
  Args:
    model: The Sklearn model, transformer, clustering algorithm, etc. which we want to get named features for.
    name: The name of the current step in the pipeline we are at.

  Returns:
    The list of feature names.  If the model does not have named features it constructs feature names
by appending an index to the provided name.
  """
    if hasattr(model, "get_feature_names"):
        return model.get_feature_names()
    elif hasattr(model, "n_clusters"):
        return [f"{name}_{x}" for x in range(model.n_clusters)]
    elif hasattr(model, "n_components"):
        return [f"{name}_{x}" for x in range(model.n_components)]
    elif hasattr(model, "components_"):
        n_components = model.components_.shape[0]
        return [f"{name}_{x}" for x in range(n_components)]
    elif hasattr(model, "classes_"):
        return classes_
    else:
        return [name]

- nbertagnolli