sklearn中VotingClassifier的roc_auc和RandomForestClassifier的使用方法

Question

sklearn中VotingClassifier的roc_auc和RandomForestClassifier的使用方法

pythonscikit-learndecision-treerocensemble-learning

5

我正在尝试计算我构建的硬投票分类器的roc_auc。我提供了具有可重现性的代码示例。现在我想计算roc_auc分数并绘制ROC曲线，但不幸的是，我得到了以下错误信息："predict_proba is not available when voting='hard'"。

# Voting Ensemble for Classification
import pandas
from sklearn import datasets
from sklearn import model_selection
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.svm import SVC
from sklearn.ensemble import VotingClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.naive_bayes import MultinomialNB
from sklearn.cross_validation import StratifiedShuffleSplit
from sklearn.model_selection import cross_val_predict
from sklearn.model_selection import cross_val_score
from sklearn.metrics import make_scorer,confusion_matrix, f1_score, precision_score, recall_score, cohen_kappa_score,accuracy_score,roc_curve
import numpy as np
np.random.seed(42)
iris = datasets.load_iris()
X = iris.data[:, :4]  # we only take the first two features.
Y = iris.target
print(Y)
seed = 7
kfold = model_selection.KFold(n_splits=10, random_state=seed)
# create the sub models
estimators = []
model1 = LogisticRegression()
estimators.append(('logistic', model1))
model2 = RandomForestClassifier(n_estimators=200, max_depth=3, random_state=0)
estimators.append(('RandomForest', model2))
model3 = MultinomialNB()
estimators.append(('NaiveBayes', model3))
model4=SVC(probability=True)
estimators.append(('svm', model4))
model5=DecisionTreeClassifier()
estimators.append(('Cart', model5))
# create the ensemble model
print('Majority Class Labels (Majority/Hard Voting)')
ensemble = VotingClassifier(estimators,voting='hard')
#accuracy
results = model_selection.cross_val_score(ensemble, X, Y, cv=kfold,scoring='accuracy')
y_pred = cross_val_predict(ensemble, X ,Y, cv=10)
print("Accuracy ensemble model : %0.2f (+/- %0.2f) " % (results.mean(), results.std() ))
print(results.mean())
#recall
recall_scorer = make_scorer(recall_score, pos_label=1)
recall = cross_val_score(ensemble, X, Y, cv=kfold, scoring=recall_scorer)
print('Recall', np.mean(recall), recall)
# Precision
precision_scorer = make_scorer(precision_score, pos_label=1)
precision = cross_val_score(ensemble, X, Y, cv=kfold, scoring=precision_scorer)
print('Precision', np.mean(precision), precision)
#f1_score
f1_scorer = make_scorer(f1_score, pos_label=1)
f1_score = cross_val_score(ensemble, X, Y, cv=kfold, scoring=f1_scorer)
print('f1_score ', np.mean(f1_score ),f1_score )
#roc_auc_score
roc_auc_score = cross_val_score(ensemble, X, Y, cv=kfold, scoring='roc_auc')
print('roc_auc_score ', np.mean(roc_auc_score ),roc_auc_score )

- Rawia Hemdan

你尝试过设置 voting='soft' 吗？ - seralouk

1个回答

网页内容由stack overflow 提供, 点击上面的

可以查看英文原文，
原文链接

- seralouk · Accepted Answer

为了计算roc_auc指标，首先需要将:

Replace: ensemble = VotingClassifier(estimators,voting='hard')

替换为: ensemble = VotingClassifier(estimators,voting='soft')。

接下来，代码的最后两行会抛出一个错误：

roc_auc_score = cross_val_score(ensemble, X, Y, cv=3, scoring='roc_auc')
print('roc_auc_score ', np.mean(roc_auc_score ),roc_auc_score )

数值错误：不支持多类格式

这是正常的，因为在Y中有3个类别（np.unique(Y) == array([0, 1, 2])）。

你不能将roc_auc作为多类模型的单一摘要指标。如果需要，可以计算每个类别的roc_auc。

如何解决：

1）仅使用两个类别来计算roc_auc_score

2）在调用roc_auc_score之前提前使用标签二值化。