以下是可复现的示例代码:
from numpy import mean
from sklearn.datasets import make_classification
from sklearn.model_selection import cross_validate
from sklearn.model_selection import StratifiedShuffleSplit
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import balanced_accuracy_score
# define dataset
X, y = make_classification(n_samples=1000, weights = [0.3,0.7], n_features=100, n_informative=75, random_state=0)
# define the model
model = RandomForestClassifier(n_estimators=10, random_state=0)
# evaluate the model
n_splits=10
cv = StratifiedShuffleSplit(n_splits, random_state=0)
n_scores = cross_validate(model, X, y, scoring='balanced_accuracy', cv=cv, n_jobs=-1, error_score='raise')
# report performance
print('Accuracy: %0.4f' % (mean(n_scores['test_score'])))
bal_acc_sum = []
for train_index, test_index in cv.split(X,y):
model.fit(X[train_index], y[train_index])
bal_acc_sum.append(balanced_accuracy_score(model.predict(X[test_index]),y[test_index]))
print(f"Accuracy: %0.4f" % (mean(bal_acc_sum)))
结果:
Accuracy: 0.6737
Accuracy: 0.7113
我自己计算的准确率结果总是比交叉验证给出的结果高。但它们应该是相同的,或者我漏掉了什么?使用相同的度量方法、相同的数据划分(KFold 带来相同的结果)、相同的固定模型(其他模型表现相同)、相同的随机状态,但却得到不同的结果。