Python：使用scikit-learn进行预测，结果为空白

Question

我在客服工作，正在使用scikit-learn来预测我们的工单的标签，给定一个训练集（大约有40,000个工单）。我使用基于这个分类模型。然而，它对我的测试集的工单仅预测出“()”这个标签，尽管在训练集中没有一个工单是没有标签的。我的标签训练数据是一个列表的列表，例如:

tags_train = [['international_solved'], ['from_build_guidelines my_new_idea eligibility'], ['dropbox other submitted_faq submitted_help'], ['my_new_idea_solved'], ['decline macro_backer_paypal macro_prob_errored_pledge_check_credit_card_us loading_problems'], ['dropbox macro__turnaround_time other plq__turnaround_time submitted_help'], ['dropbox macro_creator__logo_style_guide outreach press submitted_help']]

尽管我的工单描述的训练数据只是一串字符串，例如：

descs_train = ['description of ticket one', 'description of ticket two', etc]

这是我用于构建模型的相关代码部分：

```

import numpy as np
import scipy
from sklearn.pipeline import Pipeline
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.feature_extraction.text import TfidfTransformer
from sklearn.multiclass import OneVsRestClassifier
from sklearn.svm import LinearSVC

# We have lists called tags_train, descs_train, tags_test, descs_test with the test and train data

X_train = np.array(descs_train)
y_train = tags_train
X_test = np.array(descs_test)  

classifier = Pipeline([
    ('vectorizer', CountVectorizer()),
    ('tfidf', TfidfTransformer()),
    ('clf', OneVsRestClassifier(LinearSVC(class_weight='auto')))])

classifier.fit(X_train, y_train)
predicted = classifier.predict(X_test)

然而，"predicted"会给出一个类似于以下列表的列表：

predicted = [(), ('account_solved',), (), ('images_videos_solved',), ('my_new_idea_solved',), (), (), (), (), (), ('images_videos_solved', 'account_solved', 'macro_launched__edit_update other tips'), ('from_guidelines my_new_idea', 'from_guidelines my_new_idea macro__eligibility'), ()]

我不明白为什么它在训练集中没有空标签的情况下会预测出空标签。它难道不应该预测最相近的标签吗？有没有人能推荐一些改进我正在使用的模型的方法？

非常感谢您提前的帮助！

- jegeragh

---

CountVectorizer文档TfidfTransformer文档OneVsRestClassifier文档 - jegeragh

你想要多类别还是多标签分类？一张票是否允许被打上多个标签？ - mbatchkarov

Answer 1

问题出在你的tags_train变量上。根据OneVsRestClassifier文档，目标需要是“标签序列的序列”，而你的目标是仅有一个元素的列表。

以下是经过编辑、自包含且运行良好的代码版本。请注意tags_train的更改，尤其是tags_train是一个单元素元组的事实。

import numpy as np
import scipy
from sklearn.pipeline import Pipeline
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.feature_extraction.text import TfidfTransformer
from sklearn.multiclass import OneVsRestClassifier
from sklearn.svm import LinearSVC


# We have lists called tags_train, descs_train, tags_test, descs_test with the test and train data
tags_train = [('label', ), ('international' ,'solved'), ('international','open')]
descs_train = ['description of ticket one', 'some other ticket two', 'label']

X_train = np.array(descs_train)
y_train = tags_train
X_test = np.array(descs_train)  

classifier = Pipeline([
    ('vectorizer', CountVectorizer()),
    ('tfidf', TfidfTransformer()),
    ('clf', OneVsRestClassifier(LinearSVC(class_weight='auto')))])

classifier = classifier.fit(X_train, y_train)
predicted = classifier.predict(X_test)

print predicted

输出结果为：

[('international',), ('international',), ('international', 'open')]

- mbatchkarov

Answer 2

即使将目标从一个元素的列表转换为序列，仍然面临()预测问题。

- Dharshini