KNN算法在处理分类值时无法准确预测。

Question

KNN算法在处理分类值时无法准确预测。

machine-learningscikit-learnrandom-forestknn

3

我正在尝试构建一个模型，可以根据商品预测它属于哪个商店。

我的数据集大约有250条记录，这些记录应该是来自不同在线商店的商品。

每个记录由以下组成： 类别、子类别、价格、商店标识符（y变量） 我尝试了几个邻居的数量，使用曼哈顿距离，但很遗憾无法获得更好的结果，准确度约为0.55。随机森林的准确率约为0.7。

我的直觉告诉我，应该能够通过模型来解决这个问题。我错过了什么吗？

这是数据： https://pastebin.com/nUsSbkp4

from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import confusion_matrix
from sklearn.cross_validation import train_test_split
from sklearn.preprocessing import LabelEncoder, OneHotEncoder
from sklearn.neighbors import KNeighborsClassifier
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

dataset = pd.read_csv('data.csv')
X = dataset.iloc[:, :-1].values
y = dataset.iloc[:, 3].values

labelencoder_X_0 = LabelEncoder()
X[:, 0] = labelencoder_X_0.fit_transform(X[:, 0])

labelencoder_X_1 = LabelEncoder()
X[:, 1] = labelencoder_X_1.fit_transform(X[:, 1])

onehotencoder_0 = OneHotEncoder(categorical_features = [0])
X = onehotencoder_0.fit_transform(X).toarray()

onehotencoder_1 = OneHotEncoder(categorical_features = [1])
X = onehotencoder_1.fit_transform(X).toarray()

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.25, random_state = 0)
# classifier = RandomForestClassifier(n_estimators=25, criterion='entropy', random_state = 0)
classifier = KNeighborsClassifier(n_neighbors=3, metric='minkowski', p=2)
classifier.fit(X_train, y_train)

y_pred = classifier.predict(X_test)
cm = confusion_matrix(y_test, y_pred)

accuracy = classifier.score(X_test, y_test) 
print(accuracy)

- Steinfeld

1个回答

网页内容由stack overflow 提供, 点击上面的

可以查看英文原文，
原文链接

- Marcus V. · Accepted Answer

KNN可以利用分类预测变量产生良好的预测效果。我以前曾经成功地使用过它。但是还有一些需要注意的事项：

数值变量必须在相同的比例尺上，例如通过使用min-max scaling。
可以尝试特定设计的误差度量，例如Gower distance。

除此之外，您实际上在独热编码中存在一个错误：

在调用第一个独热编码器之后，您将获得形状为（273，21）的数组：

onehotencoder_0 = OneHotEncoder(categorical_features = [0])
X = onehotencoder_0.fit_transform(X).toarray()
print(X.shape)
print(X[:5,:])

Out:
(275, 21)
[[ 0.    0.    0.    0.    0.    0.    0.    0.    1.    0.    0.    0.
   0.    0.    0.    0.    0.    0.    0.   52.   33.99]
 [ 0.    0.    0.    0.    0.    0.    0.    0.    1.    0.    0.    0.
   0.    0.    0.    0.    0.    0.    0.   52.   33.97]
 [ 0.    0.    0.    0.    0.    0.    0.    0.    1.    0.    0.    0.
   0.    0.    0.    0.    0.    0.    0.   36.   27.97]
 [ 0.    0.    0.    0.    0.    0.    0.    0.    1.    0.    0.    0.
   0.    0.    0.    0.    0.    0.    0.   37.   13.97]
 [ 0.    0.    0.    0.    0.    0.    0.    0.    1.    0.    0.    0.
   0.    0.    0.    0.    0.    0.    0.   20.    9.97]]

然后你在第二列上调用独热编码，该列仅有两个值（零和一），因此结果为：

onehotencoder_1 = OneHotEncoder(categorical_features = [1])
X = onehotencoder_1.fit_transform(X).toarray()
print(X.shape)
print(X[:5,:])

Out:
(275, 22)
[[ 1.    0.    0.    0.    0.    0.    0.    0.    0.    1.    0.    0.
   0.    0.    0.    0.    0.    0.    0.    0.   52.   33.99]
 [ 1.    0.    0.    0.    0.    0.    0.    0.    0.    1.    0.    0.
   0.    0.    0.    0.    0.    0.    0.    0.   52.   33.97]
 [ 1.    0.    0.    0.    0.    0.    0.    0.    0.    1.    0.    0.
   0.    0.    0.    0.    0.    0.    0.    0.   36.   27.97]
 [ 1.    0.    0.    0.    0.    0.    0.    0.    0.    1.    0.    0.
   0.    0.    0.    0.    0.    0.    0.    0.   37.   13.97]
 [ 1.    0.    0.    0.    0.    0.    0.    0.    0.    1.    0.    0.
   0.    0.    0.    0.    0.    0.    0.    0.   20.    9.97]]

所以，如果你能修复它或者简单地使用管道来避免这种情况，并且像这样添加数值变量的比例缩放：

from sklearn.pipeline import Pipeline, FeatureUnion, make_pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.neighbors import KNeighborsClassifier

class Columns(BaseEstimator, TransformerMixin):
    def __init__(self, names=None):
        self.names = names

    def fit(self, X, y=None, **fit_params):
        return self

    def transform(self, X):
        return X.loc[:,self.names]

dataset = pd.read_csv('data.csv', header=None)
dataset.columns = ["cat1", "cat2", "num1", "target"]

X = dataset.iloc[:, :-1]
y = dataset.iloc[:, 3]

labelencoder_X_0 = LabelEncoder()
X.iloc[:, 0] = labelencoder_X_0.fit_transform(X.iloc[:, 0])

labelencoder_X_1 = LabelEncoder()
X.iloc[:, 1] = labelencoder_X_1.fit_transform(X.iloc[:, 1])

numeric = ["num1"]
categorical = ["cat1", "cat2"]

pipe = Pipeline([
    ("features", FeatureUnion([
        ('numeric', make_pipeline(Columns(names=numeric),StandardScaler())),
        ('categorical', make_pipeline(Columns(names=categorical), OneHotEncoder(sparse=False)))
    ])),
])

X = pipe.fit_transform(X)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.25, random_state = 0)
# classifier = RandomForestClassifier(n_estimators=25, criterion='entropy', random_state = 0)
classifier = KNeighborsClassifier(n_neighbors=3, metric='minkowski', p=2)
classifier.fit(X_train, y_train)

y_pred = classifier.predict(X_test)
cm = confusion_matrix(y_test, y_pred)

accuracy = classifier.score(X_test, y_test) 
print(accuracy)

Out: 
0.7101449275362319

正如您所看到的，这使得准确度至少与随机森林相当！

因此，您可以尝试使用高尔距离。目前正在讨论将其添加到sklearn 这里，因此可以在Ipython Notebook中查看发布的代码并尝试使用。