假设我有一些文本句子,想要使用kmeans进行聚类。
sentences = [
"fix grammatical or spelling errors",
"clarify meaning without changing it",
"correct minor mistakes",
"add related resources or links",
"always respect the original author"
]
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.cluster import KMeans
vectorizer = CountVectorizer(min_df=1)
X = vectorizer.fit_transform(sentences)
num_clusters = 2
km = KMeans(n_clusters=num_clusters, init='random', n_init=1,verbose=1)
km.fit(X)
现在,我可以预测一个新文本将属于哪个类别。
new_text = "hello world"
vec = vectorizer.transform([new_text])
print km.predict(vec)[0]
然而,假设我应用PCA将10,000个特征降至50个。
from sklearn.decomposition import RandomizedPCA
pca = RandomizedPCA(n_components=50,whiten=True)
X2 = pca.fit_transform(X)
km.fit(X2)
我不能再像以前那样预测新文本的聚类,因为向量化器的结果不再相关。
new_text = "hello world"
vec = vectorizer.transform([new_text]) ##
print km.predict(vec)[0]
ValueError: Incorrect number of features. Got 10000 features, expected 50
那么,我如何将我的新文本转换为较低维度的特征空间?