DBSCAN从图中去除噪声

5
使用DBSCAN算法,
(DBSCAN(eps=epsilon, min_samples=10, algorithm='ball_tree', metric='haversine')

我已经对一系列经纬度对进行了聚类,并使用matplotlib绘制了图形。在绘图时,它包括“噪点”坐标,也就是那些没有分配给270个创建的聚类之一的点。我想从图中删除噪点,并只绘制满足指定要求的聚类,但我不确定如何实现。我应该如何排除噪点(即那些未分配给聚类的点)?
以下是我用于聚类和绘图的代码:
df = pd.read_csv('xxx.csv')

# define the number of kilometers in one radiation
# which will be used to convert esp from km to radiation
kms_per_rad = 6371.0088

# define a function to calculate the geographic coordinate
# centroid of a cluster of geographic points
# it will be used later to calculate the centroids of DBSCAN cluster
# because Scikit-learn DBSCAN cluster class does not come with centroid attribute.
def get_centroid(cluster):
"""calculate the centroid of a cluster of geographic coordinate points
Args:
  cluster coordinates, nx2 array-like (array, list of lists, etc)
  n is the number of points(latitude, longitude)in the cluster.
Return:
  geometry centroid of the cluster

"""
cluster_ary = np.asarray(cluster)
centroid = cluster_ary.mean(axis=0)
return centroid

# convert eps to radians for use by haversine
epsilon = 0.1/kms_per_rad #1.5=1.5km  1=1km  0.5=500m 0.25=250m   0.1=100m

# Extract intersection coordinates (latitude, longitude)
tweet_coords = df.as_matrix(columns=['latitude','longitude'])

start_time = time.time()
dbsc = (DBSCAN(eps=epsilon, min_samples=10, algorithm='ball_tree', metric='haversine')
    .fit(np.radians(tweet_coords)))

tweet_cluster_labels = dbsc.labels_

# get the number of clusters
num_clusters = len(set(dbsc.labels_))

# print the outcome
message = 'Clustered {:,} points down to {:,} clusters, for {:.1f}% compression in {:,.2f} seconds'
print(message.format(len(df), num_clusters, 100*(1 - float(num_clusters) / len(df)), time.time()-start_time))
print('Silhouette coefficient:     {:0.03f}'.format(metrics.silhouette_score(tweet_coords, tweet_cluster_labels)))

# Turn the clusters into a pandas series,where each element is a cluster of points
dbsc_clusters = pd.Series([tweet_coords[tweet_cluster_labels==n] for n in  range(num_clusters)])

# get centroid of each cluster
cluster_centroids = dbsc_clusters.map(get_centroid)
# unzip the list of centroid points (lat, lon) tuples into separate lat and lon lists
cent_lats, cent_lons = zip(*cluster_centroids)
# from these lats/lons create a new df of one representative point for eac   cluster
centroids_df = pd.DataFrame({'longitude':cent_lons, 'latitude':cent_lats})
#print centroids_df

# Plot the clusters and cluster centroids
fig, ax = plt.subplots(figsize=[20, 12])
tweet_scatter = ax.scatter(df['longitude'], df['latitude'],   c=tweet_cluster_labels, cmap = cm.hot, edgecolor='None', alpha=0.25, s=50)
centroid_scatter = ax.scatter(centroids_df['longitude'], centroids_df['latitude'], marker='x', linewidths=2, c='k', s=50)
ax.set_title('Tweet Clusters & Cluser Centroids', fontsize = 30)
ax.set_xlabel('Longitude', fontsize=24)
ax.set_ylabel('Latitude', fontsize = 24)
ax.legend([tweet_scatter, centroid_scatter], ['Tweets', 'Tweets Cluster Centroids'], loc='upper right', fontsize = 20)
plt.show()

我的目标是仅呈现聚类,黑色点表示噪声,未被DBSCAN输入定义的聚类所添加,而彩色点则表示聚类。

我编辑了这篇文章,只包含一个问题。噪声是指使用算法未分配到一个簇的点。 - andrewr
我希望这次最新的编辑能够进一步澄清噪声问题,以及我试图实现的目标。我尝试包含我的结果截图,但是我的声望还不够高。 - andrewr
1
图片很有帮助。您可以将它们放入,它们将显示为链接。 - ImportanceOfBeingErnest
如果您通知我,我可以将它们放在里面。 - Martin Thoma
Martin,我能够包含指向imgur的链接。 - andrewr
1个回答

5

在原始数据帧的另一列中存储标签。

df['tweet_cluster_labels'] = tweet_cluster_labels

筛选DataFrame以仅包含非噪声点(嘈杂的样本标记为-1)

df_filtered = df[df.tweet_cluster_labels>-1]

仅绘制这些点

tweet_scatter = ax.scatter(df_filtered['longitude'], 
                df_filtered['latitude'],
                c=df_filtered.tweet_cluster_labels, 
                cmap=cm.hot, edgecolor='None', alpha=0.25, s=50)

网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接