在之前的问题中,我询问了如何使用R中的twitteR包从Haaretz Twitter feed (@haaretzcom)下载大量的Twitter关注者(以及他们的位置、创建日期、关注者数量等)。该Twitter feed有超过90,000个关注者,我可以使用下面的代码轻松地下载完整的关注者列表。请参见Work around rate limit for extracting large list of user information using twitteR package in R。
该代码可以提取大部分用户,但是当我遇到某个特定的用户时,会出现以下错误:
即使我在twitteR包之后加载RJSONIO包,我仍然遇到了这个问题。通过一些研究,似乎twitteR和RJSONIO包在解析非UTF-8或ASCII字符(如阿拉伯语等)方面存在问题http://lists.hexdump.org/pipermail/twitter-users-hexdump.org/2013-May/000335.html。是否有一种方法可以在我的代码中简单地忽略非UTF-8或ASCII,同时仍然提取所有关注者信息?任何帮助都将不胜感激。
require(twitteR)
require(ROAuth)
#Loading the Twitter OAuthorization
load("~/Dropbox/Twitter/my_oauth")
#Confirming the OAuth
registerTwitterOAuth(my_oauth)
# opening list to download
haaretz_followers<-getUser("haaretzcom")$getFollowerIDs(retryOnRateLimit=9999999)
for (follower in haaretz_followers){
Sys.sleep(5)
haaretz_followers_info<-lookupUsers(haaretz_followers)
haaretz_followers_full<-twListToDF(haaretz_followers_info)
#Export data to csv
write.table(haaretz_followers_full, file = "haaretz_twitter_followers.csv", sep=",")
}
该代码可以提取大部分用户,但是当我遇到某个特定的用户时,会出现以下错误:
Error in twFromJSON(out) :
RMate stopped at line 51
Error: Malformed response from server, was not JSON.
RMate stopped at line 51
The most likely cause of this error is Twitter returning a character which
can't be properly parsed by R. Generally the only remedy is to wait long
enough for the offending character to disappear from searches (e.g. if
using searchTwitter()).
Calls: twListToDF ... lookupUsers -> lapply -> FUN -> <Anonymous> -> twFromJSON
Execution halted
即使我在twitteR包之后加载RJSONIO包,我仍然遇到了这个问题。通过一些研究,似乎twitteR和RJSONIO包在解析非UTF-8或ASCII字符(如阿拉伯语等)方面存在问题http://lists.hexdump.org/pipermail/twitter-users-hexdump.org/2013-May/000335.html。是否有一种方法可以在我的代码中简单地忽略非UTF-8或ASCII,同时仍然提取所有关注者信息?任何帮助都将不胜感激。
readLines
,然后替换掉有问题的字符。 - Ricardo Saporta