我正在尝试编写一个小脚本,用于提取Steam游戏标签并将它们存储在CSV文件中。目前遇到的问题是不知道如何从输出中删除HTML标签。以下是我的代码:
from __future__ import absolute_import
import scrapy
from Example.items import SteamItem
from scrapy.selector import HtmlXPathSelector
class SteamSpider(scrapy.Spider):
name = 'steamspider'
allowed_domains = ['https://store.steampowered.com/app']
start_urls = ["https://store.steampowered.com/app/578080/PLAYERUNKNOWNS_BATTLEGROUNDS/",]
def parse(self, response):
hxs = HtmlXPathSelector(response)
tags = hxs.xpath('//*[@id="game_highlights"]/div[1]/div/div[4]/div/div[2]')
for sel in tags:
item = SteamItem()
item['gametags'] = sel.xpath('.//a/text()').extract()
item['gametitle'] = sel.xpath('//html/body/div[1]/div[7]/div[3]/div[1]/div[2]/div[2]/div[2]/div/div[3]/text()').extract()
yield item
我的物品类:
class SteamItem(scrapy.Item):
#defining item fields
url = scrapy.Field()
gametitle = scrapy.Field()
gametags = scrapy.Field()
我的输出看起来像这样:
[u'\r\n\t\t\t\t\t\t\t\t\t\t\t\tSurvival\t\t\t\t\t\t\t\t\t\t\t\t',
u'\r\n\t\t\t\t\t\t\t\t\t\t\t\tShooter\t\t\t\t\t\t\t\t\t\t\t\t',
u'\r\n\t\t\t\t\t\t\t\t\t\t\t\tMultiplayer\t\t\t\t\t\t\t\t\t\t\t\t',
u'\r\n\t\t\t\t\t\t\t\t\t\t\t\tPvP\t\t\t\t\t\t\t\t\t\t\t\t',
u'\r\n\t\t\t\t\t\t\t\t\t\t\t\tThird-Person Shooter\t\t\t\t\t\t\t\t\t\t\t\t',
u'\r\n\t\t\t\t\t\t\t\t\t\t\t\tFPS\t\t\t\t\t\t\t\t\t\t\t\t',
u'\r\n\t\t\t\t\t\t\t\t\t\t\t\tAction\t\t\t\t\t\t\t\t\t\t\t\t',
u'\r\n\t\t\t\t\t\t\t\t\t\t\t\tBattle Royale\t\t\t\t\t\t\t\t\t\t\t\t',
u'\r\n\t\t\t\t\t\t\t\t\t\t\t\tOnline Co-Op\t\t\t\t\t\t\t\t\t\t\t\t',
u'\r\n\t\t\t\t\t\t\t\t\t\t\t\tTactical\t\t\t\t\t\t\t\t\t\t\t\t',
u'\r\n\t\t\t\t\t\t\t\t\t\t\t\tCo-op\t\t\t\t\t\t\t\t\t\t\t\t',
u'\r\n\t\t\t\t\t\t\t\t\t\t\t\tEarly Access\t\t\t\t\t\t\t\t\t\t\t\t',
u'\r\n\t\t\t\t\t\t\t\t\t\t\t\tFirst-Person\t\t\t\t\t\t\t\t\t\t\t\t',
u'\r\n\t\t\t\t\t\t\t\t\t\t\t\tViolent\t\t\t\t\t\t\t\t\t\t\t\t',
u'\r\n\t\t\t\t\t\t\t\t\t\t\t\tStrategy\t\t\t\t\t\t\t\t\t\t\t\t',
u'\r\n\t\t\t\t\t\t\t\t\t\t\t\tThird Person\t\t\t\t\t\t\t\t\t\t\t\t',
u'\r\n\t\t\t\t\t\t\t\t\t\t\t\tCompetitive\t\t\t\t\t\t\t\t\t\t\t\t',
u'\r\n\t\t\t\t\t\t\t\t\t\t\t\tTeam-Based\t\t\t\t\t\t\t\t\t\t\t\t',
u'\r\n\t\t\t\t\t\t\t\t\t\t\t\tDifficult\t\t\t\t\t\t\t\t\t\t\t\t',
u'\r\n\t\t\t\t\t\t\t\t\t\t\t\tSimulation\t\t\t\t\t\t\t\t\t\t\t\t'],
我的目标是移除所有标签"u'\r\n\t.....\t
你有什么好的想法吗?
谢谢!
w3lib.html
只删除 HTML 标签,例如\r\n\t\t\
。我使用了.replace('\r\n\t\t\', '')
。 - Ericksan Pimentel