logo标签列表

用Python进行网页数据抓取：访问表格数据

pythonhtmlweb-scrapingbeautifulsoup
3

我正在尝试使用Beautiful Soup从http://www.killedbypolice.net/kbp2013.html这个网站进行网络爬虫，并访问表中的数据。我的当前代码如下：

url = "http://www.killedbypolice.net/kbp2013.html"
page = urllib.request.urlopen(url)
soup = BeautifulSoup(page, "html.parser")

data = soup.find_all('table')
data[0]

但是...我遇到了最大递归深度的运行时错误。我不确定如何访问表格中包含数据的'td'字段。谢谢。

1个回答
1
错误是因为HTML格式非常糟糕，使用lxml和html.parser创建soup对象时会出现RuntimeError: maximum recursion depth exceeded，唯一能正常工作的解析器是html5lib。
html = requests.get("http://www.killedbypolice.net/kbp2013.html").content
soup = BeautifulSoup(html, "html5lib")

# get all the table rows
table = soup.find("table")

这是HTML代码，翻译成中文为：“这将得到表格：”。
    <table background="/kbp/bg1.jpg" border="1" cellpadding="0" cellspacing="0" width="100%">
<tbody><tr><td><img src="http://www.killedbypolice.net/size.jpg" width="185"/><br/><b><center># since Jan 1st '14</center></b></td>
<td><b><center>St.</center></b></td>
<td><b><center>g/r</center></b></td>
<td><img src="http://www.killedbypolice.net/size.jpg" width="200"/><b><center>Name, Age</center></b></td>
<td></td>
<td><img src="http://www.killedbypolice.net/size.jpg" width="297"/><b><center>KBP link <font color="red">(plus extensive follow-ups)</font></center></b></td>
<td><b><center>News link</center></b></td>
......................................................................
</font></a></td></tr><tr><td><center>(2) May 2, 2013        </center>
</td><td>CA</td><td>M/B</td><td>Kenneth Bernard Williams, 55   </td><td><font size="2">G</font></td><td><a href="http://facebook.com/KilledByPolice/posts/622539181107556" target="new"><font size="2"></font></a><font size="2"><center><a href="http://facebook.com/KilledByPolice/posts/622539181107556" target="new">facebook.com/KilledByPolice/posts/622539181107556  </a></center></font></td><td><a href="http://www.nbclosangeles.com/news/local/Police-Shoot-Kill-Suspect-in-Skid-Row-Prompting-Angry-Crowd-to-Gather-205646861.html" target="new"><font size="2">http://www.nbclosangeles.com/news/local/Police-Shoot-Kill-Suspect-in-Skid-Row-Prompting-Angry-Crowd-to-Gather-205646861.html
</font></a></td></tr><tr><td><center>(1) May 1, 2013        </center></td><td>MI</td><td>M/B</td><td><a href="http://www.killedbypolice.net/victims/2261.jpg" target="new">Jordan West-Morson, 26   </a></td><td><font size="2">G</font></td><td><a href="http://facebook.com/KilledByPolice/posts/1033800406648096" target="new"></a><center><a href="http://facebook.com/KilledByPolice/posts/1033800406648096" target="new"><font size="2">facebook.com/KilledByPolice/posts/1033800406648096    </font></a></center></td><td><a href="http://www.mlive.com/news/detroit/index.ssf/2013/09/detroit_transit_officer_charge.html" target="new"><font size="2">http://www.mlive.com/news/detroit/index.ssf/2013/09/detroit_transit_officer_charge.html</font></a><font size="2"><br/><i>Detroit transit officer not guilty in fatal shooting: </i><a href="http://www.clickondetroit.com/news/detroit-transit-officer-not-guilty-in-fatal-shooting/32405878" target="new">http://www.clickondetroit.com/news/detroit-transit-officer-not-guilty-in-fatal-shooting/32405878

</a></font></td></tr></tbody></table>

但是只需要调用find_all函数：
print(table.find_all("tr"))

给你：
 AttributeError: 'NoneType' object has no attribute 'next_element'

HTML太混乱了，不幸的是我无法看到使用bs4解析它的简单方法，这可能是你需要使用一些正则表达式的罕见情况之一。
好的，谢谢你的帮助。我担心 HTML 是问题所在。 - user5037941
网页内容由stack overflow 提供, 点击上面的
可以查看英文原文，
原文链接