使用BeautifulSoup提取不带标签的文本

Question

我的网页看起来像这样：

<p>
  <strong class="offender">YOB:</strong> 1987<br/>
  <strong class="offender">RACE:</strong> WHITE<br/>
  <strong class="offender">GENDER:</strong> FEMALE<br/>
  <strong class="offender">HEIGHT:</strong> 5'05''<br/>
  <strong class="offender">WEIGHT:</strong> 118<br/>
  <strong class="offender">EYE COLOR:</strong> GREEN<br/>
  <strong class="offender">HAIR COLOR:</strong> BROWN<br/>
</p>

我想提取每个人的信息，例如 YOB:1987，RACE:WHITE等等...

我尝试过：

subc = soup.find_all('p')
subc1 = subc[1]
subc2 = subc1.find_all('strong')

但这只给我YOB:，RACE:等的值...

有没有一种方法可以让我以YOB:1987，RACE:WHITE格式获取数据？

- myloginid

Answer 1

只需循环遍历所有的标签，并使用next_sibling方法获取您想要的内容。就像这样：

for strong_tag in soup.find_all('strong'):
    print(strong_tag.text, strong_tag.next_sibling)

示例：

from bs4 import BeautifulSoup

html = '''
<p>
  <strong class="offender">YOB:</strong> 1987<br />
  <strong class="offender">RACE:</strong> WHITE<br />
  <strong class="offender">GENDER:</strong> FEMALE<br />
  <strong class="offender">HEIGHT:</strong> 5'05''<br />
  <strong class="offender">WEIGHT:</strong> 118<br />
  <strong class="offender">EYE COLOR:</strong> GREEN<br />
  <strong class="offender">HAIR COLOR:</strong> BROWN<br />
</p>
'''

soup = BeautifulSoup(html)

for strong_tag in soup.find_all('strong'):
    print(strong_tag.text, strong_tag.next_sibling)

这将为您提供：

YOB:  1987
RACE:  WHITE
GENDER:  FEMALE
HEIGHT:  5'05''
WEIGHT:  118
EYE COLOR:  GREEN
HAIR COLOR:  BROWN

- shaktimaan

Answer 2

我认为您可以使用subc1.text来获取它。

>>> html = """
<p>
    <strong class="offender">YOB:</strong> 1987<br />
    <strong class="offender">RACE:</strong> WHITE<br />
    <strong class="offender">GENDER:</strong> FEMALE<br />
    <strong class="offender">HEIGHT:</strong> 5'05''<br />
    <strong class="offender">WEIGHT:</strong> 118<br />
    <strong class="offender">EYE COLOR:</strong> GREEN<br />
    <strong class="offender">HAIR COLOR:</strong> BROWN<br />
</p>
"""
>>> from bs4 import BeautifulSoup
>>> soup = BeautifulSoup(html)
>>> print soup.text


YOB: 1987
RACE: WHITE
GENDER: FEMALE
HEIGHT: 5'05''
WEIGHT: 118
EYE COLOR: GREEN
HAIR COLOR: BROWN

或者，如果你想要 探索 它，你可以使用 .contents：

>>> p = soup.find('p')
>>> from pprint import pprint
>>> pprint(p.contents)
[u'\n',
 <strong class="offender">YOB:</strong>,
 u' 1987',
 <br/>,
 u'\n',
 <strong class="offender">RACE:</strong>,
 u' WHITE',
 <br/>,
 u'\n',
 <strong class="offender">GENDER:</strong>,
 u' FEMALE',
 <br/>,
 u'\n',
 <strong class="offender">HEIGHT:</strong>,
 u" 5'05''",
 <br/>,
 u'\n',
 <strong class="offender">WEIGHT:</strong>,
 u' 118',
 <br/>,
 u'\n',
 <strong class="offender">EYE COLOR:</strong>,
 u' GREEN',
 <br/>,
 u'\n',
 <strong class="offender">HAIR COLOR:</strong>,
 u' BROWN',
 <br/>,
 u'\n']

筛选出列表中必要的元素：

>>> data = dict(zip([x.text for x in p.contents[1::4]], [x.strip() for x in p.contents[2::4]]))
>>> pprint(data)
{u'EYE COLOR:': u'GREEN',
 u'GENDER:': u'FEMALE',
 u'HAIR COLOR:': u'BROWN',
 u'HEIGHT:': u"5'05''",
 u'RACE:': u'WHITE',
 u'WEIGHT:': u'118',
 u'YOB:': u'1987'}

- Sufian Latif

Answer 3

你可以在findall循环内尝试这个：

item_price = item.find('span', attrs={'class':'s-item__price'}).text

它只提取文本并将其分配给“item_price”。

- Mohamed Saeed

Answer 4

我认为你可以使用gazpacho中的.strip()来解决这个问题：

输入：

html = """\
<p>
  <strong class="offender">YOB:</strong> 1987<br />
  <strong class="offender">RACE:</strong> WHITE<br />
  <strong class="offender">GENDER:</strong> FEMALE<br />
  <strong class="offender">HEIGHT:</strong> 5'05''<br />
  <strong class="offender">WEIGHT:</strong> 118<br />
  <strong class="offender">EYE COLOR:</strong> GREEN<br />
  <strong class="offender">HAIR COLOR:</strong> BROWN<br />
</p>
"""

代码：

soup = Soup(html)
text = soup.find("p").strip(whitespace=False) # to keep \n characters intact
lines = [
    line.strip()
    for line in text.split("\n")
    if line != ""
]
data = dict([line.split(": ") for line in lines])

输出：

print(data)
# {'YOB': '1987',
#  'RACE': 'WHITE',
#  'GENDER': 'FEMALE',
#  'HEIGHT': "5'05''",
#  'WEIGHT': '118',
#  'EYE COLOR': 'GREEN',
#  'HAIR COLOR': 'BROWN'}

- emehex