我有一个使用无头浏览器的脚本,我正在使用crontab -e
运行。它前几次运行得很好,然后会崩溃,并显示以下Traceback:
Traceback (most recent call last):
File "/home/clint-selenium-firefox.py", line 83, in <module>
driver.get(url)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 248, in get
self.execute(Command.GET, {'url': url})
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Failed to decode response from marionette
我的crontab行是:
*/10 * * * * export DISPLAY=:0 && python /home/clint-selenium-firefox.py >> /home/error.log 2>&1
我不想在这里过多地涉及Python脚本,因此我已经提取出我认为相关的部分。
from pyvirtualdisplay import Display
display = Display(visible=0, size=(800, 600))
display.start()
...
driver = webdriver.Firefox()
driver.get(url)
...
driver.quit()
...
display.stop()
非常感谢你的帮助。
编辑
版本信息:Firefox 49.0.2;Selenium: 3.0.1;geckodriver: geckodriver-v0.11.1-linux64.tar.gz
错误代码(在 driver.get(url)
处失败):
driver = webdriver.Firefox()
if DEBUG: print "Opened Firefox"
for u in urls:
list_of_rows = []
list_of_old_rows = []
# get the old version of the site data
mycsvfile = u[1]
try:
with open(mycsvfile, 'r') as csvfile:
old_data = csv.reader(csvfile, delimiter=' ', quotechar='|')
for o in old_data:
list_of_old_rows.append(o)
except: pass
# get the new data
url = u[0]
if DEBUG: print url
driver.get(url)
if DEBUG: print driver.title
time.sleep(1)
page_source = driver.page_source
soup = bs4.BeautifulSoup(page_source,'html.parser')