logo标签列表

无法在使用Docker和xvfb的AWS Lambda上运行Playwright

amazon-web-servicesdockeraws-lambdaplaywrightxvfb
9

我已经尝试了几天在AWS Lambda上使用xvfb运行Playwright。我能够构建一些适用于本地的Docker镜像，但是当推送到AWS时总是遇到错误。

Dockerfile：

FROM mcr.microsoft.com/playwright/python:v1.24.0-focal 
COPY index.py /
COPY start.sh /
RUN apt install xvfb
RUN chmod 755 /start.sh
CMD ["/bin/bash", "./start.sh"]

start.sh

#!/bin/bash
xvfb-run --auto-servernum --server-num 1 --server-args "-screen 0, 1920x1080x24" python index.py

index.py

from playwright.sync_api import sync_playwright

def lambda_handler(event, context):
    with sync_playwright() as p:
        browser = p.chromium.launch(headless=False)
        page = browser.new_page()
        page.set_default_timeout(0)
        page.goto("http://example.com")
        title = page.title()
        browser.close()
        return title

看起来由于某些权限相关的原因，浏览器无法启动 - 有什么想法是什么导致了这个问题，以及如何解决它？

2022-08-30T09:50:28.053+02:00   Traceback (most recent call last):
2022-08-30T09:50:28.053+02:00   File "index.py", line 8, in lambda_handler
2022-08-30T09:50:28.053+02:00   page = browser.new_page()
2022-08-30T09:50:28.053+02:00   File "/usr/local/lib/python3.8/dist-packages/playwright/sync_api/_generated.py", line 11188, in new_page
2022-08-30T09:50:28.053+02:00   self._sync(
2022-08-30T09:50:28.053+02:00   File "/usr/local/lib/python3.8/dist-packages/playwright/_impl/_sync_base.py", line 89, in _sync
2022-08-30T09:50:28.053+02:00   return task.result()
2022-08-30T09:50:28.053+02:00   File "/usr/local/lib/python3.8/dist-packages/playwright/_impl/_browser.py", line 174, in new_page
2022-08-30T09:50:28.053+02:00   page = await context.new_page()
2022-08-30T09:50:28.053+02:00   File "/usr/local/lib/python3.8/dist-packages/playwright/_impl/_browser_context.py", line 230, in new_page
2022-08-30T09:50:28.053+02:00   return from_channel(await self._channel.send("newPage"))
2022-08-30T09:50:28.053+02:00   File "/usr/local/lib/python3.8/dist-packages/playwright/_impl/_connection.py", line 43, in send
2022-08-30T09:50:28.053+02:00   return await self._connection.wrap_api_call(
2022-08-30T09:50:28.053+02:00   File "/usr/local/lib/python3.8/dist-packages/playwright/_impl/_connection.py", line 369, in wrap_api_call
2022-08-30T09:50:28.053+02:00   return await cb()
2022-08-30T09:50:28.053+02:00   File "/usr/local/lib/python3.8/dist-packages/playwright/_impl/_connection.py", line 78, in inner_send
2022-08-30T09:50:28.053+02:00   result = next(iter(done)).result()
2022-08-30T09:50:28.053+02:00   playwright._impl._api_types.Error: Browser closed.
2022-08-30T09:50:28.053+02:00   ==================== Browser output: ====================
2022-08-30T09:50:28.053+02:00    <launching> /ms-playwright/chromium-1015/chrome-linux/chrome --disable-field-trial-config --disable-background-networking --enable-features=NetworkService,NetworkServiceInProcess --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-back-forward-cache --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=ImprovedCookieControls,LazyFrameLoading,GlobalMediaControls,DestroyProfileOnBrowserClose,MediaRouter,DialMediaRouteProvider,AcceptCHFrame,AutoExpandDetailsElement,CertificateTransparencyComponentUpdater,AvoidUnnecessaryBeforeUnloadCheckSync --allow-pre-commit-input --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --disable-sync --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --no-service-autorun --export-tagged-pdf --no-sandbox --user-data-dir=/tmp/playwright_chromiumdev_profile-gHIdYn --remote-debugging-pipe --no-startup-window
<launching> /ms-playwright/chromium-1015/chrome-linux/chrome --disable-field-trial-config --disable-background-networking --enable-features=NetworkService,NetworkServiceInProcess --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-back-forward-cache --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=ImprovedCookieControls,LazyFrameLoading,GlobalMediaControls,DestroyProfileOnBrowserClose,MediaRouter,DialMediaRouteProvider,AcceptCHFrame,AutoExpandDetailsElement,CertificateTransparencyComponentUpdater,AvoidUnnecessaryBeforeUnloadCheckSync --allow-pre-commit-input --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --disable-sync --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --no-service-autorun --export-tagged-pdf --no-sandbox --user-data-dir=/tmp/playwright_chromiumdev_profile-gHIdYn --remote-debugging-pipe --no-startup-window
2022-08-30T09:50:28.053+02:00   <launched> pid=129
2022-08-30T09:50:28.053+02:00
[pid=129][err] [0830/075021.572233:WARNING:crashpad_client_linux.cc(362)] prctl: Operation not permitted (1)
[pid=129][err] [0830/075021.572233:WARNING:crashpad_client_linux.cc(362)] prctl: Operation not permitted (1)
2022-08-30T09:50:28.053+02:00   [pid=129][err] prctl(PR_SET_NO_NEW_PRIVS) failed
2022-08-30T09:50:28.053+02:00   [pid=129][err] prctl(PR_SET_NO_NEW_PRIVS) failed
2022-08-30T09:50:28.053+02:00
[pid=129][err] [129:147:0830/075023.537911:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
[pid=129][err] [129:147:0830/075023.537911:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:147:0830/075023.538017:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:146:0830/075023.774216:ERROR:bus.cc(398)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:146:0830/075023.774254:ERROR:bus.cc(398)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
2022-08-30T09:50:28.053+02:00   [pid=129][err] prctl(PR_SET_NO_NEW_PRIVS) failed
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:146:0830/075024.183157:ERROR:bus.cc(398)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:146:0830/075024.183204:ERROR:bus.cc(398)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075024.449390:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075024.453692:ERROR:network_service_instance_impl.cc(461)] Network service crashed, restarting service.
2022-08-30T09:50:28.053+02:00   [pid=129][err] prctl(PR_SET_NO_NEW_PRIVS) failed
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075024.510964:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:161:0830/075024.511033:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:161:0830/075024.511061:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:161:0830/075024.511109:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:161:0830/075024.512161:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:161:0830/075024.512820:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075024.570748:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075024.632383:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075024.671409:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075024.674327:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:146:0830/075024.854785:ERROR:bus.cc(398)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:146:0830/075024.854822:ERROR:bus.cc(398)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075025.701411:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075025.712194:ERROR:network_service_instance_impl.cc(461)] Network service crashed, restarting service.
2022-08-30T09:50:28.053+02:00   [pid=129][err] prctl(PR_SET_NO_NEW_PRIVS) failed
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075025.793276:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [0830/075025.850673:ERROR:scoped_ptrace_attach.cc(27)] ptrace: Operation not permitted (1)
2022-08-30T09:50:28.053+02:00   [pid=129][err] Received signal 11 SEGV_MAPERR 000000000000
2022-08-30T09:50:28.053+02:00   [pid=129][err] #0 0x55fa8ab9efb9 base::debug::CollectStackTrace()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #1 0x55fa8ab11853 base::debug::StackTrace::StackTrace()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #2 0x55fa8ab9eac1 base::debug::(anonymous namespace)::StackDumpSignalHandler()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #3 0x7f93d4dc4420 (/usr/lib/x86_64-linux-gnu/libpthread-2.31.so+0x1441f)
2022-08-30T09:50:28.053+02:00   [pid=129][err] #4 0x55fa88ae4589 content::WebContentsImpl::SetDeviceEmulationSize()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #5 0x55fa8864714b content::protocol::EmulationHandler::SetDeviceMetricsOverride()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #6 0x55fa886476e8 content::protocol::EmulationHandler::SetDeviceMetricsOverride()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #7 0x55fa884a52af content::protocol::Emulation::DomainDispatcherImpl::setDeviceMetricsOverride()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #8 0x55fa89b0c0c3 v8_crdtp::UberDispatcher::DispatchResult::Run()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #9 0x55fa886286fd content::DevToolsSession::HandleCommandInternal()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #10 0x55fa8862866b content::DevToolsSession::HandleCommand()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #11 0x55fa8862ac6e base::internal::Invoker<>::RunOnce()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #12 0x55fa8d741a18 ChromeDevToolsSession::HandleCommand()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #13 0x55fa8d740048 ChromeDevToolsManagerDelegate::HandleCommand()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #14 0x55fa8862858e content::DevToolsSession::DispatchProtocolMessageInternal()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #15 0x55fa8862823f content::DevToolsSession::DispatchProtocolMessage()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #16 0x55fa88624162 base::internal::FunctorTraits<>::Invoke<>()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #17 0x55fa8ab5b010 base::TaskAnnotator::RunTaskImpl()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #18 0x55fa8ab6ebfc base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWorkImpl()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #19 0x55fa8ab6e7ad base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWork()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #20 0x55fa8ab6f172 base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWork()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #21 0x55fa8ab1834b base::(anonymous namespace)::WorkSourceDispatch()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #22 0x7f93d4c7917d g_main_context_dispatch
2022-08-30T09:50:28.053+02:00   [pid=129][err] #23 0x7f93d4c79400 (/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.6400.6+0x523ff)
2022-08-30T09:50:28.053+02:00   [pid=129][err] #24 0x7f93d4c794a3 g_main_context_iteration
2022-08-30T09:50:28.053+02:00   [pid=129][err] #25 0x55fa8ab18177 base::MessagePumpGlib::Run()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #26 0x55fa8ab6f3e3 base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::Run()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #27 0x55fa8ab37aed base::RunLoop::Run()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #28 0x55fa8859dcad content::BrowserMainLoop::RunMainMessageLoop()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #29 0x55fa8859f4f2 content::BrowserMainRunnerImpl::Run()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #30 0x55fa8859b7dc content::BrowserMain()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #31 0x55fa8a67ab88 content::RunBrowserProcessMain()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #32 0x55fa8a67c0da content::ContentMainRunnerImpl::RunBrowser()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #33 0x55fa8a67bc70 content::ContentMainRunnerImpl::Run()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #34 0x55fa8a678e1e content::RunContentProcess()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #35 0x55fa8a6797de content::ContentMain()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #36 0x55fa86f8f227 ChromeMain
2022-08-30T09:50:28.053+02:00   [pid=129][err] #37 0x7f93d3eef083 __libc_start_main
2022-08-30T09:50:28.053+02:00   [pid=129][err] #38 0x55fa86f8f02a _start
2022-08-30T09:50:28.053+02:00   [pid=129][err] r8: aaaaaaaaaaaaaaaa r9: 00007fff828b2460 r10: 0000000000000003 r11: 00000000000002d0
2022-08-30T09:50:28.053+02:00   [pid=129][err] r12: 00007fff828b2730 r13: 00007fff828b2460 r14: 0000000000000000 r15: 00000000000002d0
2022-08-30T09:50:28.053+02:00   [pid=129][err] di: 00003ac400a8e070 si: 00007fff828b2460 bp: 00007fff828b2280 bx: 00003ac40108cc00
2022-08-30T09:50:28.053+02:00   [pid=129][err] dx: aaaaaaaaaaaaaaaa ax: 0000000000000000 cx: 000055fa90552dd8 sp: 00007fff828b2230
2022-08-30T09:50:28.053+02:00   [pid=129][err] ip: 000055fa88ae4589 efl: 0000000000010206 cgf: 002b000000000033 erf: 0000000000000004
2022-08-30T09:50:28.053+02:00   [pid=129][err] trp: 000000000000000e msk: 0000000000000000 cr2: 0000000000000000
2022-08-30T09:50:28.053+02:00   [pid=129][err] [end of stack trace]
1个回答
3
关于您的lambda函数，我已经取得了一些进展。首先，我使用AWS SAM命令sam build && sam local invoke [fn-name]在本地测试了您的lambda函数，然后使用sam deploy --guided将堆栈部署到AWS上。为此，我为您的lambda代码创建了一个SAM模板，我使用Playwright镜像作为基础，并在多阶段构建的第二阶段中添加了AWS Lambda运行时接口。
当我在AWS上运行lambda函数时，我注意到如果入口脚本中使用了xvfb-run，则需要添加--single-process启动参数。经过一些测试，我成功在AWS上运行了lambda函数：
browser = p.chromium.launch(headless=True, args=["--single-process"])

很遗憾，即使在入口脚本中使用了xvfb-run，我仍然无法切换到非无头模式。但是，我相信您想使用xvfb-run在虚拟帧缓冲区中查看浏览器窗口。

在以前的项目中，我成功地使用了xvfb和x11vnc与headless=False运行的Playwright docker容器建立了VNC连接。

值得注意的是，连接到Lambda上运行的帧缓冲区可能不可能，因为这将是传入的流量，因此您可能仍需要以无头模式运行Playwright。此外，Lambda最多只运行15分钟。

考虑到这些限制，我建议使用像Fargate这样的服务来运行带有xvfb和x11vnc的Docker镜像。这将为您的项目提供更多的控制和灵活性。此外，我认为允许传入的VNC over SSH流量会更容易。

5
如果我理解正确，@LordWilmore 的意思是开启悬赏并不意味着你可以不遵守礼仪规范（如果我理解有误，请纠正我，LordWilmore）。虽然我同意部分答案通常不应该发布为答案，但我认为这个答案肯定至少有些用处，不值得删除。其他人可能不同意我的看法，但我不支持删除这个答案。 - Michael M.
我遇到了一个相关的问题，通过在启动浏览器时添加 --single-process 参数解决了。 - undefined
网页内容由stack overflow 提供, 点击上面的
可以查看英文原文，
原文链接