AWS Beanstalk中Docker异常：`shim reaped`

Question

我正在使用AWS beanstalk上的docker环境部署一个应用程序（nodejs websocket服务器）到生产环境中。

周期性地，容器会“崩溃”（实际上是容器中的主要进程重新启动），但我无法弄清原因。 在这时候，/var/log/docker中包含以下日志：

time="2018-12-07T00:48:46Z" level=info msg="shim reaped" id=0af18fa159c07b167a29012b34c6c925c877f98d9a09dcd67078aa6c12f4ef2f 
time="2018-12-07T00:48:46.052832134Z" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
time="2018-12-07T00:48:46Z" level=info msg="shim docker-containerd-shim started" address="/containerd-shim/moby/0af18fa159c07b167a29012b34c6c925c877f98d9a09dcd67078aa6c12f4ef2f/shim.sock" debug=false pid=9192

此时CPU和RAM似乎正常。有人知道是怎么回事吗？

编辑：还有其他日志，但我怀疑这是后果：

/var/log/nginx/error.log:

2018/12/07 00:48:45 [error] 4268#0: *10397 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: 172.31.43.209, server: , request: "GET /stream?s=000 HTTP/1.1", upstream: "http://172.17.0.2:80/stream?s=000", host: "..."
2018/12/07 00:48:45 [error] 4268#0: *1009 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: 172.31.43.209, server: , request: "GET /stream?s=000 HTTP/1.1", upstream: "http://172.17.0.2:80/stream?s=000", host: "..."
2018/12/07 00:48:46 [error] 4267#0: *11092 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.12.149, server: , request: "GET /stream?s=000 HTTP/1.1", upstream: "http://172.17.0.2:80/stream?s=000", host: "..."

/var/log/docker-events.log

2018-12-07T00:48:46.052880449Z container die 0af18fa159c07b167a29012b34c6c925c877f98d9a09dcd67078aa6c12f4ef2f (exitCode=1, image=2fc4abcada2b, name=inspiring_euler)
2018-12-07T00:48:46.176330610Z network disconnect 94c449d445a5a434af70517a1c8734c540c5c1f9ddbbc1a53a002f25dbc7f581 (container=0af18fa159c07b167a29012b34c6c925c877f98d9a09dcd67078aa6c12f4ef2f, name=bridge, type=bridge)
2018-12-07T00:48:46.626514590Z network connect 94c449d445a5a434af70517a1c8734c540c5c1f9ddbbc1a53a002f25dbc7f581 (container=0af18fa159c07b167a29012b34c6c925c877f98d9a09dcd67078aa6c12f4ef2f, name=bridge, type=bridge)
2018-12-07T00:48:46.869988171Z container start 0af18fa159c07b167a29012b34c6c925c877f98d9a09dcd67078aa6c12f4ef2f (image=2fc4abcada2b, name=inspiring_euler)

- p9f

---

请问您能否检查一下您实际拥有的内存是否足够？docker run -it kunalkushwaha/high-mem-allocator 4 可以模拟内存使用情况。 - theshemul

Answer 1

这个失败可能是由于在启用THP（透明大页）的系统上运行containerd导致的。内存管理方案与容器的内存分配模式不对齐，从而导致失败。类似的问题在https://github.com/containerd/containerd/issues/2202上有报道。

不幸的是，您无法调整Elastic Beanstalk主机的内核设置以解决此问题。该解决方案已记录在mongodb文档中，因为它也存在与THP类似的问题。

https://docs.mongodb.com/manual/tutorial/transparent-huge-pages/

- Oxidizing1