logo标签列表

pq: 无法调整共享内存段大小。设备上没有剩余空间。

postgresqldockergrafana
78

我在一个仪表盘上展示数据点图表，使用查询docker化的PostgreSQL数据库实例来制作。面板数量大约为6个。

这些面板以前都很正常工作，但最近有些停止了并报告了这样的错误：

pq: could not resize shared memory segment "/PostgreSQL.2058389254" to 12615680 bytes: No space left on device

这是什么原因？如何解决？Docker容器在通过ssh访问的远程主机上运行。

编辑

磁盘空间：

$df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1       197G  140G   48G  75% /
devtmpfs        1.4G     0  1.4G   0% /dev
tmpfs           1.4G  4.0K  1.4G   1% /dev/shm
tmpfs           1.4G  138M  1.3G  10% /run
tmpfs           1.4G     0  1.4G   0% /sys/fs/cgroup
/dev/dm-16       10G   49M   10G   1% /var/lib/docker/devicemapper/mnt/a0f3c5ab84aa06d5b2db00c4324dd6bf7141500ff4c83e23e9aba7c7268bcad4
/dev/dm-1        10G  526M  9.5G   6% /var/lib/docker/devicemapper/mnt/8623a774d736ed3dc0d2db89b7d07cae85c3d1bcafc245180eec4ffd738f93a5
shm              64M     0   64M   0% /var/lib/docker/containers/260552ebcdf2bf0961329108d3d975110f8ada0a41325f5e7dd81b8ddad9d18b/mounts/shm
/dev/dm-4        10G  266M  9.8G   3% /var/lib/docker/devicemapper/mnt/6f873e62607e7cac4c4b658c72874c787b90290f74d1159eca81af61cb467cfb
shm              64M   50M   15M  78% /var/lib/docker/containers/84c66d9fb5b6ae023d051766f4d35ced87a519a1fee68ca5c89d61ff87cf1e5a/mounts/shm
/dev/dm-2        10G  383M  9.7G   4% /var/lib/docker/devicemapper/mnt/cb3df1ae654ed78802c2e5bd7a51a1b0bdd562855a7c7803750b80b33f5c206e
shm              64M     0   64M   0% /var/lib/docker/containers/22ba2ae2b6859c24623703dcb596527d64257d2d61de53f4d88e00a8e2335211/mounts/shm
/dev/dm-3        10G   99M  9.9G   1% /var/lib/docker/devicemapper/mnt/492a19fc8f3e254c4e5cc691c3300b5fee9d1a849422673bf0c19b4b2d1db571
shm              64M     0   64M   0% /var/lib/docker/containers/39abe855a9b107d4921807332309517697f024b2d169ebc5f409436208f766d0/mounts/shm
/dev/dm-7        10G  276M  9.8G   3% /var/lib/docker/devicemapper/mnt/55c6a6c17c892d149c1cc91fbf42b98f1340ffa30a1da508e3526af7060f3ce2
shm              64M     0   64M   0% /var/lib/docker/containers/bf2e7254cd7e2c6000da61875343580ec6ff5cbf40c017a398ba7479af5720ec/mounts/shm
/dev/dm-8        10G  803M  9.3G   8% /var/lib/docker/devicemapper/mnt/4e51f48d630041316edd925f1e20d3d575fce4bf19ef39a62756b768460d1a3a
shm              64M     0   64M   0% /var/lib/docker/containers/72d4ae743de490ed580ec9265ddf8e6b90e3a9d2c69bd74050e744c8e262b342/mounts/shm
/dev/dm-6        10G   10G   20K 100% /var/lib/docker/devicemapper/mnt/3dcddaee736017082fedb0996e42b4c7b00fe7b850d9a12c81ef1399fa00dfa5
shm              64M     0   64M   0% /var/lib/docker/containers/9f2bf4e2736d5128d6c240bb10da977183676c081ee07789bee60d978222b938/mounts/shm
/dev/dm-5        10G  325M  9.7G   4% /var/lib/docker/devicemapper/mnt/65a2bf48cbbfe42f0c235493981e62b90363b4be0a2f3aa0530bbc0b5b29dbe3
shm              64M     0   64M   0% /var/lib/docker/containers/e53d5ababfdefc5c8faf65a4b2d635e2543b5a807b65a4f3cd8553b4d7ef2d06/mounts/shm
/dev/dm-9        10G  1.2G  8.9G  12% /var/lib/docker/devicemapper/mnt/3216c48346c3702a5cd2f62a4737cc39666983b8079b481ab714cdb488400b08
shm              64M     0   64M   0% /var/lib/docker/containers/5cd0774a742f54c7c4fe3d4c1307fc93c3c097a861cde5f611a0fa9b454af3dd/mounts/shm
/dev/dm-10       10G  146M  9.9G   2% /var/lib/docker/devicemapper/mnt/6a98acd1428ae670e8f1da62cb8973653c8b11d1c98a8bf8be78f59d2ddba062
shm              64M     0   64M   0% /var/lib/docker/containers/a878042353f6a605167e7f9496683701fd2889f62ba1d6c0dc39c58bc03a8209/mounts/shm
tmpfs           285M     0  285M   0% /run/user/0

编辑二

$df -ih
Filesystem     Inodes IUsed IFree IUse% Mounted on
/dev/vda1         13M  101K   13M    1% /
devtmpfs         354K   394  353K    1% /dev
tmpfs            356K     2  356K    1% /dev/shm
tmpfs            356K   693  356K    1% /run
tmpfs            356K    16  356K    1% /sys/fs/cgroup
/dev/dm-16        10M  2.3K   10M    1% /var/lib/docker/devicemapper/mnt/a0f3c5ab84aa06d5b2db00c4324dd6bf7141500ff4c83e23e9aba7c7268bcad4
/dev/dm-1         10M   19K   10M    1% /var/lib/docker/devicemapper/mnt/8623a774d736ed3dc0d2db89b7d07cae85c3d1bcafc245180eec4ffd738f93a5
shm              356K     1  356K    1% /var/lib/docker/containers/260552ebcdf2bf0961329108d3d975110f8ada0a41325f5e7dd81b8ddad9d18b/mounts/shm
/dev/dm-4         10M   11K   10M    1% /var/lib/docker/devicemapper/mnt/6f873e62607e7cac4c4b658c72874c787b90290f74d1159eca81af61cb467cfb
shm              356K     2  356K    1% /var/lib/docker/containers/84c66d9fb5b6ae023d051766f4d35ced87a519a1fee68ca5c89d61ff87cf1e5a/mounts/shm
/dev/dm-2         10M  5.6K   10M    1% /var/lib/docker/devicemapper/mnt/cb3df1ae654ed78802c2e5bd7a51a1b0bdd562855a7c7803750b80b33f5c206e
shm              356K     1  356K    1% /var/lib/docker/containers/22ba2ae2b6859c24623703dcb596527d64257d2d61de53f4d88e00a8e2335211/mounts/shm
/dev/dm-3         10M  4.6K   10M    1% /var/lib/docker/devicemapper/mnt/492a19fc8f3e254c4e5cc691c3300b5fee9d1a849422673bf0c19b4b2d1db571
shm              356K     1  356K    1% /var/lib/docker/containers/39abe855a9b107d4921807332309517697f024b2d169ebc5f409436208f766d0/mounts/shm
/dev/dm-7         10M  7.5K   10M    1% /var/lib/docker/devicemapper/mnt/55c6a6c17c892d149c1cc91fbf42b98f1340ffa30a1da508e3526af7060f3ce2
shm              356K     1  356K    1% /var/lib/docker/containers/bf2e7254cd7e2c6000da61875343580ec6ff5cbf40c017a398ba7479af5720ec/mounts/shm
/dev/dm-8         10M   12K   10M    1% /var/lib/docker/devicemapper/mnt/4e51f48d630041316edd925f1e20d3d575fce4bf19ef39a62756b768460d1a3a
shm              356K     1  356K    1% /var/lib/docker/containers/72d4ae743de490ed580ec9265ddf8e6b90e3a9d2c69bd74050e744c8e262b342/mounts/shm
/dev/dm-6        7.9K  7.3K   623   93% /var/lib/docker/devicemapper/mnt/3dcddaee736017082fedb0996e42b4c7b00fe7b850d9a12c81ef1399fa00dfa5
shm              356K     1  356K    1% /var/lib/docker/containers/9f2bf4e2736d5128d6c240bb10da977183676c081ee07789bee60d978222b938/mounts/shm
/dev/dm-5         10M   27K   10M    1% /var/lib/docker/devicemapper/mnt/65a2bf48cbbfe42f0c235493981e62b90363b4be0a2f3aa0530bbc0b5b29dbe3
shm              356K     1  356K    1% /var/lib/docker/containers/e53d5ababfdefc5c8faf65a4b2d635e2543b5a807b65a4f3cd8553b4d7ef2d06/mounts/shm
/dev/dm-9         10M   53K   10M    1% /var/lib/docker/devicemapper/mnt/3216c48346c3702a5cd2f62a4737cc39666983b8079b481ab714cdb488400b08
shm              356K     1  356K    1% /var/lib/docker/containers/5cd0774a742f54c7c4fe3d4c1307fc93c3c097a861cde5f611a0fa9b454af3dd/mounts/shm
/dev/dm-10        10M  5.2K   10M    1% /var/lib/docker/devicemapper/mnt/6a98acd1428ae670e8f1da62cb8973653c8b11d1c98a8bf8be78f59d2ddba062
shm              356K     1  356K    1% /var/lib/docker/containers/a878042353f6a605167e7f9496683701fd2889f62ba1d6c0dc39c58bc03a8209/mounts/shm
tmpfs            356K     1  356K    1% /run/user/0

编辑-3 postgres 容器服务：

version: "3.5"
services:

#other containers go here..

 postgres:
    restart: always
    image: postgres:10
    hostname: postgres
    container_name: fiware-postgres
    expose:
      - "5432"
    ports:
      - "5432:5432"
    networks:
      - default
    environment:
      - "POSTGRES_PASSWORD=password"
      - "POSTGRES_USER=postgres"
      - "POSTGRES_DB=postgres"
    volumes:
      - ./postgres-data:/var/lib/postgresql/data
    build:
      context: .
      shm_size: '4gb'

数据库大小：

postgres=# SELECT pg_size_pretty( pg_database_size('postgres'));
 pg_size_pretty
----------------
 42 GB
(1 row)

编辑-4

抱歉，与此问题相关的所有解决方法都不起作用，包括这个 在仪表板上，我有5个面板用于显示数据点。查询类似，除了每个查询显示温度、相对湿度、光照度、颗粒和O3的不同参数。这是查询:

SELECT to_timestamp(floor((extract('epoch' from recvtime)/ 1800 )) * 1800) as time,
avg(attrvalue::float) as illuminance
FROM urbansense.weather WHERE attrname='illuminance' AND attrvalue<>'null' GROUP BY time ORDER BY time asc;

区别在于WHERE attrname=#parameterValue语句中。我修改了postgresql.conf文件以写入日志，但是日志似乎没有提供有用的提示：以下是日志：

$ vim postgres-data/log/postgresql-2019-06-26_150012.log
.
.
2019-06-26 15:03:39.298 UTC [45] LOG:  statement: SELECT to_timestamp(floor((extract('epoch' from recvtime)/ 1800 )) * 1800) as time,
        avg(attrvalue::float) as o3
        FROM urbansense.airquality WHERE attrname='O3' AND attrvalue<>'null' GROUP BY time ORDER BY time asc;
2019-06-26 15:03:40.903 UTC [41] ERROR:  could not resize shared memory segment "/PostgreSQL.1197429420" to 12615680 bytes: No space left on device
2019-06-26 15:03:40.903 UTC [41] STATEMENT:  SELECT to_timestamp(floor((extract('epoch' from recvtime)/ 1800 )) * 1800) as time,
        avg(attrvalue::float) as illuminance
        FROM urbansense.weather WHERE attrname='illuminance' AND attrvalue<>'null' GROUP BY time ORDER BY time asc;
2019-06-26 15:03:40.905 UTC [42] FATAL:  terminating connection due to administrator command
2019-06-26 15:03:40.905 UTC [42] STATEMENT:  SELECT to_timestamp(floor((extract('epoch' from recvtime)/ 1800 )) * 1800) as time,
        avg(attrvalue::float) as illuminance
        FROM urbansense.weather WHERE attrname='illuminance' AND attrvalue<>'null' GROUP BY time ORDER BY time asc;
2019-06-26 15:03:40.909 UTC [43] FATAL:  terminating connection due to administrator command
2019-06-26 15:03:40.909 UTC [43] STATEMENT:  SELECT to_timestamp(floor((extract('epoch' from recvtime)/ 1800 )) * 1800) as time,
        avg(attrvalue::float) as illuminance
        FROM urbansense.weather WHERE attrname='illuminance' AND attrvalue<>'null' GROUP BY time ORDER BY time asc;
2019-06-26 15:03:40.921 UTC [1] LOG:  worker process: parallel worker for PID 41 (PID 42) exited with exit code 1
2019-06-26 15:03:40.922 UTC [1] LOG:  worker process: parallel worker for PID 41 (PID 43) exited with exit code 1
2019-06-26 15:07:04.058 UTC [39] LOG:  temporary file: path "base/pgsql_tmp/pgsql_tmp39.0", size 83402752
2019-06-26 15:07:04.058 UTC [39] STATEMENT:  SELECT to_timestamp(floor((extract('epoch' from recvtime)/ 1800 )) * 1800)as time,
        avg(attrvalue::float) as relativeHumidity
        FROM urbansense.weather WHERE attrname='relativeHumidity' AND attrvalue<>'null' GROUP BY time ORDER BY time asc;
2019-06-26 15:07:04.076 UTC [40] LOG:  temporary file: path "base/pgsql_tmp/pgsql_tmp40.0", size 83681280
2019-06-26 15:07:04.076 UTC [40] STATEMENT:  SELECT to_timestamp(floor((extract('epoch' from recvtime)/ 1800 )) * 1800)as time,
        avg(attrvalue::float) as relativeHumidity
        FROM urbansense.weather WHERE attrname='relativeHumidity' AND attrvalue<>'null' GROUP BY time ORDER BY time asc;
2019-06-26 15:07:04.196 UTC [38] LOG:  temporary file: path "base/pgsql_tmp/pgsql_tmp38.0", size 84140032

有没有人有解决这个问题的想法？

你检查过硬盘了吗？你确实有足够的可用空间吗？ - Mostafa Hussein
@MostafaHussein 是的，我有足够的磁盘空间（大约有50GB可用），可以在问题编辑中看到。 - arilwan
你需要检查你的postgresql配置中的共享内存设置，以及docker/主机设置所施加的共享内存限制。 - Richard Huxton
你能检查inode的使用情况吗？可以用命令df -ih。 - mchawre
@mchawre，我在问题编辑2中检查了这个。 - arilwan
6个回答
160
这是因为 Docker 默认情况下 将共享内存大小限制为 64MB。
您可以通过在 docker run 中使用 --shm-size 选项来覆盖此默认值。 docker run -itd --shm-size=1g postgres 或者在 docker-compose 中：
db:
  image: "postgres:11.3-alpine"
  shm_size: 1g

看看这个。更多信息在这里。

2
是的，我仍然明白。即使设置了shm_size: '50gb'也没有改变，最近的错误为：pq: could not resize shared memory segment "/PostgreSQL.1336373456" to 12615680 bytes: No space left on device。 - arilwan
我遇到了同样的问题，而且似乎不是数据库设置（工作进程、内存等）的问题。 - PCamargo
@PCamargo，尝试增加您的postgres数据库共享内存大小（默认为64MB）。 - arilwan
@arilwan 如果您发布一个包含所有细节的单独问题，那将是很好的。 - mchawre
1
它真的起作用了，@arilwan。我不得不创建一个新的容器（无法更改现有容器），但它运行得非常好！ - PCamargo
显示剩余5条评论
31

非常抱歉回复晚了。建立一个新镜像是不必要的。但是你必须确保重新创建了容器，并且没有使用旧容器。 Docker-compose文件必须进行更改，在服务级别上添加shm_size。构建部分是不必要的。

version: "3.5"
services:

#other containers go here..

 postgres:
    restart: always
    image: postgres:10

    #THIS MUST BE ADDED AT SERVICE LEVEL
    shm_size: 1gb 

    hostname: postgres
    container_name: fiware-postgres
    expose:
      - "5432"
    ports:
      - "5432:5432"
    networks:
      - default
    environment:
      - "POSTGRES_PASSWORD=password"
      - "POSTGRES_USER=postgres"
      - "POSTGRES_DB=postgres"
    volumes:
      - ./postgres-data:/var/lib/postgresql/data

那么你必须完全重新创建容器

docker-compose rm postgres 
# alternatively you can docker-compose down to destroy all containers
docker-compose up -d

销毁旧容器并创建新容器。

您可以在容器内检查更改(使用docker-compose exec postgres bash)并运行df -h | grep shm

参考：docker SHM_SIZE /dev/shm: resizing shared memory

1
很好。你可以使用docker-compose rm postgres命令来删除预先存在的Postgres容器，而不是使用docker-compose down命令。 - The Alchemist
@Javier Dottori，解决方案还可以，但是检查shm命令不正确。应该是docker exec postgres df -h |grep shm或者docker exec -it postgres bash，然后连接终端后运行df -h |grep shm。 - André Carvalho
嗨，André，感谢您的建议。无论如何，如果使用docker-compose创建容器，则最好使用其包装器，这样您就不需要为其分配名称。如果您使用命名容器，则需要小心，以便多个docker-compose文件不会在所有计算机上共享名称。docker-compose exec主要是docker exec -ti的包装器，如果要使用docker本机命令，则应使用container_name，即fiware-postgres（我保留它，因为它在原始问题中）。 - Javier Dottori
很好。你更新了docker-compose文件并执行了docker-compose up -d命令。 - LokiAlice
9

您可以在不重新启动/重建容器的情况下通过重新挂载shm来增加其大小

mount -o remount,size=256m -t tmpfs /var/lib/docker/containers/your-container-id/mounts/shm

更改您的容器ID和所需大小（256m）

运行得很好！我正在检查一个长时间运行的进程，我不想中断它并且共享内存用完了。尝试了一下，结果奏效了。 - Stephan
1
最好在docker-compose.yml中加入shm_size: 256m来使用它。有时重新启动容器后，我需要重新挂载它。不知道为什么，但这种情况经常发生。 - Николай Агеев
1
但是我的任务最终还是崩溃了。暂时来说，这是一个快速的解决方案。 - Stephan
1

这个关于共享内存的错误信息是指Postgres自己的work_mem配置 - 默认值只有4 MB。

您可以在命令中指定它：

version: '3.3'
services:
  db:
    image: postgres:12
    command:
     - -c
     - work_mem=64MB

看起来你至少需要 13MB

1

在增加shm_size没有帮助我的情况下，我转向了数据库优化，并发现添加一些缺失的索引可以解决我的特定错误。

具体来说，我有几个视图使用了没有索引的字段进行连接，在数据库达到典型大小（连接的“多”方有1M行）后，这些错误开始出现。

0

您需要使用增加了shm_size的新镜像进行构建

Dockerfile

FROM postgres:10.7

docker-compose.release.yml

version: '3.7'
services:
  postgres:
    image: registry.my-site.com/postgres:latest
    build:
      shm_size: '4gb'

运行

docker-compose -f docker-compose.release.yml build

然后，您可以使用您的镜像 registry.my-site.com/postgres:latest 部署容器，并增加 shm_size

3
构建自定义镜像并不是必需的，因为这是在创建容器时进行设置的（docker run --rm -it --shm-size 2gb....）。 - The Alchemist
网页内容由stack overflow 提供, 点击上面的
可以查看英文原文，
原文链接