MaskRCNN的分割IoU指标为什么等于0？

Question

当我在我的多类实例分割自定义数据集上训练MaskRCNN时，输入格式为：

image   -)  shape: torch.Size([3, 850, 600]),   dtype: torch.float32, min: tensor(0.0431),               max: tensor(0.9137)
boxes   -)  shape: torch.Size([4, 4]),          dtype: torch.float32, min: tensor(47.),                  max: tensor(807.)
masks   -)  shape: torch.Size([850, 600, 600]), dtype: torch.uint8,   min: tensor(0, dtype=torch.uint8), max: tensor(1, dtype=torch.uint8)
areas   -)  shape: torch.Size([4]),             dtype: torch.float32, min: tensor(1479.),                max: tensor(8014.)
labels  -)  shape: torch.Size([4]),             dtype: torch.int64,   min: tensor(1),                    max: tensor(1)
iscrowd -)  shape: torch.Size([4]),             dtype: torch.int64,   min: tensor(0),                    max: tensor(0)

我始终如一地获得以下所示的所有分割IoU指标：

DONE (t=0.03s).
IoU metric: bbox
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.004
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.010
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.004
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.001
IoU metric: segm
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.000
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.000
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000

我该怎么思考、调试和修复这个问题？

- Índio

Answer 1

由于您的输入图像大小为（850，600）（H，W），并且考虑到对于此给定图像，您有4个对象，而不是850个带有（600，600）掩码。因此，您的掩码张量应具有维度（对象数量，850，600），因此您的输入应为:

image   -)  shape: torch.Size([3, 850, 600]),   dtype: torch.float32, min: tensor(0.0431),               max: tensor(0.9137)
boxes   -)  shape: torch.Size([4, 4]),          dtype: torch.float32, min: tensor(47.),                  max: tensor(807.)
masks   -)  shape: torch.Size([4, 850, 600]), dtype: torch.uint8,   min: tensor(0, dtype=torch.uint8), max: tensor(1, dtype=torch.uint8)
areas   -)  shape: torch.Size([4]),             dtype: torch.float32, min: tensor(1479.),                max: tensor(8014.)
labels  -)  shape: torch.Size([4]),             dtype: torch.int64,   min: tensor(1),                    max: tensor(1)
iscrowd -)  shape: torch.Size([4]),             dtype: torch.int64,   min: tensor(0),                    max: tensor(0)

如何修复它 由于您正在尝试解决实例分割问题，请确保将每个（850, 600）掩码堆叠以产生一个张量，形状为（掩码数量，850，600）。

- Rakesh Gupta