如何使用mask_rcnn保存图像结果

Question

我想仅保留使用mask_rcnn的图像中的一部分。例如, 我要从test.jpg中提取仅袋子这个类(class)，并使用python的图像裁剪(crop)方法得到只包含该袋子的图像，但是代码过于复杂难以理解。

# Load a random image from the images folder
file_names = next(os.walk(IMAGE_DIR))[2]
image = skimage.io.imread(os.path.join(IMAGE_DIR, "test2.jpg"))

# Run detection
results = model.detect([image], verbose=1)

# Visualize results
r = results[0]
visualize.display_instances(image, r['rois'], r['masks'], r['class_ids'], 
                            class_names, r['scores'])

使用 Python crop 命令，将仅包含一个袋子的图像裁剪并保存在名为 test.jpg 的文件中。

- liyan

Answer 1

你可以通过编辑 visualize.py 并在代码 ax.imshow(masked_image.astype(np.uint8)) 后插入 plt.savefig('your_desired_path_to_image.jpg',bbox_inches='tight', pad_inches=-0.5,orientation= 'landscape') 来保存图像。这里是链接 https://github.com/matterport/Mask_RCNN/issues/134。

- Ankin_

---

如何保存到GDrive？我尝试了plt.savefig('/content/Mask_RCNN/out.png',bbox_inches='tight', pad_inches=-0.5,orientation= 'landscape')。 - aldo

你解决了吗？检查一下你传递的路径是否正确，并且你能否访问到该驱动器。 - undefined

Answer 2

如果我理解正确，您想要提取/裁剪出特定类别的图像。以下是方法：

# Load a random image from the images folder
file_names = next(os.walk(IMAGE_DIR))[2]
image = skimage.io.imread(os.path.join(IMAGE_DIR, "test2.jpg"))

# Run detection
results = model.detect([image], verbose=1)
r = results[0]

r 是一个 Python 字典，其中 r['rois'] 是预测框（ndarray）的坐标，r['class_ids'] 是相应的类别 ID（ndarray）。 假设需要裁剪的类别是 2（class_id）。

class_id = 2

images_cropped = []
class_fltr = r['class_ids'] == class_id
boxes = r['rois'][class_fltr, :]
for box in boxes:
  y1, x1, y2, x2 = box
  cropped = image[y1: y2, x1: x2]
  images_cropped.append(cropped)

所有被裁剪的图像（如果有的话）都会被附加在列表images_cropped中。 您可以按如下方式绘制图像：

import matplotib.pyplot as plt
img = images_cropped[0]
plt.imshow(img)

- utkarsh-21st

---

在裁剪图像的同时，我们如何获取类名？ - Cybermakarov

Answer 3

确保您已经设置了 "os.directory"。

from PIL import Image
import numpy as np

def display_instances(image, boxes, masks, class_ids, class_names,
                  scores=None, title="",
                  figsize=(16, 16), ax=None,
                  show_mask=True, show_bbox=True,
                  colors=None, captions=None):
"""
boxes: [num_instance, (y1, x1, y2, x2, class_id)] in image coordinates.
masks: [height, width, num_instances]
class_ids: [num_instances]
class_names: list of class names of the dataset
scores: (optional) confidence scores for each box
title: (optional) Figure title
show_mask, show_bbox: To show masks and bounding boxes or not
figsize: (optional) the size of the image
colors: (optional) An array or colors to use with each object
captions: (optional) A list of strings to use as captions for each object
"""
# Number of instances
N = boxes.shape[0]
if not N:
    print("\n*** No instances to display *** \n")
else:
    assert boxes.shape[0] == masks.shape[-1] == class_ids.shape[0]

# If no axis is passed, create one and automatically call show()
auto_show = False
if not ax:
    _, ax = plt.subplots(1, figsize=figsize)
    auto_show = True

# Generate random colors
colors = colors or random_colors(N)

# Show area outside image boundaries.
height, width = image.shape[:2]
ax.set_ylim(height + 10, -10)
ax.set_xlim(-10, width + 10)
ax.axis('off')
ax.set_title(title)

masked_image = image.astype(np.uint32).copy()
for i in range(N):
    color = colors[i]

    # Bounding box
    if not np.any(boxes[i]):
        # Skip this instance. Has no bbox. Likely lost in image cropping.
        continue
    y1, x1, y2, x2 = boxes[i]
    if show_bbox:
        p = patches.Rectangle((x1, y1), x2 - x1, y2 - y1, linewidth=2,
                            alpha=0.7, linestyle="dashed",
                            edgecolor=color, facecolor='none')
        ax.add_patch(p)

    # Label
    if not captions:
        class_id = class_ids[i]
        score = scores[i] if scores is not None else None
        label = class_names[class_id]
        caption = "{} {:.3f}".format(label, score) if score else label
    else:
        caption = captions[i]
    ax.text(x1, y1 + 8, caption,
            color='w', size=11, backgroundcolor="none")

    # Mask
    mask = masks[:, :, i]
    if show_mask:
        masked_image = apply_mask(masked_image, mask, color)

    # Mask Polygon
    # Pad to ensure proper polygons for masks that touch image edges.
    padded_mask = np.zeros(
        (mask.shape[0] + 2, mask.shape[1] + 2), dtype=np.uint8)
    padded_mask[1:-1, 1:-1] = mask
    contours = find_contours(padded_mask, 0.5)
    for verts in contours:
        # Subtract the padding and flip (y, x) to (x, y)
        verts = np.fliplr(verts) - 1
        p = Polygon(verts, facecolor="none", edgecolor=color)
        ax.add_patch(p)
ax.imshow(masked_image.astype(np.uint8))
img1 = Image.fromarray(masked_image.astype(np.uint8), 'RGB')
img1.save('my.png')
img1.show()

if auto_show:
    plt.show()

- Abdul Rehman