如何去除乳腺X线检查标记伪影

Question

我有一个乳腺X光图像数据集（mini DDSM）。这些图片展示字母图案，指示左或右乳房，以及其他对我的机器学习模型无用的信息，因此我想在训练模型之前筛选这个数据集。

在本文中，他们使用Otsu二值化和开运算来清洁乳腺X光图像（第5/10页），链接如下：Preprocessing of Digital Mammogram Image Based on Otsu’s Threshold。

以下是他们的结果：

点此查看

到目前为止，我已经编写了以下代码：

im = io.imread('/content/drive/MyDrive/TFM/DDSMPNG/ALL2/0.jpg')

# thresholding
thresh = im > filters.threshold_otsu(im)

# opening with a disk structure
disk = morphology.disk(5)
opened = morphology.binary_opening(thresh,disk)

# plotting

plt.figure(figsize=(10, 10))

plt.subplot(131)
plt.imshow(im,cmap='gray')
plt.subplot(132)
plt.imshow(opened,cmap='gray')

plt.imsave('/content/drive/MyDrive/TFM/DDSMPNG/Blackened/0.jpg',opened)

以下是图表:

结果

我还尝试了更高的磁盘形状来做开运算，它似乎能够去除一些小字母形状的白色区域，但也会对乳腺X线照片进行一定程度的裁剪:

disk = morphology.disk(45)
opened = morphology.binary_opening(thresh,disk)

结果：

带有圆盘形状 (45,45) 的结果

我猜我需要用二值化创建某种掩膜，并将其应用于原始图像，但我对图像处理库并不熟悉，也不确定如何实现这些结果。

编辑1：我尝试了 @fmw42 的建议，但遇到了一些问题（我在 Google Colab 上工作，不知道是否有关系...）：

首先，在你的代码中使用的示例图像上，它似乎无法正常工作，不知道为什么，我只是修改了图像路径以及一些子绘图以查看结果：

# read image
img = cv2.imread('/content/drive/MyDrive/TFM/DDSMPNG/ALL2/0.jpg')
hh, ww = img.shape[:2]

# convert to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# apply otsu thresholding
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_OTSU )[1] 

# apply morphology close to remove small regions
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5,5))
morph = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel)

# apply morphology open to separate breast from other regions
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5,5))
morph = cv2.morphologyEx(morph, cv2.MORPH_OPEN, kernel)

# get largest contour
contours = cv2.findContours(morph, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
big_contour = max(contours, key=cv2.contourArea)

# draw largest contour as white filled on black background as mask
mask = np.zeros((hh,ww), dtype=np.uint8)
cv2.drawContours(mask, [big_contour], 0, 255, cv2.FILLED)

# dilate mask
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (55,55))
mask = cv2.morphologyEx(mask, cv2.MORPH_DILATE, kernel)

# apply mask to image
result = cv2.bitwise_and(img, img, mask=mask)

# save results

cv2.imwrite('/content/drive/MyDrive/TFM/DDSMPNG/Blackened/0.jpg', result)

# show resultls

plt.figure(figsize=(10, 10))

plt.subplot(141)
plt.imshow(thresh,cmap='gray')
plt.subplot(142)
plt.imshow(morph,cmap='gray')
plt.subplot(143)
plt.imshow(mask,cmap='gray')
plt.subplot(144)
plt.imshow(result,cmap='gray')

结果：

其次，对于其余的图像，它似乎对大多数图像都有效，但是会剪切一些乳房表面：

在您的结果图像中，它似乎更加平滑，我该如何实现它？

先行致谢！

编辑2：@fmw42的解决方案很好用，如果有人遇到相同的问题，您只需要调整形态学滤波器的核大小，直到图像的表现方式与他在答案中的结果相似。

非常感谢！

- user15468984

---

他们使用连通组件来过滤掉较小的区域，仅保留乳房区域。您可以采用同样的方法或更简单的方法来获取轮廓并过滤所有除最大轮廓之外的其他轮廓。 - fmw42

Answer 1

以下是使用Python/OpenCV处理图像的一种方法。

 - Read the input
 - Convert to grayscale
 - Otsu threshold
 - Morphology processing
 - Get largest contour from external contours
 - Draw all contours as white filled on a black background except the largest as a mask and invert mask
 - Apply the mask to the input image
 - Save the results

输入：

import cv2
import numpy as np

# read image
img = cv2.imread("mammogram.png")
hh, ww = img.shape[:2]

# convert to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# apply otsu thresholding
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_OTSU )[1] 

# apply morphology close to remove small regions
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5,5))
morph = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel)

# apply morphology open to separate breast from other regions
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5,5))
morph = cv2.morphologyEx(morph, cv2.MORPH_OPEN, kernel)

# apply morphology dilate to compensate for otsu threshold not getting some areas
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (29,29))
morph = cv2.morphologyEx(morph, cv2.MORPH_DILATE, kernel)

# get largest contour
contours = cv2.findContours(morph, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
big_contour = max(contours, key=cv2.contourArea)
big_contour_area = cv2.contourArea(big_contour)

# draw all contours but the largest as white filled on black background as mask
mask = np.zeros((hh,ww), dtype=np.uint8)
for cntr in contours:
    area = cv2.contourArea(cntr)
    if area != big_contour_area:
        cv2.drawContours(mask, [cntr], 0, 255, cv2.FILLED)
    
# invert mask
mask = 255 - mask

# apply mask to image
result = cv2.bitwise_and(img, img, mask=mask)

# save results
cv2.imwrite('mammogram_thresh.jpg', thresh)
cv2.imwrite('mammogram_morph.jpg', morph)
cv2.imwrite('mammogram_mask.jpg', mask)
cv2.imwrite('mammogram_result.jpg', result)

# show resultls
cv2.imshow('thresh', thresh)
cv2.imshow('morph', morph)
cv2.imshow('mask', mask)
cv2.imshow('result', result)
cv2.waitKey(0)
cv2.destroyAllWindows()

阈值化图像：

形态学处理后的图像：

从轮廓中遮罩图像：

结果图像：

备用

- Read the input
- Convert to grayscale
- Otsu threshold
- Morphology processing
- Get largest contour from external contours
- Draw largest as white filled on black background as a mask 
- Dilate mask
- Apply the mask to the input image
- Save the results

输入：

import cv2
import numpy as np

# read image
img = cv2.imread("mammogram.png")
hh, ww = img.shape[:2]

# convert to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# apply otsu thresholding
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_OTSU )[1] 

# apply morphology close to remove small regions
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5,5))
morph = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel)

# apply morphology open to separate breast from other regions
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5,5))
morph = cv2.morphologyEx(morph, cv2.MORPH_OPEN, kernel)

# get largest contour
contours = cv2.findContours(morph, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
big_contour = max(contours, key=cv2.contourArea)

# draw largest contour as white filled on black background as mask
mask = np.zeros((hh,ww), dtype=np.uint8)
cv2.drawContours(mask, [big_contour], 0, 255, cv2.FILLED)

# dilate mask
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (55,55))
mask = cv2.morphologyEx(mask, cv2.MORPH_DILATE, kernel)

# apply mask to image
result = cv2.bitwise_and(img, img, mask=mask)

# save results
cv2.imwrite('mammogram_thresh.jpg', thresh)
cv2.imwrite('mammogram_morph2.jpg', morph)
cv2.imwrite('mammogram_mask2.jpg', mask)
cv2.imwrite('mammogram_result2.jpg', result)

# show resultls
cv2.imshow('thresh', thresh)
cv2.imshow('morph', morph)
cv2.imshow('mask', mask)
cv2.imshow('result', result)
cv2.waitKey(0)
cv2.destroyAllWindows()

阈值图像：

形态学处理后的图像：

遮罩图像：

结果：

加法

这是应用于您的较大JPG图像的第二种处理方法。我注意到它的宽度和高度大约是6倍。因此，我将形态学核心从5增加到31。我还将图像边框裁剪了40个像素，并添加了相同数量的黑色边框。

输入：

import cv2
import numpy as np

# read image
img = cv2.imread("mammogram.jpg")
hh, ww = img.shape[:2]

# convert to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# shave 40 pixels all around
gray = gray[40:hh-40, 40:ww-40]

# add 40 pixel black border all around
gray = cv2.copyMakeBorder(gray, 40,40,40,40, cv2.BORDER_CONSTANT, value=0)

# apply otsu thresholding
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_OTSU )[1] 

# apply morphology close to remove small regions
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (31,31))
morph = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel)

# apply morphology open to separate breast from other regions
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (31,31))
morph = cv2.morphologyEx(morph, cv2.MORPH_OPEN, kernel)

# get largest contour
contours = cv2.findContours(morph, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
big_contour = max(contours, key=cv2.contourArea)

# draw largest contour as white filled on black background as mask
mask = np.zeros((hh,ww), dtype=np.uint8)
cv2.drawContours(mask, [big_contour], 0, 255, cv2.FILLED)

# dilate mask
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (305,305))
mask = cv2.morphologyEx(mask, cv2.MORPH_DILATE, kernel)

# apply mask to image
result = cv2.bitwise_and(img, img, mask=mask)

# save results
cv2.imwrite('mammogram_thresh.jpg', thresh)
cv2.imwrite('mammogram_morph2.jpg', morph)
cv2.imwrite('mammogram_mask2.jpg', mask)
cv2.imwrite('mammogram_result2.jpg', result)

# show resultls
cv2.imshow('thresh', thresh)
cv2.imshow('morph', morph)
cv2.imshow('mask', mask)
cv2.imshow('result', result)
cv2.waitKey(0)
cv2.destroyAllWindows()

阈值图像：

形态学图像：

口罩图像：

结果：

- fmw42

---

太棒了！！非常感谢，我会尽快试一下，只要我能到我的电脑。 - user15468984

我明白了，关于第一个问题，两种解决方案都不能处理你用作示例的图像？当我复制你的代码进行测试时，在其他乳腺X线照片中运行良好，但在那张照片上无法去除标记... - user15468984

第一种方法在形态滤波器上失败了，它没有像您的形态滤波器结果那样膨胀轮廓。第二种方法在掩模结果上失败了，在我的结果中两者仍然存在，而应该只有乳房轮廓。 - user15468984

OpenCV: 4.1.2 Python: Python 3.7.10 - user15468984

我可以将第一种方法的膨胀核大小从（29,29）编辑为（157,157），这样可以正确地扩张jpg，但是两个轮廓连接在一起，它无法提取最大的轮廓以便仅使用属于标记的最小轮廓创建掩模... - user15468984