使用OpenCV和色卡进行颜色校正

Question

我正在寻找自动完成一些基本的颜色校正的方法，然后我看到了这篇博客文章。

https://www.pyimagesearch.com/2021/02/15/automatic-color-correction-with-opencv-and-python/

python color_correction.py --reference ref.jpg  --input input.jpg

总结一下这篇博客，它可以在给定的输入图像中识别出 Pantone 色卡，并修改直方图以匹配实际颜色。任何由于光照引起的颜色偏移都将在输入的颜色卡中进行调整。

我有一个问题作为博客文章用例的扩展。虽然直方图匹配在两个裁剪到颜色卡边界的图像之间很好地进行 - 但现在仅应用于存在颜色卡的裁剪输入图像。我想在整个输入图像上应用此直方图转换 - 包括颜色卡之外的区域 - 我该如何做？ 我们能保存 match_histpgram 函数的转换并将其应用于整个图像吗？

编辑1： 这是我尝试过的。 https://github.com/Sum-Al/color_correction

- Sum-Al

---

是的，我认为你可以做到这一点，除了这个人写的博客文章中没有任何源代码之外。OpenCV有一个完整的模块可以实现这个功能，我相信在OpenCV或其他博客上都有实际的示例。--请展示你尝试实现这个功能的过程。 - Christoph Rackwitz

所以核心是 skimage.exposure.match_histograms ... - Christoph Rackwitz

查看一下skimage是否可以提供图像比较结果的某些描述，然后您可以将其应用于其他图像。 - 您可能需要手动构建累积分布和其他调用才能实现这一点。 - Christoph Rackwitz

同时... match_histograms 绝对不是正确的方法。它会破坏图像内容，颜色不会被正确地调整匹配。它仅仅重新塑形直方图。这只是看起来能够工作，但这是偶然的。 - Christoph Rackwitz

https://docs.opencv.org/4.x/d9/d7e/tutorial_mcc_basic_chart_detection.html 和 https://docs.opencv.org/4.x/dd/d19/group__mcc.html 以及大量的"色彩科学"。 - Christoph Rackwitz

Answer 1

如果你按照skimage教程所示的步骤，你可以得出以下方法，该方法利用任何类型的图像而不是调色板：

import matplotlib.pyplot as plt
import numpy as np
from skimage import data
from skimage import exposure
from skimage.exposure import match_histograms

reference = np.array(data.coffee(), dtype=np.uint8)
image = np.array(data.chelsea(), dtype=np.uint8)
matched = match_histograms(image, reference, channel_axis=-1)

test = match_histograms(matched, image, channel_axis=-1)

fig, (ax1, ax2, ax3) = plt.subplots(nrows=1, ncols=3, figsize=(8, 3), sharex=True, sharey=True)
for aa in (ax1, ax2, ax3):
    aa.set_axis_off()

ax1.imshow(image)
ax1.set_title('Source')
ax2.imshow(matched)
ax2.set_title('Reference')
ax3.imshow(test)
ax3.set_title('Matched')

plt.tight_layout()
plt.show()

这将产生以下结果：

您还可以查看相应的直方图：

fig, axes = plt.subplots(nrows=3, ncols=3, figsize=(8, 8))


for i, img in enumerate((image, matched, test)):
    for c, c_color in enumerate(('red', 'green', 'blue')):
        img_hist, bins = exposure.histogram(img[..., c], source_range='dtype')
        axes[c, i].plot(bins, img_hist / img_hist.max())
        img_cdf, bins = exposure.cumulative_distribution(img[..., c])
        axes[c, i].plot(bins, img_cdf)
        axes[c, 0].set_ylabel(c_color)

axes[0, 0].set_title('Source')
axes[0, 1].set_title('Reference')
axes[0, 2].set_title('Matched')

plt.tight_layout()
plt.show()

正如您所看到的，在匹配过程后，参考图像和“匹配”图像的直方图看起来很相似。

编辑：请注意，从版本0.19开始，multichannel参数已被弃用，应使用channel_axis参数。参考

编辑2：如果您想要存储这个“转换”，有两种选择：

第一种直接的方法是在应用匹配时继续传递参考图像。

另一种选择是存储由skimage的_match_cumulative_cdf函数计算的每个通道的相关分位数，该函数由match_histograms在幕后使用，并以相同的方式应用插值。

def _match_cumulative_cdf(source, template):
    """
    Return modified source array so that the cumulative density function of
    its values matches the cumulative density function of the template.
    """
    src_values, src_unique_indices, src_counts = np.unique(source.ravel(),
                                                           return_inverse=True,
                                                           return_counts=True)
    tmpl_values, tmpl_counts = np.unique(template.ravel(), return_counts=True)

    # calculate normalized quantiles for each array
    src_quantiles = np.cumsum(src_counts) / source.size
    tmpl_quantiles = np.cumsum(tmpl_counts) / template.size

    interp_a_values = np.interp(src_quantiles, tmpl_quantiles, tmpl_values)
    return interp_a_values[src_unique_indices].reshape(source.shape)

- code-lukas

---

谢谢。但是你的“_match_cumulative_cdf”函数只有两个参数，就足够处理所有情况了吗？最终，输入图像也需要被处理。_match_cumulative_cdf(source, match, reference)。 - dazzafact

如果你深入了解match_histograms的实现方式，你会发现这个函数被广泛使用。它的核心是从第73行到第76行的循环。该函数将分别对三个RGB通道进行处理，因此你需要为每个通道存储相关的分位数。或者像之前提到的那样，不断将参考图像传递给match_histograms，但这会稍微增加计算量。为了避免这种情况，可以存储tmpl_quantiles和tmpl_values。 - code-lukas

你能给我一个基于给定Github脚本的示例代码吗？我不是Python专家，无法实现你的片段。Github脚本几乎完美运行，但没有输入图像修改。 - dazzafact

Answer 2

好的，这是最终可用的脚本。还要感谢“code-lukas”提供的提示。 您只需要一个已经优化颜色的输入图像和另一个未经过颜色优化的图像。两个图像都带有彩色卡，使用ArUCo标记（您可以将它们粘在每个图像卡的角落上以便于检测） https://github.com/dazzafact/image_color_correction 输入的颜色优化图像: 输入未进行颜色优化的图像 颜色优化输出图像 使用以下参数运行脚本： python color_correction.py --reference ref.jpg --input input.jpg --output out.jpg https://github.com/dazzafact/image_color_correction

from imutils.perspective import four_point_transform
from skimage import exposure
import numpy as np
import argparse
import imutils
import cv2
import sys
from os.path import exists
import os.path as pathfile
from PIL import Image


def find_color_card(image):
    # load the ArUCo dictionary, grab the ArUCo parameters, and
    # detect the markers in the input image
    arucoDict = cv2.aruco.Dictionary_get(cv2.aruco.DICT_ARUCO_ORIGINAL)
    arucoParams = cv2.aruco.DetectorParameters_create()
    (corners, ids, rejected) = cv2.aruco.detectMarkers(image,
                                                       arucoDict, parameters=arucoParams)

    # try to extract the coordinates of the color correction card
    try:
        # otherwise, we've found the four ArUco markers, so we can
        # continue by flattening the ArUco IDs list
        ids = ids.flatten()

        # extract the top-left marker
        i = np.squeeze(np.where(ids == 923))
        topLeft = np.squeeze(corners[i])[0]

        # extract the top-right marker
        i = np.squeeze(np.where(ids == 1001))
        topRight = np.squeeze(corners[i])[1]

        # extract the bottom-right marker
        i = np.squeeze(np.where(ids == 241))
        bottomRight = np.squeeze(corners[i])[2]

        # extract the bottom-left marker
        i = np.squeeze(np.where(ids == 1007))
        bottomLeft = np.squeeze(corners[i])[3]

    # we could not find color correction card, so gracefully return
    except:
        return None

    # build our list of reference points and apply a perspective
    # transform to obtain a top-down, bird’s-eye view of the color
    # matching card
    cardCoords = np.array([topLeft, topRight,
                           bottomRight, bottomLeft])
    card = four_point_transform(image, cardCoords)
    # return the color matching card to the calling function
    return card


def _match_cumulative_cdf_mod(source, template, full):
    """
    Return modified full image array so that the cumulative density function of
    source array matches the cumulative density function of the template.
    """
    src_values, src_unique_indices, src_counts = np.unique(source.ravel(),
                                                           return_inverse=True,
                                                           return_counts=True)
    tmpl_values, tmpl_counts = np.unique(template.ravel(), return_counts=True)

    # calculate normalized quantiles for each array
    src_quantiles = np.cumsum(src_counts) / source.size
    tmpl_quantiles = np.cumsum(tmpl_counts) / template.size

    interp_a_values = np.interp(src_quantiles, tmpl_quantiles, tmpl_values)

    # Here we compute values which the channel RGB value of full image will be modified to.
    interpb = []
    for i in range(0, 256):
        interpb.append(-1)

    # first compute which values in src image transform to and mark those values.

    for i in range(0, len(interp_a_values)):
        frm = src_values[i]
        to = interp_a_values[i]
        interpb[frm] = to

    # some of the pixel values might not be there in interp_a_values, interpolate those values using their
    # previous and next neighbours
    prev_value = -1
    prev_index = -1
    for i in range(0, 256):
        if interpb[i] == -1:
            next_index = -1
            next_value = -1
            for j in range(i + 1, 256):
                if interpb[j] >= 0:
                    next_value = interpb[j]
                    next_index = j
            if prev_index < 0:
                interpb[i] = (i + 1) * next_value / (next_index + 1)
            elif next_index < 0:
                interpb[i] = prev_value + ((255 - prev_value) * (i - prev_index) / (255 - prev_index))
            else:
                interpb[i] = prev_value + (i - prev_index) * (next_value - prev_value) / (next_index - prev_index)
        else:
            prev_value = interpb[i]
            prev_index = i

    # finally transform pixel values in full image using interpb interpolation values.
    wid = full.shape[1]
    hei = full.shape[0]
    ret2 = np.zeros((hei, wid))
    for i in range(0, hei):
        for j in range(0, wid):
            ret2[i][j] = interpb[full[i][j]]
    return ret2


def match_histograms_mod(inputCard, referenceCard, fullImage):
    """
        Return modified full image, by using histogram equalizatin on input and
         reference cards and applying that transformation on fullImage.
    """
    if inputCard.ndim != referenceCard.ndim:
        raise ValueError('Image and reference must have the same number '
                         'of channels.')
    matched = np.empty(fullImage.shape, dtype=fullImage.dtype)
    for channel in range(inputCard.shape[-1]):
        matched_channel = _match_cumulative_cdf_mod(inputCard[..., channel], referenceCard[..., channel],
                                                    fullImage[..., channel])
        matched[..., channel] = matched_channel
    return matched


# construct the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-r", "--reference", required=True,
                help="path to the input reference image")
ap.add_argument("-v", "--view", required=False, default=False, action='store_true',
                help="Image Preview?")
ap.add_argument("-o", "--output", required=False, default=False,
                help="Image Output Path")
ap.add_argument("-i", "--input", required=True,
                help="path to the input image to apply color correction to")
args = vars(ap.parse_args())

# load the reference image and input images from disk
print("[INFO] loading images...")
# raw = cv2.imread(args["reference"])
# img1 = cv2.imread(args["input"])
file_exists = pathfile.isfile(args["reference"])
print(file_exists)

if not file_exists:
    print('[WARNING] Referenz File not exisits '+str(args["reference"]))
    sys.exit()


raw = cv2.imread(args["reference"])
img1 = cv2.imread(args["input"])
# resize the reference and input images

#raw = imutils.resize(raw, width=301)
#img1 = imutils.resize(img1, width=301)
raw = imutils.resize(raw, width=600)
img1 = imutils.resize(img1, width=600)
# display the reference and input images to our screen
if args['view']:
    cv2.imshow("Reference", raw)
    cv2.imshow("Input", img1)

# find the color matching card in each image
print("[INFO] finding color matching cards...")
rawCard = find_color_card(raw)
imageCard = find_color_card(img1)
# if the color matching card is not found in either the reference
# image or the input image, gracefully exit
if rawCard is None or imageCard is None:
    print("[INFO] could not find color matching card in both images")
    sys.exit(0)

# show the color matching card in the reference image and input image,
# respectively
if args['view']:
    cv2.imshow("Reference Color Card", rawCard)
    cv2.imshow("Input Color Card", imageCard)
# apply histogram matching from the color matching card in the
# reference image to the color matching card in the input image
print("[INFO] matching images...")

# imageCard2 = exposure.match_histograms(img1, ref,
# inputCard = exposure.match_histograms(inputCard, referenceCard, multichannel=True)
result2 = match_histograms_mod(imageCard, rawCard, img1)
 
# show our input color matching card after histogram matching
cv2.imshow("Input Color Card After Matching", inputCard)


if args['view']:
    cv2.imshow("result2", result2)

if args['output']:
    file_ok = exists(args['output'].lower().endswith(('.png', '.jpg', '.jpeg', '.tiff', '.bmp', '.gif')))

    if file_ok:
        cv2.imwrite(args['output'], result2)
        print("[SUCCESSUL] Your Image was written to: "+args['output']+"")
    else:
        print("[WARNING] Sorry, But this is no valid Image Name "+args['output']+"\nPlease Change Parameter!")

if args['view']:
    cv2.waitKey(0)

if not args['view']:
    if not args['output']:
        print('[EMPTY] You Need at least one Paramter "--view" or "--output".')

- dazzafact

---

你比我快，我正准备画出来。很高兴看到结果！ - code-lukas