使用Python difflib比较超过两个文件

4

我想了解例如多台(3+)电脑的ldd依赖关系列表,并将它们相互比较并突出显示区别。例如,如果我有以下字典:

my_ldd_outputs = {
  01:"<ldd_output>",
  02:"<ldd_output>", 
  ...
  09:"<ldd_output>",
  10:"<ldd_output>"
}

我希望输出的结果看起来像这样。
<identical line 1>
<identical line 2>
<identical line 3>
<differing line 4> (computer 01 02)
<differing line 4> (computer 04 05 06 07)
<differing line 4> (computer 08 09 10)
<identical line 5>
<identical line 6>
...

我的第一次尝试涉及Python difflib,我的想法是首先获得一个数据结构,在该数据结构中,所有来自上述my_ldd_outputs字典的ldd_output列表(只是用\n分割的结果)具有相同的长度,并且任何存在于另一个ldd_output字符串中的缺失行都将添加上一个字符串。因此,如果两个文件看起来像这样:

ldd_1 = """
<identical line 1>
<identical line 2>
<differing line 3>
<identical line 4>
<extra line 5>
<identical line 6>
"""

ldd_2 = """
<identical line 1>
<identical line 2>
<differing line 3>
<identical line 4>
<identical line 6>
"""

我的目标是将这些文件存储为

ldd_1 = """
<identical line 1>
<identical line 2>
<differing line 3>
<identical line 4>
<extra line 5>
<identical line 6>
"""

ldd_2 = """
<identical line 1>
<identical line 2>
<differing line 3>
<identical line 4>
<None>
<identical line 6>
"""

最终只需要迭代转换后文件的每一行(现在它们都有相同的长度),并比较它们之间的差异,忽略任何<None>条目,以便可以连续打印差异。

我创建了一个使用Python difflib的函数,用<None>字符串填充其他文件中缺失的行。然而,我不确定如何扩展该函数以包含任意数量的diffs。

def generate_diff(file_1, file_2):
    #differing hashvalues from ldd can be ignored, we only care about version and path
    def remove_hashvalues(input):
        return re.sub("([a-zA-Z0-9_.-]{32}\/|\([a-zA-Z0-9_.-]*\))", "<>", input)
    diff = [line.strip() for line in difflib.ndiff(remove_hashvalues(base).splitlines(keepends=True),remove_hashvalues(file_2).splitlines(keepends=True))]
    list_1 = []
    list_2 = []
    i = 0
    while i<len(diff):
        if diff[i].strip():
            if diff[i][0:2]=="- ":
                lost = []
                gained = []
                while diff[i][0:2]=="- " or diff[i][0:2]=="? ":
                    if diff[i][0:2]=="- ": lost.append(diff[i][1:].strip())
                    i+=1
                while diff[i][0:2]=="+ " or diff[i][0:2]=="? ":
                    if diff[i][0:2]=="+ ": gained.append(diff[i][1:].strip())
                    i+=1
                while len(lost) != len(gained):
                    lost.append("<None>") if len(lost)<len(gained) else gained.insert(0,"<None>")
                list_1+=lost; list_2+=gained
            elif diff[i][0:2]=="+ ":
                list_1.append("<None>"); list_2.append(diff[i][1:].strip())
            if not diff[i][0:2]=="? ":
                list_1.append(diff[i].strip()); list_2.append(diff[i].strip())
        i+=1
    return list_1, list_2

我也发现了这个工具,可以比较多个文件,但不幸的是它并不适用于比较代码。

编辑:我调整了@AyoubKaanich 的解决方案建议,创建了一个更简化的版本,能够实现我想要的功能:

from collections import defaultdict
import re
def transform(input):
    input = re.sub("([a-zA-Z0-9_.-]{32}\/|\([a-zA-Z0-9_.-]*\))", "<>", input) # differing hashvalues can be ignored, we only care about version and path
    return sorted(input.splitlines())
def generate_diff(outputs: dict):
    mapping = defaultdict(set)
    for target, output in outputs.items():
        for line in transform(output):
            mapping[line.strip()].add(target)
    result = []
    current_line = None
    color_index = 0
    for line in sorted(mapping.keys()):
        if len(outputs) == len(mapping[line]):
            if current_line: current_line = None
            result.append((line))
        else:
            if current_line != line.split(" ")[0]:
                current_line = line.split(" ")[0]
                color_index+=1
            result.append((f"\033[3{color_index%6+1}m{line}\033[0m",mapping[line]))
    return result

唯一的缺点是这并不适用于字符串在任意部分变化而不仅仅是开头的情况,difflib 专注于检测开头变化。但是对于ldd的情况,由于依赖项始终在首位列出,因此按字母顺序排序并取字符串的第一节即可奏效。
2个回答

1

纯Python解决方案,无需使用库或额外依赖。

注意:此解决方案基于一些假设:

  • 行的顺序不重要
  • 一行要么存在,要么缺失(没有检查行之间的相似性的逻辑)

from collections import defaultdict
import re

def transform(input):
    # differing hashvalues from ldd can be ignored, we only care about version and path
    input = re.sub("([a-zA-Z0-9_.-]{32}\/|\([a-zA-Z0-9_.-]*\))", "<>", input)
    return sorted(input.splitlines())

def generate_diff(outputs: dict, common_threshold = 0):
    """
        common_threshold: how many outputs need to contain line to consider it common
            and mark outputs that do not have it as missing
    """
    assert(common_threshold <= len(outputs))

    mapping = defaultdict(set)
    for target, output in outputs.items():
        for line in transform(output):
            mapping[line].add(target)
    
    for line in sorted(mapping.keys()):
        found = mapping[line]
        if len(outputs) == len(found):
            print('  ' + line)
        elif len(found) >= common_threshold:
            missed_str = ",".join(map(str, set(outputs.keys()) - found))
            print(f'- {line}  ({missed_str})')
        else:
            added_str = ",".join(map(str, found))
            print(f'+ {line}  ({added_str})')

示例执行


my_ldd_outputs = {
'A': """
linux-vdso.so.1 (0x00007ffde4f09000)
libtinfo.so.6 => /lib/x86_64-linux-gnu/libtinfo.so.6 (0x00007fe0594f3000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe0592cb000)
/lib64/ld-linux-x86-64.so.2 (0x00007fe059690000)
""",
'B': """
linux-vdso.so.1 (0x00007fff697b6000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1c54045000)
/lib64/ld-linux-x86-64.so.2 (0x00007f1c54299000)
""",
'C': """
linux-vdso.so.1 (0x00007fffd61f9000)
libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3 (0x00007f08a51a3000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f08a4f7b000)
/lib64/ld-linux-x86-64.so.2 (0x00007f08a5612000)
""",
'D': """
linux-vdso.so.1 (0x00007ffcf9ddd000)
libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3 (0x00007fa2e381b000)
libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007fa2e37ef000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa2e35c7000)
libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0 (0x00007fa2e3530000)
/lib64/ld-linux-x86-64.so.2 (0x00007fa2e3cd7000)
""",
'E': """
linux-vdso.so.1 (0x00007ffc2deab000)
libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3 (0x00007f31fed91000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f31fed75000)
libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007f31fed49000)
libgssapi_krb5.so.2 => /lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007f31fecf5000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f31feacd000)
libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0 (0x00007f31fea34000)
/lib64/ld-linux-x86-64.so.2 (0x00007f31ff2af000)
libkrb5.so.3 => /lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007f31fe969000)
libk5crypto.so.3 => /lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007f31fe93a000)
libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f31fe934000)
libkrb5support.so.0 => /lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007f31fe926000)
libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007f31fe91f000)
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f31fe909000)
"""
}
generate_diff(my_ldd_outputs, 2)

输出

  /lib64/ld-linux-x86-64.so.2 <>
  libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 <>
+ libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 <>  (E)
- libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3 <>  (B,A)
+ libgssapi_krb5.so.2 => /lib/x86_64-linux-gnu/libgssapi_krb5.so.2 <>  (E)
+ libk5crypto.so.3 => /lib/x86_64-linux-gnu/libk5crypto.so.3 <>  (E)
+ libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 <>  (E)
+ libkrb5.so.3 => /lib/x86_64-linux-gnu/libkrb5.so.3 <>  (E)
+ libkrb5support.so.0 => /lib/x86_64-linux-gnu/libkrb5support.so.0 <>  (E)
- libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0 <>  (C,B,A)
+ libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 <>  (E)
- libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 <>  (C,B,A)
+ libtinfo.so.6 => /lib/x86_64-linux-gnu/libtinfo.so.6 <>  (A)
+ libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 <>  (E)
  linux-vdso.so.1 <>

我在我的帖子中采用了你的解决方案,如果没有更好的建议,我会将其标记为已解决。 - Yes

1

网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接