有两个文件名分别为"a.txt"和"b.txt",它们都有一个单词列表。现在我想检查在"a.txt"中哪些单词是多余的且不在"b.txt"中。
我需要一种高效的算法,因为我需要比较两个字典。
有两个文件名分别为"a.txt"和"b.txt",它们都有一个单词列表。现在我想检查在"a.txt"中哪些单词是多余的且不在"b.txt"中。
我需要一种高效的算法,因为我需要比较两个字典。
mkdir temp
mkdir results
cp /usr/share/dict/american-english ~/temp/american-english-dictionary
cp /usr/share/dict/british-english ~/temp/british-english-dictionary
cat ~/temp/american-english-dictionary | wc -l > ~/results/count-american-english-dictionary
cat ~/temp/british-english-dictionary | wc -l > ~/results/count-british-english-dictionary
grep -Fxf ~/temp/american-english-dictionary ~/temp/british-english-dictionary > ~/results/common-english
grep -Fxvf ~/results/common-english ~/temp/american-english-dictionary > ~/results/unique-american-english
grep -Fxvf ~/results/common-english ~/temp/british-english-dictionary > ~/results/unique-british-english
$ cat a.txt
one
two
three
four
four
$ cat b.txt
three
two
one
awk:
$ awk '
NR==FNR { # process b.txt or the first file
seen[$0] # hash words to hash seen
next # next word in b.txt
} # process a.txt or all files after the first
!($0 in seen)' b.txt a.txt # if word is not hashed to seen, output it
输出重复项:
four
four
seen
哈希表中:$ awk '
NR==FNR {
seen[$0]
next
}
!($0 in seen) { # if word is not hashed to seen
seen[$0] # hash unseen a.txt words to seen to avoid duplicates
print # and output it
}' b.txt a.txt
输出:
four
$ cat a.txt
four,four,three,three,two,one
five,six
$ cat b.txt
one,two,three
你需要多跑几圈(for
循环):
awk -F, ' # comma-separated input
NR==FNR {
for(i=1;i<=NF;i++) # loop all comma-separated fields
seen[$i]
next
}
{
for(i=1;i<=NF;i++)
if(!($i in seen)) {
seen[$i] # this time we buffer output (below):
buffer=buffer (buffer==""?"":",") $i
}
if(buffer!="") { # output unempty buffers after each record in a.txt
print buffer
buffer=""
}
}' b.txt a.txt
这次的输出:
four
five,six
diff a.txt b.txt
不够吗? - ThanksForAllTheFish