使用awk根据列查找重复行，并打印出两行及其行号。

Question

I have a following file:

userID PWD_HASH
test 1234
admin 1234
user 6789
abcd 5555
efgh 6666
root 1234

使用AWK，我需要找到原始行和它们的重复行以及它们的行号，从而得到如下输出：

NR $0
1 test 1234
2 admin 1234
6 root 1234

我尝试了以下内容，但NR并没有打印出正确的行数:

awk 'n=x[$2]{print NR" "n;print NR" "$0;} {x[$2]=$0;}' file.txt

欢迎求助！如果您需要帮助，请告诉我们。

- skazichris

Answer 1

使用GAWK，您可以通过以下结构实现：-

awk '
{
    NR>1
    {
       a[$2][NR-1 " " $0];
    }
}
END {
    for (i in a)
       if(length(a[i]) > 1)
          for (j in a[i])
             print j;
}
' Input_File.txt   

创建一个二维数组。

在第一维中，存储 PWD_HASH，在第二维中，存储行号(NR-1)与整行($0)连接后的结果。

要仅显示重复的行，可以使用 length(a[i] > 1) 条件。

- ganit44

Answer 2

$ awk '
($2 in a) {          # look for duplicates in $2
    if(a[$2]) {      # if found
        print a[$2]  # output the first, stored one
        a[$2]=""     # mark it outputed
    }
    print NR,$0      # print the duplicated one
    next             # skip the storing part that follows
}
{
    a[$2]=NR OFS $0  # store the first of each with NR and full record
}' file

输出（带有file头文件）：

2 test 1234
3 admin 1234
7 root 1234

- James Brown

Answer 3

请尝试以下操作。

awk '
FNR==NR{
  a[$2]++
  b[$2,FNR]=FNR==1?FNR:(FNR-1) OFS $0
  next
}
a[$2]>1{
  print b[$2,FNR]
}
'  Input_file  Input_file

输出将如下所示。

1 test 1234
2 admin 1234
6 root 1234

解释：以下是上述代码的解释。

awk '                                        ##Starting awk program here.
FNR==NR{                                     ##Checking condition here FNR==NR which will be TRUE when first time Input_file is being read.
  a[$2]++                                    ##Creating an array named a whose index is $1 and incrementing its value to 1 each time it sees same index.
  b[$2,FNR]=FNR==1?FNR:(FNR-1) OFS $0        ##Creating array b whose index is $2,FNR and concatenating its value to its own.
  next                                       ##Using next for skipping all further statements from here.
}
a[$2]>1{                                     ##Checking condition where value of a[$2] is greater than 1, this will be executed when 2nd time Input_file read.
  print b[$2,FNR]                            ##Printing value of array b whose index is $2,FNR here.
}
'  Input_file  Input_file                    ##Mentioning Input_file(s) names here 2 times.

- RavinderSingh13

---

我已经完成了。我必须获得更多的声望（超过15）才能添加更多的投票。再次感谢！ - skazichris

Answer 4

不使用awk，而是使用GNU coreutils工具：

tail -n+2 file | nl | sort -k3n | uniq -D -f2

tail 去除第一行。
nl 添加行号。
sort 基于第三个字段排序。
uniq 仅基于第三个字段打印重复项。

- oliv