logo标签列表

从元组列表中删除重复项

python
3

我有一个元组列表，不幸的是其中包含重复项，例如：

[(67, u'top-coldestcitiesinamerica'), (66, u'ecofriendlyideastocelebrateindependenceday-phpapp'), (65, u'a-b-c-ca-d-ab-ea-d-c-c'), (64, u'a-b-c-ca-d-ab-ea-d-c-c'), (63, u'alexandre-meybeck-faowhatisclimate-smartagriculture-backgroundopportunitiesandchallenges'), (62, u'ghgemissions'), (61, u'top-coldestcitiesinamerica'), (58, u'infographicthe-stateofdigitaltransformationaltimetergroup'), (57, u'culture'), (55, u'cas-k-ihaveanidea'), (54, u'trendsfor'), (53, u'batteryimpedance'), (52, u'evs-howey-full'), (51, u'bericht'), (49, u'classiccarinsurance'), (47, u'uploaded_file'), (46, u'x_file'), (45, u's-s-main'), (44, u'vehicle-propulsion'), (43, u'x_file')]

问题在于元组的第一个元素（基于0的排序）是我想检查重复项的条目。所以，我可以看到：
(67, u'top-coldestcitiesinamerica')
(61, u'top-coldestcitiesinamerica')

这些元组存在重复，我想要删除其中一个（类似于set）。因此，最终我要得到一个无重复元素的干净元组列表，就像这样（即在元组的第一个元素上没有重复）：

[(67, u'top-coldestcitiesinamerica'), (66, u'ecofriendlyideastocelebrateindependenceday-phpapp'), (65, u'a-b-c-ca-d-ab-ea-d-c-c') (63, u'alexandre-meybeck-faowhatisclimate-smartagriculture-backgroundopportunitiesandchallenges'), (62, u'ghgemissions'), (58, u'infographicthe-stateofdigitaltransformationaltimetergroup'), (57, u'culture'), (55, u'cas-k-ihaveanidea'), (54, u'trendsfor'), (53, u'batteryimpedance'), (52, u'evs-howey-full'), (51, u'bericht'), (49, u'classiccarinsurance'), (47, u'uploaded_file'), (46, u'x_file'), (45, u's-s-main'), (44, u'vehicle-propulsion')]

如何用Pythonic的方式实现这一点？ 谢谢！

4个回答
5

您可以使用来自如何在保留顺序的同时从Python列表中删除重复项？的set方法，使用x [1]作为唯一标识符：

def unique_second_element(seq):
    seen = set()
    seen_add = seen.add
    return [x for x in seq if not (x[1] in seen or seen_add(x[1]))]

请注意，如果您想保留最后一次出现，则OrderedDict方法也适用；对于第一次出现，您需要颠倒输入，然后再颠倒输出。
您可以通过支持一个key函数使其更通用：
def unique_preserve_order(seq, key=None):
    if key is None:
        key = lambda elem: elem
    seen = set()
    seen_add = seen.add
    augmented = ((key(x), x) for x in seq)
    return [x for k, x in augmented if not (k in seen or seen_add(k))]

然后使用。
import operator

unique_preserve_order(yourlist, key=operator.itemgetter(1))

示例：

>>> def unique_preserve_order(seq, key=None):
...     if key is None:
...         key = lambda elem: elem
...     seen = set()
...     seen_add = seen.add
...     augmented = ((key(x), x) for x in seq)
...     return [x for k, x in augmented if not (k in seen or seen_add(k))]
... 
>>> from pprint import pprint
>>> import operator
>>> yourlist = [(67, u'top-coldestcitiesinamerica'), (66, u'ecofriendlyideastocelebrateindependenceday-phpapp'), (65, u'a-b-c-ca-d-ab-ea-d-c-c'), (64, u'a-b-c-ca-d-ab-ea-d-c-c'), (63, u'alexandre-meybeck-faowhatisclimate-smartagriculture-backgroundopportunitiesandchallenges'), (62, u'ghgemissions'), (61, u'top-coldestcitiesinamerica'), (58, u'infographicthe-stateofdigitaltransformationaltimetergroup'), (57, u'culture'), (55, u'cas-k-ihaveanidea'), (54, u'trendsfor'), (53, u'batteryimpedance'), (52, u'evs-howey-full'), (51, u'bericht'), (49, u'classiccarinsurance'), (47, u'uploaded_file'), (46, u'x_file'), (45, u's-s-main'), (44, u'vehicle-propulsion'), (43, u'x_file')]
>>> pprint(unique_preserve_order(yourlist, operator.itemgetter(1)))
[(67, u'top-coldestcitiesinamerica'),
 (66, u'ecofriendlyideastocelebrateindependenceday-phpapp'),
 (65, u'a-b-c-ca-d-ab-ea-d-c-c'),
 (63,
  u'alexandre-meybeck-faowhatisclimate-smartagriculture-backgroundopportunitiesandchallenges'),
 (62, u'ghgemissions'),
 (58, u'infographicthe-stateofdigitaltransformationaltimetergroup'),
 (57, u'culture'),
 (55, u'cas-k-ihaveanidea'),
 (54, u'trendsfor'),
 (53, u'batteryimpedance'),
 (52, u'evs-howey-full'),
 (51, u'bericht'),
 (49, u'classiccarinsurance'),
 (47, u'uploaded_file'),
 (46, u'x_file'),
 (45, u's-s-main'),
 (44, u'vehicle-propulsion')]
抱歉回复晚了 - 我最终使用了你的 unique_second_element 方法 - 真是太棒了。非常感谢！ - AJW
1
作为另一种选择，您可以使用 itertools.groupby()，如果您有一个巨大的列表，这可能会有所帮助，但不如 set 好：
>>> from itertools import groupby
>>> from operator import itemgetter
>>> [next(g) for _,g in groupby(sorted(l,key=itemgetter(1)),itemgetter(1))]
[(65, u'a-b-c-ca-d-ab-ea-d-c-c'), (63, u'alexandre-meybeck-faowhatisclimate-smartagriculture-backgroundopportunitiesandchallenges'), (53, u'batteryimpedance'), (51, u'bericht'), (55, u'cas-k-ihaveanidea'), (49, u'classiccarinsurance'), (57, u'culture'), (66, u'ecofriendlyideastocelebrateindependenceday-phpapp'), (52, u'evs-howey-full'), (62, u'ghgemissions'), (58, u'infographicthe-stateofdigitaltransformationaltimetergroup'), (45, u's-s-main'), (67, u'top-coldestcitiesinamerica'), (54, u'trendsfor'), (47, u'uploaded_file'), (44, u'vehicle-propulsion'), (46, u'x_file')]
这会破坏排序，使其成为O(NlogN)的解决方案，而不是我的O(N)方法。 - Martijn Pieters
@MartijnPieters 不幸的是，是的！但也许对OP来说这不重要！我已经提到set是更好的选择了！ - Mazdak
0
  1. 定义一个 Check list 变量来添加键。
  2. 遍历输入列表中的每个项。
  3. 检查键是否存在于 Check list 中。
  4. 如果不存在，则将该项添加到结果列表并更新 Check list。
  5. 打印结果。

代码：

input_list = [(67, u'top-coldestcitiesinamerica'), (66, u'ecofriendlyideastocelebrateindependenceday-phpapp'), (65, u'a-b-c-ca-d-ab-ea-d-c-c'), (64, u'a-b-c-ca-d-ab-ea-d-c-c'), (63, u'alexandre-meybeck-faowhatisclimate-smartagriculture-backgroundopportunitiesandchallenges'), (62, u'ghgemissions'), (61, u'top-coldestcitiesinamerica'), (58, u'infographicthe-stateofdigitaltransformationaltimetergroup'), (57, u'culture'), (55, u'cas-k-ihaveanidea'), (54, u'trendsfor'), (53, u'batteryimpedance'), (52, u'evs-howey-full'), (51, u'bericht'), (49, u'classiccarinsurance'), (47, u'uploaded_file'), (46, u'x_file'), (45, u's-s-main'), (44, u'vehicle-propulsion'), (43, u'x_file')]

check_list = set()
result = []
for i in input_list:
    if not i[1] in check_list:
        result.append(i)
        check_list.add(i[1])

import pprint
pprint.pprint(result)

输出：

$ python task4.py 
[(67, u'top-coldestcitiesinamerica'),
 (66, u'ecofriendlyideastocelebrateindependenceday-phpapp'),
 (65, u'a-b-c-ca-d-ab-ea-d-c-c'),
 (63,
  u'alexandre-meybeck-faowhatisclimate-smartagriculture-backgroundopportunitiesandchallenges'),
 (62, u'ghgemissions'),
 (58, u'infographicthe-stateofdigitaltransformationaltimetergroup'),
 (57, u'culture'),
 (55, u'cas-k-ihaveanidea'),
 (54, u'trendsfor'),
 (53, u'batteryimpedance'),
 (52, u'evs-howey-full'),
 (51, u'bericht'),
 (49, u'classiccarinsurance'),
 (47, u'uploaded_file'),
 (46, u'x_file'),
 (45, u's-s-main'),
 (44, u'vehicle-propulsion')]
@MartijnPieters：抱歉，我使用了set。 - Vivek Sable
0

我用了一种非常简单易懂的方式来完成它。

lst=[(67, u'top-coldestcitiesinamerica'), (66, u'ecofriendlyideastocelebrateindependenceday-phpapp'), (65, u'a-b-c-ca-d-ab-ea-d-c-c'), (64, u'a-b-c-ca-d-ab-ea-d-c-c'), (63, u'alexandre-meybeck-faowhatisclimate-smartagriculture-backgroundopportunitiesandchallenges'), (62, u'ghgemissions'), (61, u'top-coldestcitiesinamerica'), (58, u'infographicthe-stateofdigitaltransformationaltimetergroup'), (57, u'culture'), (55, u'cas-k-ihaveanidea'), (54, u'trendsfor'), (53, u'batteryimpedance'), (52, u'evs-howey-full'), (51, u'bericht'), (49, u'classiccarinsurance'), (47, u'uploaded_file'), (46, u'x_file'), (45, u's-s-main'), (44, u'vehicle-propulsion'), (43, u'x_file')]

lst2 = [] #empty list to fill with unique tuples
lst_banned = [] #empty list to fill with banned elements

for tup in lst:
    if tup[-1] not in lst_banned:
        lst_banned.append(tup[-1])
        lst2.append(tup)

lst=lst2
del lst2
del lst_banned
我刚才写这个的时候看到有一个类似的答案发布了。抱歉！ :) - Robin Kastner
2
同样的评论适用于您：使用列表来跟踪唯一元素是慢的，因为每个测试需要最多len(lst_banned)步。集合允许您在常数时间内测试成员资格。 - Martijn Pieters
好观点！“set”更符合Python的风格...我认为，这也是问题的重点所在！ - Robin Kastner
网页内容由stack overflow 提供, 点击上面的
可以查看英文原文，
原文链接