logo标签列表

使用嵌套 'if' 语句加速 Python 的 'for' 循环

pythonperformancelistfor-loopif-statement
3

我正在尝试加快这个循环的速度，用于将数据分为两个类别。通常情况下，我并不太关心速度，但是我发现在多次迭代后，这段代码的速度实际上会急剧减慢。以下是我编写代码的方式：

plane1Data = []
plane2Data = []
plane1Times = []
plane2Times = []
plane1Dets = []
plane2Dets = []
t1 = time.time()
for i in range(0,len(adcBoardVals)):#10000):
    tic = time.time()
    if adcBoardVals[i] == 5:
        if adcChannel[i] == 0:
            #detectorVal = detectorVal + [0]
            plane1Data = plane1Data + [rawDataMat[i,:]]
            plane1Times = plane1Times + [timeVals[i]]
            plane1Dets = plane1Dets + [0]
        elif adcChannel[i] == 1:
            #detectorVal = detectorVal + [1]
            plane1Data = plane1Data + [rawDataMat[i,:]]
            plane1Times = plane1Times + [timeVals[i]]
            plane1Dets = plane1Dets + [1]
        elif adcChannel[i] == 2:
            #detectorVal = detectorVal + [2]
            plane1Data = plane1Data + [rawDataMat[i,:]]
            plane1Times = plane1Times + [timeVals[i]]
            plane1Dets = plane1Dets + [2]
        elif adcChannel[i] == 3:
            #detectorVal = detectorVal + [3]
            plane1Data = plane1Data + [rawDataMat[i,:]]
            plane1Times = plane1Times + [timeVals[i]]
            plane1Dets = plane1Dets + [3]
        elif adcChannel[i] == 4:
            #detectorVal = detectorVal + [4]
            plane1Data = plane1Data + [rawDataMat[i,:]]
            plane1Times = plane1Times + [timeVals[i]]
            #plane1Dets = plane1Dets + [4]
        elif adcChannel[i] == 5:
            #detectorVal = detectorVal + [5]
            plane1Data = plane1Data + [rawDataMat[i,:]]
            plane1Times = plane1Times + [timeVals[i]]
            plane1Dets = plane1Dets + [5]
        elif adcChannel[i] == 6:
            #detectorVal = detectorVal + [6]
            plane1Data = plane1Data + [rawDataMat[i,:]]
            plane1Times = plane1Times + [timeVals[i]]
            plane1Dets = plane1Dets + [6]
        elif adcChannel[i] == 7:
            #detectorVal = detectorVal + [7]
            plane1Data = plane1Data + [rawDataMat[i,:]]
            plane1Times = plane1Times + [timeVals[i]]
            plane1Dets = plane1Dets + [7]
    elif adcBoardVals[i] == 7:
        if adcChannel[i] == 0:
            #detectorVal = detectorVal + [16]
            plane2Data = plane2Data + [rawDataMat[i,:]]
            plane2Times = plane2Times + [timeVals[i]]
            plane2Dets = plane2Dets + [16]
        elif adcChannel[i] == 1:
            #detectorVal = detectorVal + [17]
            plane2Data = plane2Data + [rawDataMat[i,:]]
            plane2Times = plane2Times + [timeVals[i]]
            plane2Dets = plane2Dets + [17]
        elif adcChannel[i] == 2:
            #detectorVal = detectorVal + [18]
            plane2Data = plane2Data + [rawDataMat[i,:]]
            plane2Times = plane2Times + [timeVals[i]]
            plane2Dets = plane2Dets + [18]
        elif adcChannel[i] == 3:
            #detectorVal = detectorVal + [19]
            plane2Data = plane2Data + [rawDataMat[i,:]]
            plane2Times = plane2Times + [timeVals[i]]
            plane2Dets = plane2Dets + [19]
        elif adcChannel[i] == 4:
            #detectorVal = detectorVal + [20]
            plane2Data = plane2Data + [rawDataMat[i,:]]
            plane2Times = plane2Times + [timeVals[i]]
            plane2Dets = plane2Dets + [20]
        elif adcChannel[i] == 5:
            #detectorVal = detectorVal + [21]
            plane2Data = plane2Data + [rawDataMat[i,:]]
            plane2Times = plane2Times + [timeVals[i]]
            plane2Dets = plane2Dets + [21]
        elif adcChannel[i] == 6:
            #detectorVal = detectorVal + [22]
            plane2Data = plane2Data + [rawDataMat[i,:]]
            plane2Times = plane2Times + [timeVals[i]]
            plane2Dets = plane2Dets + [22]
        elif adcChannel[i] == 7:
            #detectorVal = detectorVal + [23]
            plane2Data = plane2Data + [rawDataMat[i,:]]
            plane2Times = plane2Times + [timeVals[i]]
            plane2Dets = plane2Dets + [23]
    elif adcBoardVals[i] == 6:
        if adcChannel[i] == 0:
            #detectorVal = detectorVal + [8]
            plane1Data = plane1Data + [rawDataMat[i,:]]
            plane1Times = plane1Times + [timeVals[i]]
            plane1Dets = plane1Dets + [8]
        elif adcChannel[i] == 1:
            #detectorVal = detectorVal + [9]
            plane1Data = plane1Data + [rawDataMat[i,:]]
            plane1Times = plane1Times + [timeVals[i]]
            plane1Dets = plane1Dets + [9]
        elif adcChannel[i] == 2:
            #detectorVal = detectorVal + [10]
            plane1Data = plane1Data + [rawDataMat[i,:]]
            plane1Times = plane1Times + [timeVals[i]]
            plane1Dets = plane1Dets + [10]
        elif adcChannel[i] == 3:
            #detectorVal = detectorVal + [11]
            plane1Data = plane1Data + [rawDataMat[i,:]]
            plane1Times = plane1Times + [timeVals[i]]
            plane1Dets = plane1Dets + [11]
        elif adcChannel[i] == 4:
            #detectorVal = detectorVal + [12]
            plane2Data = plane2Data + [rawDataMat[i,:]]
            plane2Times = plane2Times + [timeVals[i]]
            plane2Dets = plane2Dets + [12]
        elif adcChannel[i] == 5:
            #detectorVal = detectorVal + [13]
            plane2Data = plane2Data + [rawDataMat[i,:]]
            plane2Times = plane2Times + [timeVals[i]]
            plane2Dets = plane2Dets + [13]
        elif adcChannel[i] == 6:
            #detectorVal = detectorVal + [14]
            plane2Data = plane2Data + [rawDataMat[i,:]]
            plane2Times = plane2Times + [timeVals[i]]
            plane2Dets = plane2Dets + [14]
        elif adcChannel[i] == 7:
            #detectorVal = detectorVal + [15]
            plane2Data = plane2Data + [rawDataMat[i,:]]
            plane2Times = plane2Times + [timeVals[i]]
            plane2Dets = plane2Dets + [15]
    if i%100000 == 0:
        print('k = ',i)   
        toc = time.time()
        print('tictoc = ',toc-tic)
        print('elapsed = ',toc-t1)
    elif i>900000:
        if i%1000 == 0:
            print('k = ',i)
            toc = time.time()
            print('tictoc = ',toc-tic)
            print('elapsed = ',toc-t1)

#detectorVal = np.array(detectorVal,dtype='float')
plane1Data = np.array(plane1Data,dtype='float')
plane2Data = np.array(plane2Data,dtype='float')
plane1Times = np.array(plane1Times,dtype='float')
plane2Times = np.array(plane2Times,dtype='float')
plane1Dets = np.array(plane1Dets,dtype='int')
plane2Dets = np.array(plane2Dets,dtype='int')

我模糊地记得在我之前学习的一门C++课程中，你可以创建比嵌套的“if”语句更快的列表。如果是这样，那么我能在Python中做到吗？我现在正在运行Python 3.5。谢谢你的帮助。

1
这个问题可能适合在CodeReview SE上提问。 - Zach Gates
1
  1. 使用 append 而不是 + 来向列表中添加元素，因为后者速度较慢。
  2. 如果你打算创建 NumPy 数组，请考虑先预分配它们（例如使用 np.empty），然后在循环中设置值。
 - jdehesa
  1. 当我使用append函数时，第一个值总是因某种原因设置为“None”。例如，我正在使用
plane2Data = np.append(plane2Data,rawDataMat[i,:])
  1. 我可以使用np.empty生成一个未知长度的数组吗？谢谢。
 - Adam
@jdehesa在这两点上都是正确的。如果你将使用numpy，你可以使用它的高性能数据结构而不是普通的列表——这可能会稍微加快速度，但这与你真正的性能问题无关，即列表的复制。 - alexis
你可以独立地通过在最后一个赋值中使用 adcChannel[i] 的值来大大简化你的代码-- 你将能够省略整个嵌入式 if-else 块。 - alexis
1个回答
7
您的问题，也是一个主要的时间浪费者，就是以下形式的语句：
list_variable = list_variable + [ new_value ]

在每次循环迭代中，您调用其中三个，例如：

plane1Data = plane1Data + [rawDataMat[i,:]]

因为您可能对list_variable指向的列表有其他引用，所以Python在每次调用时构建完整的列表副本，然后在执行赋值操作时丢弃原始副本。 使用以下形式扩展所有列表，您将看到巨大的改进：

list_variable += [ new_value ]

这是真的发生了的证明：

>>> from timeit import timeit
>>> x=list(range(100000))
>>> timeit("x += [99]", "from __main__ import x", number=1000)
0.00023529794998466969
>>> x=list(range(100000))
>>> timeit("x = x + [99]", "from __main__ import x", number=1000)
0.7576854809885845
>>> 0.7576854809885845 / 0.00023529794998466969
3220.110846855868

就是这样。对于这个由100000个元素组成的列表，原地追加比复制和赋值快三千多倍。如果想要测量您的收益，可以对自己数据的子集进行分析。

我不知道——测量它。 - alexis
我的猜测是，这两种方法都足够快。但我可能错了。 - alexis
嗨Alexis，我进行了检查，使用+=比使用np.append()快大约100倍。 - Adam
嗨，Alexis。你有没有什么想法可以帮我解决嵌套的for循环问题？ - Adam
你的另一个问题不是关于嵌套循环，而是关于如何更好地处理长列表的搜索。我建议将plane2数据进行排序和/或分组，每组包含10k秒的数据，这样你只需要在2个分组中搜索附近的飞机。也许我稍后会发布一个答案... - alexis
显示剩余2条评论
网页内容由stack overflow 提供, 点击上面的
可以查看英文原文，
原文链接