你可以用以下方法加速它:
matching = df['col2'].head(n).str.contains('substr', case=True, regex=False)
rows = df['col1'].head(n)[matching==True]
然而,这个解决方案将在前
n
行中检索匹配的结果,而不是前
n
个匹配的结果。
如果您确实想要前
n
个匹配的结果,您应该使用:
rows = df['col1'][df['col2'].str.contains("substr")==True].head(n)
但是这个选项显然会更慢。
受 @ScottBoston 的答案启发,您可以使用以下方法来实现 完整的更快解决方案:
rows = df['col1'][pd.Series(['substr' in i for i in df['col2']])==True].head(n)
这种方法比使用整个结果稍微快一些,但并不是非常快。使用这种解决方案,您可以获得前 n
个匹配结果。
通过下面的测试代码,我们可以看到每个解决方案的速度和结果:
import pandas as pd
import time
n = 10
a = ["Result", "from", "first", "column", "for", "this", "matching", "test", "end"]
b = ["This", "is", "a", "test", "has substr", "also has substr", "end", "of", "test"]
col1 = a*1000000
col2 = b*1000000
df = pd.DataFrame({"col1":col1,"col2":col2})
start_time = time.time()
matching = df['col2'].str.contains('substr', case=True, regex=False)
rows = df[matching].col1.drop_duplicates()
print("--- %s seconds ---" % (time.time() - start_time))
start_time = time.time()
matching_fast = df['col2'].head(n).str.contains('substr', case=True, regex=False)
rows_fast = df['col1'].head(n)[matching==True]
print("--- %s seconds for fast solution ---" % (time.time() - start_time))
start_time = time.time()
rows_other = df['col1'][df['col2'].str.contains("substr")==True].head(n)
print("--- %s seconds for other solution ---" % (time.time() - start_time))
start_time = time.time()
rows_complete = df['col1'][pd.Series(['substr' in i for i in df['col2']])==True].head(n)
print("--- %s seconds for complete solution ---" % (time.time() - start_time))
这将输出:
>>>
而产生的系列将是:
>>> rows
4 for
5 this
Name: col1, dtype: object
>>> rows_fast
4 for
5 this
Name: col1, dtype: object
>>> rows_other
4 for
5 this
13 for
14 this
22 for
23 this
31 for
32 this
40 for
41 this
Name: col1, dtype: object
>>> rows_complete
4 for
5 this
13 for
14 this
22 for
23 this
31 for
32 this
40 for
41 this
Name: col1, dtype: object
.head()
要慢,也就是不限制搜索。 - Julio