在每个分组中跳过前n行

Question

假设我有一个 Pandas 数据框(Dataframe):

df = pd.DataFrame({'Company': ['company A']*5 + ['company B']*5, 
         'Date': ['01.01.2020', '01.02.2020', '01.03.2020', '01.04.2020', '01.05.2020'] + 
                 ['01.04.2020', '01.05.2020', '01.06.2020', '01.07.2020', '01.08.2020'], 
         'Revenue': np.random.rand(1, 10)[0]*10000})

     Company        Date      Revenue
0  company A  01.01.2020  5033.243098
1  company A  01.02.2020  5967.112256
2  company A  01.03.2020  6328.425874
3  company A  01.04.2020  7289.514777
4  company A  01.05.2020  9642.728016
5  company B  01.04.2020   805.708717
6  company B  01.05.2020   162.177508
7  company B  01.06.2020  7549.296095
8  company B  01.07.2020  4398.211089
9  company B  01.08.2020  1651.938946

目标是获得一个DF，其中排除了每个公司的前N个月：

     Company        Date      Revenue
2  company A  01.03.2020  5731.949686
3  company A  01.04.2020  4300.537741
4  company A  01.05.2020  4283.022397
7  company B  01.06.2020  8011.727731
8  company B  01.07.2020  1935.579432
9  company B  01.08.2020  3866.649045

例如像这样：

for company in df['Company'].unique():
    company_df = df[df['Company'] == company].sort_values(by='Date')
    ind_to_drop = company_df.iloc[:2].index
    df = df.drop(ind_to_drop)

我正在寻找一种更有效率的方法。

- Vladimir Ber

Answer 1

你可以使用：

(df.sort_values(['Date']) # sort values by 'Date'
    .groupby('Company', as_index=False) # group by 'Company'
    .apply(lambda x: x.iloc[2:]) # skip first two rows
    .droplevel(0)) # drop first index level

输出：

     Company        Date      Revenue
2  company A  01.03.2020   559.525103
3  company A  01.04.2020  4692.250518
4  company A  01.05.2020  8206.546659
7  company B  01.06.2020  3519.014808
8  company B  01.07.2020  4902.521804
9  company B  01.08.2020  6533.685687

- Mykola Zotko

Answer 2

我会使用groupby来消除按公司重复的过滤器。此外，我认为一次性删除所有索引略微提高了性能 - 再次，仅需对数据库进行单次扫描即可。

ind_to_drop = list()
for _, data in df.groupby(by=['Company']):
    data = data.sort_values(by='Date')
    ind_to_drop += list(data.iloc[:2].index)

df = df.drop(ind_to_drop)

- Kate Melnykova

Answer 3

你可以使用 groupby().head() 提取索引，然后使用 drop：

df.drop(df.sort_values(['Date']).groupby('Company').head(1).index)

输出：

     Company        Date      Revenue
1  company A  01.02.2020  8354.050677
2  company A  01.03.2020  9867.805507
3  company A  01.04.2020  4072.178342
4  company A  01.05.2020  9626.621319
6  company B  01.05.2020  8712.769956
7  company B  01.06.2020  6751.648895
8  company B  01.07.2020   492.769737
9  company B  01.08.2020  1709.737424

- Quang Hoang