与这个问题非常相似,但我需要考虑日期和时间; 我找不到indexer_between_time
支持的任何日期时间格式。
我有一个dask数据框,它看起来像这样:
logger_volt lat lon
time
2017-01-01 00:01:20 12.0112 37.150902 -98.362
2017-01-01 00:01:40 12.0113 37.150902 -98.362
2017-01-01 00:02:00 12.0057 37.150902 -98.362
2017-01-01 00:02:20 12.0113 37.150902 -98.362
2017-01-01 00:02:40 12.0058 37.150902 -98.362
2017-01-01 00:03:00 12.0113 37.150902 -98.362
以下是需要在特定时间范围内掩盖的列列表(这些范围内的数据被视为“不良数据”,应该返回 None
)以Python元组列表的形式:
[ # var start of mask end of mask
('lat', '2017-01-01 00:01:40', '2017-01-01 00:02:00'),
('lon', '2017-01-01 00:02:40', '2017-01-01 00:03:00'),
]
期望结果:
logger_volt lat lon
time
2017-01-01 00:01:20 12.0112 37.150902 -98.362
2017-01-01 00:01:40 12.0113 None -98.362
2017-01-01 00:02:00 12.0057 None -98.362
2017-01-01 00:02:20 12.0113 37.150902 -98.362
2017-01-01 00:02:40 12.0058 37.150902 None
2017-01-01 00:03:00 12.0113 37.150902 None
非工作代码:
dqrs = [ # var start of mask end of mask
('lat', '2017-01-01 00:01:40', '2017-01-01 00:02:00'),
('lon', '2017-01-01 00:02:40', '2017-01-01 00:03:00'),
]
df = xarray.open_dataset('filename.cdf').to_dask_dataframe()
dqr_mask = (df == df) | df.isnull() # create a dummy mask that's all True
for var, start, end in dqrs:
dqr_mask |= ((df.columns == var) & (df.index >= start) & (df.index >= end))
df = df.mask(dqr_mask).compute()
其他方法存在的问题:
- Dask数据框目前尚未实现切片赋值,因此类似
df[start:end] = None
的操作不起作用。