如何使用groupby和apply为xarray数据集添加新变量？

Question

我在理解xarray.groupby的工作原理方面遇到了严重困难。我试图对xarray DatasetGroupBy集合中的每个组应用给定函数"f"，使得“f”可以向原始xr.DataSet的每个应用组添加新变量。

---

简要介绍：

我的问题通常出现在地球科学、遥感等领域。

目标是逐像素（或逐格子）地应用给定函数于数组上。

示例

假设我想针对给定区域评估风场的风速分量（u,v）与新方向的关系。因此，我希望计算旋转版本的'u'和'v'分量，即u_rotated和v_rotated。

假设这个新方向相对于风场中的每个像素位置逆时针旋转30°。因此，新的风速分量将是(u_30_degrees和v_30_degrees)。

我最初的尝试是将每个x和y坐标（或经纬度）堆叠成一个名为pixel的新维度，并稍后通过这个新维度（“pixel”）进行groupby，并应用一个函数，该函数会执行矢量风旋转操作。

以下是我最初尝试的代码片段:

# First, let's create some functions for vector rotation:

def rotate_2D_vector_per_given_degrees(array2D, angle=30):
    '''
        
    
        Parameters
        ----------
        array2D : 1D length 2 numpy array
            
        angle : float angle in degrees (optional)
            DESCRIPTION. The default is 30.
    
        Returns
        -------
        Rotated_2D_Vector : 1D of length 2 numpy array
            

    '''
        
    R = get_rotation_matrix(rotation = angle)
        
    
    Rotated_2D_Vector = np.dot(R, array2D)
    
    return Rotated_2D_Vector

def get_rotation_matrix(rotation=90):
    '''
    Description:
    
        This function creates a rotation matrix given a defined rotation angle (in degrees)
    
    Parameters:
        rotation: in degrees
    
    Returns:
        rotation matrix
    '''
    
    theta = np.radians(rotation) # degrees
    c, s = np.cos(theta), np.sin(theta)
    R = np.array(((c, -s), (s, c)))
    return R
    


# Then let's create a reproducible dataset for analysis:

u_wind = xr.DataArray(np.ones( shape=(20, 30)),
                     dims=('x', 'y'),
                     coords={'x': np.arange(0, 20),
                             'y': np.arange(0, 30)},
                     name='u')


v_wind = xr.DataArray(np.ones( shape=(20, 30))*0.3,
                     dims=('x', 'y'),
                     coords={'x': np.arange(0, 20),
                             'y': np.arange(0, 30)},
                     name='v')
 
data = xr.merge([u_wind, v_wind])


# Let's create the given function that will be applied per each group in the dataset:



def rotate_wind(array, degrees=30):
    
    # This next line, I create a 1-dimension vector of length 2, 
    # with wind speed of the u and v components, respectively.

    # The best solution I found has been conver the dataset into a single xr.DataArray
    # by stacking the 'u' and 'v' components into a single variable named 'wind'.

    vector = array.to_array(dim='wind').values
    
    # Now, I rotate the wind vector given a rotation angle in degrees

    Rotated = rotate_2D_vector_per_given_degrees(vector, degrees)
    
    # Ensuring numerical division problems as 1e-17  == 0.
    Rotated = np.where( np.abs(Rotated - 6.123234e-15) < 1e-15, 0, Rotated)
    
    # sanity check for each point position:

    print('Coords: ', array['point'].values, 
          'Wind Speed: ', vector, 
          'Response :', Rotated, 
          end='\n\n'+'-'*20+'\n')
    
    components = [a for a in data.variables if a not in data.dims]
    
    for dim, value in zip(components, Rotated):
        
        array['{0}_rotated_{1}'.format(dim, degrees)] = value
        
    return array



# Finally, lets stack our dataset per grid-point, groupby this new dimension, and apply the desired function:

stacked = data.stack(point = ['x', 'y'])

stacked = stacked.groupby('point').apply(rotate_wind)

# lets unstack the data to recover the original dataset:

data = stacked.unstack('point')

# Let's check if the function worked correctly
data.to_dataframe().head(30)

---

尽管上面的例子看起来运行正常，但我仍然不确定它的结果是否正确，甚至不确定groupby-apply函数实现是否高效（干净、非冗余、快速等）。

欢迎提供任何见解！

诚挚地，

- Philipe Riskalla Leal

Answer 1

您可以简单地将整个数组乘以旋转矩阵，类似于np.dot(R, da)。

因此，如果您有以下数据集：

>>> dims = ("x", "y")
>>> sizes = (20, 30)

>>> ds = xr.Dataset(
        data_vars=dict(u=(dims, np.ones(sizes)), v=(dims, np.ones(sizes) * 0.3)),
        coords={d: np.arange(s) for d, s in zip(dims, sizes)},
    )
>>> ds
<xarray.Dataset>
Dimensions:  (x: 20, y: 30)
Coordinates:
  * x        (x) int64 0 1 2 3 4 ... 16 17 18 19
  * y        (y) int64 0 1 2 3 4 ... 26 27 28 29
Data variables:
    u        (x, y) float64 1.0 1.0 ... 1.0 1.0
    v        (x, y) float64 0.3 0.3 ... 0.3 0.3

就像你所做的那样，转换成以下的 DataArray：

>>> da = ds.stack(point=["x", "y"]).to_array(dim="wind")
>>> da
<xarray.DataArray (wind: 2, point: 600)>
array([[1. , 1. , 1. , ..., 1. , 1. , 1. ],
       [0.3, 0.3, 0.3, ..., 0.3, 0.3, 0.3]])
Coordinates:
  * point    (point) MultiIndex
  - x        (point) int64 0 0 0 0 ... 19 19 19 19
  - y        (point) int64 0 1 2 3 ... 26 27 28 29
  * wind     (wind) <U1 'u' 'v'

然后，通过np.dot(R, da)，您可以获得旋转后的值：

>>> np.dot(R, da).shape
(2, 600)

>>> type(np.dot(R, da))
<class 'numpy.ndarray'>

但它是一个numpy的ndarray。如果你想回到xarray的DataArray，你可以使用类似这样的技巧（也可能存在其他解决方案）：

def rotate(da, dim, angle):

    # Put dim first
    dims_orig = da.dims
    da = da.transpose(dim, ...)

    # Rotate
    R = rotation_matrix(angle)
    rotated = da.copy(data=np.dot(R, da), deep=True)

    # Rename values of "dim" coord according to rotation
    rotated[dim] = [f"{orig}_rotated_{angle}" for orig in da[dim].values]

    # Transpose back to orig
    return rotated.transpose(*dims_orig)

然后像这样使用:

>>> da_rotated = rotate(da, dim="wind", angle=30)
>>> da_rotated
<xarray.DataArray (wind: 2, point: 600)>
array([[0.7160254 , 0.7160254 , 0.7160254 , ..., 0.7160254 , 0.7160254 ,
        0.7160254 ],
       [0.75980762, 0.75980762, 0.75980762, ..., 0.75980762, 0.75980762,
        0.75980762]])
Coordinates:
  * point    (point) MultiIndex
  - x        (point) int64 0 0 0 0 ... 19 19 19 19
  - y        (point) int64 0 1 2 3 ... 26 27 28 29
  * wind     (wind) <U12 'u_rotated_30' 'v_rotated_30'

最终，您可以像这样返回到原始的数据集结构：

>>> ds_rotated = da_rotated.to_dataset(dim="wind").unstack(dim="point")
>>> ds_rotated
<xarray.Dataset>
Dimensions:       (x: 20, y: 30)
Coordinates:
  * x             (x) int64 0 1 2 3 ... 17 18 19
  * y             (y) int64 0 1 2 3 ... 27 28 29
Data variables:
    u_rotated_30  (x, y) float64 0.716 ... 0.716
    v_rotated_30  (x, y) float64 0.7598 ... 0.7598

- paime

---

感谢您的回复，内容非常详细，我会关注您提出的变更。 - Philipe Riskalla Leal