无效参数错误：输出形状的内部维度必须与更新形状的内部维度匹配。

Question

我正在尝试在Keras中实现SPL损失。我需要做的事情非常简单，我将用numpy编写来解释我需要什么：

def spl_loss(y_true, y_pred, lmda):
    # compute any arbitrary loss function
    L = categorical_cross_entropy(y_true, y_pred)
    # set to zero those values with an error greater than lambda
    L[L>lmda] = 0
    return L

我正在尝试实现按照这个教程，但是在将值设置为零的步骤中遇到了问题。

目前我有以下代码：

def spl_loss(lmda, loss_fn):
    def loss(y_true, y_pred):
         # compute an arbitrary loss function, L
        loss_value = loss_fn(y_true, y_pred) # tensor of shape (64,)
        # get the mask of L greater than lmda
        mask = tf.greater( loss_value, tf.constant( float(lmda) ) )    # tensor of shape (64,)
        # compute indexes for the mask
        indexes = tf.reshape(tf.where(mask), [-1])  # tensor of shape (n,); where n<=64
        # set to zero values on indexes
        spl_loss_value = tf.tensor_scatter_nd_update(loss_value, indexes, tf.zeros_like(loss_value, dtype=loss_value.dtype) )  # this line gives the error
        
        return spl_loss_value
    return loss

根据文档，tensor_scatter_nd_update操作应该执行赋值操作，但是它会出现以下错误：

    spl_loss_value = tf.tensor_scatter_nd_update(loss_value, indexes, tf.zeros_like(loss_value, dtype=loss_value.dtype) )
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py:201 wrapper  **
        return target(*args, **kwargs)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/array_ops.py:5512 tensor_scatter_nd_update
        tensor=tensor, indices=indices, updates=updates, name=name)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gen_array_ops.py:11236 tensor_scatter_update
        _ops.raise_from_not_ok_status(e, name)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py:6862 raise_from_not_ok_status
        six.raise_from(core._status_to_exception(e.code, message), None)
    <string>:3 raise_from
        

    InvalidArgumentError: Inner dimensions of output shape must match inner dimensions of updates shape. Output: [64] updates: [64] [Op:TensorScatterUpdate]

我正在Colab上运行它，在这里你可以尝试一下。

我尝试了几次重新调整形状，因为我知道这是期望形状与实际形状之间的问题，但我找不到方法。这里出了什么问题？

提前感谢。

- Rodrigo Laguna

Answer 1

您收到此错误的原因是在 tf.tensor_scatter_nd_update 中的 indices 至少需要两个轴，或者需要满足 tf.rank(indices) > = 2。在标量更新中，indices 的 2D（二维）形式是为了保存两个信息，一个是更新的长度（num_updates），另一个是索引向量的长度。有关此的详细概述，请查看以下答案：Tensorflow 2 - what is 'index depth' in tensor_scatter_nd_update?。请注意保留 HTML 标签。

---

这是在tensorflow中正确实现SPL损失的方法。

def spl_loss(lmda):
    def loss(y_true, y_pred):
         # compute an arbitrary loss function, L
        loss_value = keras.losses.sparse_categorical_crossentropy(y_true, y_pred)

        # get the mask of L greater than lmda
        mask = tf.greater( loss_value, tf.constant(float(lmda) ) )    

        # compute indexes for the mask
        indexes = tf.where(mask) # tensor of shape (n,); where n<=64
        updates = tf.reshape(tf.zeros_like(indexes, dtype=tf.float32), [-1])

        # scaler update check
        num_updates, index_depth = indexes.shape.as_list()
        assert updates.shape == [num_updates]
        assert index_depth == tf.rank(loss_value)


        # print()
        # print('A', tf.reshape(tf.where(mask), [-1])[:10].numpy()) 
        # print('B', tf.where(mask).numpy()[:10]) 
        # print('Ranks: ', tf.rank(loss_value).numpy(), 
        #                  tf.rank(indices).numpy(), 
        #                   tf.rank(updates).numpy())
        # print('Shape: ', loss_value.shape, indexes.shape, updates.shape)

        # set to zero values on indexes
        spl_loss_value = tf.tensor_scatter_nd_update(loss_value, indexes, updates )

        return spl_loss_value
    return loss

...
model.compile(optimizer="adam", loss=spl_loss(lmda=2.), run_eagerly=True)
...

---

参考：tf.tensor_scatter_nd_update

- Innat