在tensorflow.keras中使用sklearn的宏平均F1分数作为度量标准

Question

我已为tensorflow.keras定义了自定义指标来计算每个epoch后的宏F1分数，如下所示：

from tensorflow import argmax as tf_argmax
from sklearn.metric import f1_score

def macro_f1(y_true, y_pred):
    # labels are one-hot encoded. so, need to convert
    # [1,0,0] to 0 and
    # [0,1,0] to 1 and
    # [0,0,1] to 2. Then pass these arrays to sklearn f1_score.
    y_true = tf_argmax(y_true, axis=1)
    y_pred = tf_argmax(y_pred, axis=1)
    return f1_score(y_true, y_pred, average='macro')

并在模型编译期间使用它

model_4.compile(loss = 'categorical_crossentropy',
                optimizer = Adam(lr=init_lr, decay=init_lr / num_epochs),
                metrics = [Recall(name='recall') #, weighted_f1
                           macro_f1])

当我尝试像这样适应:

history_model_4 = model_4.fit(train_image_generator.flow(x=train_imgs, y=train_targets, batch_size=batch_size),
                            validation_data = (val_imgs, val_targets),
                            epochs=num_epochs,
                            class_weight=mask_weights_train,
                            callbacks=[model_save_cb, early_stop_cb, epoch_times_cb],
                            verbose=2)

这是错误信息:

OperatorNotAllowedInGraphError: in user code:

    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:806 train_function  *
        return step_function(self, iterator)
    <ipython-input-57-a890ea61878e>:6 macro_f1  *
        return f1_score(y_true, y_pred, average='macro')
    /usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py:1095 f1_score  *
        return fbeta_score(y_true, y_pred, 1, labels=labels,
    /usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py:1217 fbeta_score  *
        _, _, f, _ = precision_recall_fscore_support(y_true, y_pred,
    /usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py:1478 precision_recall_fscore_support  *
        labels = _check_set_wise_labels(y_true, y_pred, average, labels,
    /usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py:1301 _check_set_wise_labels  *
        y_type, y_true, y_pred = _check_targets(y_true, y_pred)
    /usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py:80 _check_targets  *
        check_consistent_length(y_true, y_pred)
    /usr/local/lib/python3.6/dist-packages/sklearn/utils/validation.py:209 check_consistent_length  *
        uniques = np.unique(lengths)
    <__array_function__ internals>:6 unique  **
        
    /usr/local/lib/python3.6/dist-packages/numpy/lib/arraysetops.py:263 unique
        ret = _unique1d(ar, return_index, return_inverse, return_counts)
    /usr/local/lib/python3.6/dist-packages/numpy/lib/arraysetops.py:311 _unique1d
        ar.sort()
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:877 __bool__
        self._disallow_bool_casting()
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:487 _disallow_bool_casting
        "using a `tf.Tensor` as a Python `bool`")
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:474 _disallow_when_autograph_enabled
        " indicate you are trying to use an unsupported feature.".format(task))

    OperatorNotAllowedInGraphError: using a `tf.Tensor` as a Python `bool` is not allowed: AutoGraph did convert this function. This might indicate you are trying to use an unsupported feature.

什么导致了这些错误，如何修复并在每个epoch结束时使用它作为我的评估指标之一？

编辑1：
注意：所有这些都是在Jupyter Notebook中完成的，我已添加“ >>> ”以分隔行。

# getting a batch to pass to model
>>> a_batch = train_image_generator.flow(x=train_imgs, y=train_targets, batch_size=batch_size).next()
# checking its' type to ensure that it's what i though it is
>>> type(a_batch)
# passing the batch to the model
>>> logits = model_4(a_batch)
# checking the type of output
>>> type(logits)
tensorflow.python.framework.ops.EagerTensor
# extracting only the passed targets to calculate f1-score
>>> _, dummy_targets = a_batch
# checking it's type
>>> type(dummy_targets)
numpy.ndarray
>>> macro_f1(y_true=dummy_targets, y_pred=logits)
0.0811965811965812

- Naveen Reddy Marthala

---

无法复制，可能与您的数据有关。编译模型后，请尝试使用 logits = model(sample_X_batch)；f1 = macro_f1(sample_y_batch, logits) 进行调试。您可以在 macro_f1 中的 y_true 和 y_pred 后添加打印。 - user2827262

@user2827262，请检查我的编辑。 - Naveen Reddy Marthala

该函数按预期工作并返回一个浮点分数。它已经运行，我无法弄清楚是什么导致了错误。 - Naveen Reddy Marthala

Answer 1

sklearn 不是 TensorFlow 代码，通常建议避免在 TF 内执行任意 Python 代码。

TensorFlow addons 已经有 F1 分数的实现 (tfa.metrics.F1Score)，因此请将您的代码更改为使用它而不是您自定义的指标。

确保您先 pip install tensorflow-addons，然后使用上述实现。

import tensorflow_addons as tfa

model_4.compile(loss = 'categorical_crossentropy',
                optimizer = Adam(lr=init_lr, decay=init_lr / num_epochs),
                metrics = [Recall(name='recall') #, weighted_f1
                           tfa.metrics.F1Score(average='macro')])

- bluesummers