我想使用Bert来训练一个21类文本分类模型。但是我的训练数据非常少,所以我下载了一个包含5类、200万个样本的类似数据集,并使用Bert提供的uncased预训练模型对下载的数据进行微调,获得了大约98%的验证准确率。现在,我想将这个模型用作我自己的小型自定义数据的预训练模型。但是,由于检查点模型有5类而我的自定义数据有21类,所以出现了“来自checkpoint reader的张量output_bias形状不匹配”的错误。
NFO:tensorflow:Calling model_fn.
INFO:tensorflow:Running train on CPU
INFO:tensorflow:*** Features ***
INFO:tensorflow: name = input_ids, shape = (32, 128)
INFO:tensorflow: name = input_mask, shape = (32, 128)
INFO:tensorflow: name = is_real_example, shape = (32,)
INFO:tensorflow: name = label_ids, shape = (32, 21)
INFO:tensorflow: name = segment_ids, shape = (32, 128)
Tensor("IteratorGetNext:3", shape=(32, 21), dtype=int32)
WARNING:tensorflow:From /home/user/Spine_NLP/bert/modeling.py:358: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
WARNING:tensorflow:From /home/user/Spine_NLP/bert/modeling.py:671: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.dense instead.
INFO:tensorflow:num_labels:21;logits:Tensor("loss/BiasAdd:0", shape=(32, 21), dtype=float32);labels:Tensor("loss/Cast:0", shape=(32, 21), dtype=float32)
INFO:tensorflow:Error recorded from training_loop: Shape of variable output_bias:0 ((21,)) doesn't match with shape of tensor output_bias ([5]) from checkpoint reader.