AdamW的实现已被弃用，并将在未来版本中删除。请使用PyTorch实现的torch.optim.AdamW。

Question

如何修复已弃用的AdamW模型？

我尝试使用BERT模型对酒店评论进行情感分析，但当我运行这段代码时，它会提示以下警告。我仍在学习transformers，并且不希望代码很快就被弃用。我在网上搜索了一下，但还没有找到解决方案。

我找到了这条信息，但不知道如何将其应用到我的代码中。

要切换优化器，请在TrainingArguments中放置optim="adamw_torch" （默认为"adamw_hf"）

有人可以帮忙吗？

from transformers import BertTokenizer, BertForSequenceClassification
import torch_optimizer as optim
from torch.utils.data import DataLoader
from transformers import AdamW
import pandas as pd
import torch
import random
import numpy as np
import torch.nn as nn
from torch.nn import CrossEntropyLoss
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score, confusion_matrix, roc_auc_score, classification_report
from sklearn.model_selection import train_test_split
from sklearn import preprocessing
from tqdm.notebook import tqdm
import json
from collections import OrderedDict
import logging
from torch.utils.tensorboard import SummaryWriter

跳过一些代码...

param_optimizer = list(model.named_parameters())
no_decay = ['bias', 'gamma', 'beta']
optimizer_grouped_parameters = [{
    'params':
    [p for n, p in param_optimizer if not any(nd in n for nd in no_decay)],
    'weight_decay_rate':
    0.01
}, {
    'params':
    [p for n, p in param_optimizer if any(nd in n for nd in no_decay)],
    'weight_decay_rate':
    0.0
}]

#
optimizer = AdamW(optimizer_grouped_parameters, lr=1e-5)           ##deprecated 
#optimizer = optim.AdamW(optimizer_grouped_parameters, lr=1e-5)      ##torch.optim.AdamW  (not working)

step = 0
best_acc = 0
epoch = 10
writer = SummaryWriter(log_dir='model_best')
for epoch in tqdm(range(epoch)):
    for idx, batch in tqdm(enumerate(train_loader),
                           total=len(train_texts) // batch_size,
                           leave=False):
        optimizer.zero_grad()
        input_ids = batch['input_ids'].to(device)
        attention_mask = batch['attention_mask'].to(device)
        labels = batch['labels'].to(device)
        outputs = model(input_ids=input_ids, attention_mask=attention_mask, labels=labels)
        loss = outputs[0]  # Calculate Loss
        logging.info(
            f'Epoch-{epoch}, Step-{step}, Loss: {loss.cpu().detach().numpy()}')
        step += 1
        loss.backward()
        optimizer.step()
        writer.add_scalar('train_loss', loss.item(), step)
    logging.info(f'Epoch {epoch}, present best acc: {best_acc}, start evaluating.')
    accuracy, precision, recall, f1 = eval_model(model, eval_loader)  # Evaluate Model
    writer.add_scalar('dev_accuracy', accuracy, step)
    writer.add_scalar('dev_precision', precision, step)
    writer.add_scalar('dev_recall', recall, step)
    writer.add_scalar('dev_f1', f1, step)
    if accuracy > best_acc:
        model.save_pretrained('model_best')  # Save Model
        tokenizer.save_pretrained('model_best')
        best_acc = accuracy

- Panco

Answer 1

如果您注释掉这两行代码：

import torch_optimizer as optim
from transformers import AdamW

然后使用：

optimizer = torch.optim.AdamW(optimizer_grouped_parameters, lr=1e-5)

它工作吗？如果不工作，出现了什么错误？

要切换优化器，请在您的TrainingArguments中设置optim="adamw_torch"（默认为"adamw_hf"）

这是指Huggingface的Trainer，它配置了一个TrainingArguments实例。但是由于您正在使用自己的训练循环，这对您不适用。

- Darren Cook

---

达伦，非常感谢，这个工作完美无缺。 - Panco