我正在运行一个导入作业,之前一直运行得很好,但几天前实体数量急剧增加后,出现了锁等待超时的问题。应用程序然后重试,但由于我再次调用了 em.getTransaction().begin() ,所以会抛出异常。
为解决这个问题,我将 innodb_lock_wait_timeout 更改为120,并将批处理大小降低到50个实体。
我无法确定如何在代码中正确处理所有这些。我不想因为锁定而使整个导入失败。你会如何处理?有没有示例代码?或者有其他想法?请尽情发挥! 我的BatchPersister:
为解决这个问题,我将 innodb_lock_wait_timeout 更改为120,并将批处理大小降低到50个实体。
我无法确定如何在代码中正确处理所有这些。我不想因为锁定而使整个导入失败。你会如何处理?有没有示例代码?或者有其他想法?请尽情发挥! 我的BatchPersister:
public class BatchPersister implements Persister {
private final static Log log = getLog(BatchPersister.class);
private WorkLogger workLog = WorkLogger.instance();
private static final int BATCH_SIZE = 500;
private int persistedObjects;
private long startTime;
private UpdateBatch batch;
private String dataSource;
public BatchPersister(String dataSource) {
this.dataSource = dataSource;
}
public void persist(Persistable obj) {
persistedObjects++;
logProgress(100);
if (batch == null)
batch = new UpdateBatch(BATCH_SIZE, dataSource);
batch.add(obj);
if (batch.isFull()) {
batch.persist();
batch = null;
}
}
}
UpdateBatch
public class UpdateBatch {
private final static Log log = LogFactory.getLog(UpdateBatch.class);
private WorkLogger workLogger = WorkLogger.instance();
private final Map<Object, Persistable> batch;
private final EntityManager em;
private int size;
/**
* Initializes the batch and specifies its size.
*/
public UpdateBatch(int size, String dataSource) {
this.size = size;
batch = new LinkedHashMap<Object, Persistable>();
em = EmFactory.getEm(dataSource);
}
public void persist() {
log.info("Persisting " + this);
em.getTransaction().begin();
persistAllToDB();
em.getTransaction().commit();
WorkLog batchLog = new WorkLog(IMPORT_PERSIST, IN_PROGRESS);
batchLog.setAffectedItems(batch.size());
workLogger.log(batchLog);
em.close();
}
/**
* Persists all data in this update batch
*/
private void persistAllToDB() {
for (Persistable persistable : batch.values())
em.persist(persistable);
}
@Override
public String toString() {
final ArrayList<Persistable> values = new ArrayList<Persistable>(batch.values());
Persistable first = values.get(0);
Persistable last = values.get(values.size() - 1);
return "UpdateBatch[" +
first.getClass().getSimpleName() + "(" + first.getId() + ")" +
" - " +
last.getClass().getSimpleName() + "(" + last.getId() + ")" +
"]";
}
}
}