Keras LSTM-VAE（变分自编码器）用于时间序列异常检测

Question

我正在尝试使用Keras对时间序列进行重建的LSTM-VAE模型。

我参考了https://github.com/twairball/keras_lstm_vae/blob/master/lstm_vae/vae.py和https://machinelearningmastery.com/lstm-autoencoders/来创建LSTM-VAE架构。

我在训练网络时遇到了麻烦，我在eager执行模式下训练时得到以下错误：

  InvalidArgumentError: Incompatible shapes: [8,1] vs. [32,1] [Op:Mul]

输入形状为(7752,30,1)，其中30个时间步长和1个特征。

模型编码器：

# encoder
latent_dim = 1
inter_dim = 32

#sample,timesteps, features
input_x = keras.layers.Input(shape= (X_train.shape[1], X_train.shape[2])) 

#intermediate dimension 
h = keras.layers.LSTM(inter_dim)(input_x)

#z_layer
z_mean = keras.layers.Dense(latent_dim)(h)
z_log_sigma = keras.layers.Dense(latent_dim)(h)
z = Lambda(sampling)([z_mean, z_log_sigma])

模型解码器：

# Reconstruction decoder
decoder1 = RepeatVector(X_train.shape[1])(z)
decoder1 = keras.layers.LSTM(100, activation='relu', return_sequences=True)(decoder1)
decoder1 = keras.layers.TimeDistributed(Dense(1))(decoder1)

抽样函数：

batch_size = 32
def sampling(args):
    z_mean, z_log_sigma = args
    epsilon = K.random_normal(shape=(batch_size, latent_dim),mean=0., stddev=1.)
    return z_mean + z_log_sigma * epsilon

VAE损失函数：

def vae_loss2(input_x, decoder1):
    """ Calculate loss = reconstruction loss + KL loss for each data in minibatch """
    # E[log P(X|z)]
    recon = K.sum(K.binary_crossentropy(input_x, decoder1), axis=1)
    # D_KL(Q(z|X) || P(z|X)); calculate in closed form as both dist. are Gaussian
    kl = 0.5 * K.sum(K.exp(z_log_sigma) + K.square(z_mean) - 1. - z_log_sigma, axis=1)

    return recon + kl

LSTM-VAE模型架构

有没有建议可以使模型工作得更好？

- JINU RAJ

---

VAE LSTM 用于时间序列：https://towardsdatascience.com/time-series-generation-with-vae-lstm-5a6426365a1c - Marco Cerliani

Answer 1

您需要在采样函数内部推断batch_dim，并注意您的损失...您的损失函数使用之前层的输出，因此您需要处理这个问题。我使用 model.add_loss(...)来实现这一点。

# encoder
latent_dim = 1
inter_dim = 32
timesteps, features = 100, 1

def sampling(args):
    z_mean, z_log_sigma = args
    batch_size = tf.shape(z_mean)[0] # <================
    epsilon = K.random_normal(shape=(batch_size, latent_dim), mean=0., stddev=1.)
    return z_mean + z_log_sigma * epsilon

# timesteps, features
input_x = Input(shape= (timesteps, features)) 

#intermediate dimension 
h = LSTM(inter_dim, activation='relu')(input_x)

#z_layer
z_mean = Dense(latent_dim)(h)
z_log_sigma = Dense(latent_dim)(h)
z = Lambda(sampling)([z_mean, z_log_sigma])

# Reconstruction decoder
decoder1 = RepeatVector(timesteps)(z)
decoder1 = LSTM(inter_dim, activation='relu', return_sequences=True)(decoder1)
decoder1 = TimeDistributed(Dense(features))(decoder1)

def vae_loss2(input_x, decoder1, z_log_sigma, z_mean):
    """ Calculate loss = reconstruction loss + KL loss for each data in minibatch """
    # E[log P(X|z)]
    recon = K.sum(K.binary_crossentropy(input_x, decoder1))
    # D_KL(Q(z|X) || P(z|X)); calculate in closed form as both dist. are Gaussian
    kl = 0.5 * K.sum(K.exp(z_log_sigma) + K.square(z_mean) - 1. - z_log_sigma)

    return recon + kl

m = Model(input_x, decoder1)
m.add_loss(vae_loss2(input_x, decoder1, z_log_sigma, z_mean)) #<===========
m.compile(loss=None, optimizer='adam')

这里是正在运行的笔记本

- Marco Cerliani

---

谢谢@Marco Cerliani！在TF 2.3版本中它起作用了！ - JINU RAJ