我的目标是在将潜在层传递给自编码器的解码器模块之前,使用SVD对PCA进行白化处理。我已经使用了tf.linalg.svd,但它不起作用,因为它不包含必要的Keras参数。因此,我尝试将其包装在Lambda中,但出现了以下错误
AttributeError: 'tuple' object has no attribute 'shape'
我尝试了SO(例如在Keras / tensorflow中的自定义层中使用SVD),并在Google中搜索了SVD在Keras中的相关问题,但没有找到任何答案。我在这里附上了一个简化但功能齐全的代码:
要重现错误,请取消注释这些行并注释其前面的行:
AttributeError: 'tuple' object has no attribute 'shape'
我尝试了SO(例如在Keras / tensorflow中的自定义层中使用SVD),并在Google中搜索了SVD在Keras中的相关问题,但没有找到任何答案。我在这里附上了一个简化但功能齐全的代码:
import numpy as np
import tensorflow as tf
from sklearn import preprocessing
from keras.layers import Lambda, Input, Dense, Multiply, Subtract
from keras.models import Model
from keras import backend as K
from keras.losses import mse
from keras import optimizers
from keras.callbacks import EarlyStopping
x = np.random.randn(100, 5)
train_data = preprocessing.scale(x)
input_shape = (5, )
original_dim = train_data.shape[1]
intermediate_dim_1 = 64
intermediate_dim_2 = 16
latent_dim = 2
batch_size = 10
epochs = 15
# build encoder model
inputs = Input(shape=input_shape, name='encoder_input')
layer_1 = Dense(intermediate_dim_1, activation='tanh') (inputs)
layer_2 = Dense(intermediate_dim_2, activation='tanh') (layer_1)
encoded_layer = Dense(latent_dim, name='latent_layer') (layer_2)
encoder = Model(inputs, encoded_layer, name='encoder')
encoder.summary()
# build decoder model
latent_inputs = Input(shape=(latent_dim,))
layer_1 = Dense(intermediate_dim_1, activation='tanh') (latent_inputs)
layer_2 = Dense(intermediate_dim_2, activation='tanh') (layer_1)
outputs = Dense(original_dim,activation='sigmoid') (layer_2)
decoder = Model(latent_inputs, outputs, name='decoder')
decoder.summary()
# mean removal and pca whitening
meanX = Lambda(lambda x: tf.reduce_mean(x, axis=0, keepdims=True))(encoded_layer)
standardized = Subtract()([encoded_layer, meanX])
sigma2 = K.dot(K.transpose(standardized), standardized)
sigma2 = Lambda(lambda x: x / batch_size)(sigma2)
s, u ,v = tf.linalg.svd(sigma2,compute_uv=True)
# s ,u ,v = Lambda(lambda x: tf.linalg.svd(x,compute_uv=True))(sigma2)
epsilon = 1e-6
# sqrt of number close to 0 leads to problem hence replace it with epsilon
si = tf.where(tf.less(s, epsilon), tf.sqrt(1 / epsilon) * tf.ones_like(s),
tf.math.truediv(1.0, tf.sqrt(s)))
whitening_layer = u @ tf.linalg.diag(si) @ tf.transpose(v)
whitened_encoding = K.dot(standardized, whitening_layer)
# Connect models
z_decoded = decoder(standardized)
# z_decoded = decoder(whitened_encoding)
# Define losses
reconstruction_loss = mse(inputs,z_decoded)
# Instantiate autoencoder
ae = Model(inputs, z_decoded, name='autoencoder')
ae.add_loss(reconstruction_loss)
# callback = EarlyStopping(monitor='val_loss', patience=5)
adam = optimizers.adam(learning_rate=0.002)
ae.compile(optimizer=adam)
ae.summary()
ae.fit(train_data, epochs=epochs, batch_size=batch_size,
validation_split=0.2, shuffle=True)
要重现错误,请取消注释这些行并注释其前面的行:
如果有人能告诉我如何在Keras层中包装SVD或备用实现,我将不胜感激。 请注意,为了保持代码简单,我没有包括重新参数化技巧来计算损失。 谢谢!
z_decoded = decoder(whitened_encoding)
s,u,v = Lambda(lambda x:tf.linalg.svd(x,compute_uv = True))(sigma2)