我是TensorFlow的新手,目前遇到一个非常烦人的问题。
我正在编写一个程序,它从tfrecord文件中使用tf.WholeFileReader.read(image_name_queue)
加载图像的"原始数据",然后使用tf.image.decode_jpeg(raw_data, channels=3)
进行解码,最后将其传递给向量化函数。
主要代码
logging.info('setting up folder')
create_image_data_folder()
save_configs()
logging.info('creating graph')
filename_queue = tf.train.string_input_producer([
configs.TFRECORD_IMAGES_PATH],
num_epochs=1)
image_tensor, name_tensor = read_and_decode(filename_queue)
image_batch_tensor, name_batch_tensor = tf.train.shuffle_batch(
[image_tensor, name_tensor],
configs.BATCH_SIZE,
1000 + 3 * configs.BATCH_SIZE,
min_after_dequeue=1000)
image_embedding_batch_tensor = configs.IMAGE_EMBEDDING_FUNCTION(image_batch_tensor)
init = tf.initialize_all_variables()
init_local = tf.initialize_local_variables()
logging.info('starting session')
with tf.Session().as_default() as sess:
sess.run(init)
sess.run(init_local)
tf.train.start_queue_runners()
logging.info('vectorizing')
data_points = []
for _ in tqdm(xrange(get_n_batches())):
name_batch = sess.run(name_batch_tensor)
image_embedding_batch = sess.run(image_embedding_batch_tensor)
for vector, name in zip(list(image_embedding_batch), name_batch):
data_points.append((vector, name))
logging.info('saving')
save_pkl_file(data_points, 'vectors.pkl')
read_and_decode函数
def read_and_decode(tfrecord_file_queue):
logging.debug('reading image and decodes it from queue')
reader = tf.TFRecordReader()
_, serialized_example = reader.read(tfrecord_file_queue)
features = tf.parse_single_example(serialized_example,
features={
'image': tf.FixedLenFeature([], tf.string),
'name': tf.FixedLenFeature([], tf.string)
}
)
image = process_image_data(features['image'])
return image, features['name']
这段代码本来是有效的,但最终它遇到了一个坏的、非jpeg格式的文件,就会抛出错误并且程序停止运行。
错误
InvalidArgumentError (see above for traceback): Invalid JPEG data, size 556663
我想跳过这些“错误”。我尝试用try
和except
包围代码。
新代码
for _ in tqdm(xrange(get_n_batches())):
try:
name_batch = sess.run(name_batch_tensor)
image_embedding_batch = sess.run(image_embedding_batch_tensor)
for vector, name in zip(list(image_embedding_batch), name_batch):
data_points.append((vector, name))
except Exception as e:
logging.warning('error occured: {}'.format(e))
当我再次运行程序时,仍然出现相同的错误,try
和 except
似乎无法处理该错误。如何处理这些异常?另外,如果您发现我误解了 TensorFlow 的“结构”,请提醒我。
tf.data.experimental.ignore_errors
成功解决了这个问题。 - sumitkanoje