Tensorflow训练的while_loop

Question

Tensorflow训练的while_loop

13

在我的问题中，我需要在每个训练步骤中使用数据中的一个示例来运行GD。众所周知，session.run()存在开销，因此训练模型的时间太长。为了避免开销，我尝试使用while_loop并在一次run()调用中对所有数据进行训练。但是这种方法不起作用，train_op甚至没有执行。下面是我正在做的简单示例：

data = [k*1. for k in range(10)]
tf.reset_default_graph()

i = tf.Variable(0, name='loop_i')
q_x = tf.FIFOQueue(100000, tf.float32)
q_y = tf.FIFOQueue(100000, tf.float32)

x = q_x.dequeue()
y = q_y.dequeue()
w = tf.Variable(0.)
b = tf.Variable(0.)
loss = (tf.add(tf.mul(x, w), b) - y)**2

gs = tf.Variable(0)

train_op = tf.train.GradientDescentOptimizer(0.05).minimize(loss, global_step=gs)

s = tf.Session()
s.run(tf.initialize_all_variables())

def cond(i):
    return i < 10

def body(i):
    return tf.tuple([tf.add(i, 1)], control_inputs=[train_op])


loop = tf.while_loop(cond, body, [i])

for _ in range(1):
    s.run(q_x.enqueue_many((data, )))
    s.run(q_y.enqueue_many((data, )))

s.run(loop)
s.close()

我做错了什么？还是有其他解决这个开销过高问题的方法吗？

谢谢！

- Andrey Atanov

1个回答

网页内容由stack overflow 提供, 点击上面的

可以查看英文原文，
原文链接

- mrry · Accepted Answer

模型似乎没有训练的原因是，输入读取、梯度计算和minimize()调用都定义在循环体外部（因此，在数据流术语中，它们都在循环之前）。这意味着，所有这些模型的部分只运行一次，在循环执行之前，而循环本身没有任何效果。

稍微重构一下——将dequeue()操作、梯度计算和minimize()调用移到循环内部——可以解决问题，并允许程序进行训练：

optimizer = tf.train.GradientDescentOptimizer(0.05)

def cond(i):
    return i < 10

def body(i):
    # Dequeue a new example each iteration.
    x = q_x.dequeue()
    y = q_y.dequeue()

    # Compute the loss and gradient update based on the current example.
    loss = (tf.add(tf.mul(x, w), b) - y)**2
    train_op = optimizer.minimize(loss, global_step=gs)

    # Ensure that the update is applied before continuing.
    return tf.tuple([tf.add(i, 1)], control_inputs=[train_op])

loop = tf.while_loop(cond, body, [i])

更新：这是一个完整的程序，执行了while循环，基于您在问题中提供的代码：

import tensorflow as tf

# Define a single queue with two components to store the input data.
q_data = tf.FIFOQueue(100000, [tf.float32, tf.float32])

# We will use these placeholders to enqueue input data.
placeholder_x = tf.placeholder(tf.float32, shape=[None])
placeholder_y = tf.placeholder(tf.float32, shape=[None])
enqueue_data_op = q_data.enqueue_many([placeholder_x, placeholder_y])

gs = tf.Variable(0)
w = tf.Variable(0.)
b = tf.Variable(0.)
optimizer = tf.train.GradientDescentOptimizer(0.05)

# Construct the while loop.
def cond(i):
    return i < 10

def body(i):
    # Dequeue a single new example each iteration.
    x, y = q_data.dequeue()
    # Compute the loss and gradient update based on the current example.
    loss = (tf.add(tf.multiply(x, w), b) - y) ** 2
    train_op = optimizer.minimize(loss, global_step=gs)
    # Ensure that the update is applied before continuing.
    with tf.control_dependencies([train_op]):
        return i + 1

loop = tf.while_loop(cond, body, [tf.constant(0)])

data = [k * 1. for k in range(10)]

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for _ in range(1):
        # NOTE: Constructing the enqueue op ahead of time avoids adding
        # (potentially many) copies of `data` to the graph.
        sess.run(enqueue_data_op,
                 feed_dict={placeholder_x: data, placeholder_y: data})
    print (sess.run([gs, w, b]))  # Prints before-loop values.
    sess.run(loop)
    print (sess.run([gs, w, b]))  # Prints after-loop values.