Tensorflow总是预测相同的结果。

Question

我正在尝试使用自己的数据运行TensorFlow示例，但是分类器总是选择每个测试样例相同的类。输入数据始终在之前进行了洗牌。我有约4000张图像作为训练集和500张图像作为测试集。

我得到的结果看起来像：

Result: [[ 1.  0.]] Actually: [ 1.  0.] 
Result: [[ 1.  0.]] Actually: [ 0.  1.] 
Result: [[ 1.  0.]] Actually: [ 1.  0.] 
Result: [[ 1.  0.]] Actually: [ 1.  0.] 
Result: [[ 1.  0.]] Actually: [ 0.  1.] 
Result: [[ 1.  0.]] Actually: [ 0.  1.]
...

右侧仍为所有500张图像[1. 0.]。分类是二进制的，所以只有两个标签。

以下是我的源代码：

import tensorflow as tf
import input_data as id

test_images, test_labels = id.read_images_from_csv(
    "/home/johnny/Desktop/tensorflow-examples/46-model.csv")

train_images = test_images[:4000]
train_labels = test_labels[:4000]
test_images = test_images[4000:]
test_labels = test_labels[4000:]

print len(train_images)
print len(test_images)

pixels = 200 * 200
labels = 2

sess = tf.InteractiveSession()

# Create the model
x = tf.placeholder(tf.float32, [None, pixels])
W = tf.Variable(tf.zeros([pixels, labels]))
b = tf.Variable(tf.zeros([labels]))
y_prime = tf.matmul(x, W) + b
y = tf.nn.softmax(y_prime)

# Define loss and optimizer
y_ = tf.placeholder(tf.float32, [None, labels])
cross_entropy = tf.nn.softmax_cross_entropy_with_logits(y_prime, y_)
train_step = tf.train.GradientDescentOptimizer(0.01).minimize(cross_entropy)

# Train
tf.initialize_all_variables().run()
for i in range(10):
    res = train_step.run({x: train_images, y_: train_labels})
# Test trained model
correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print(accuracy.eval({x: test_images, y_: test_labels}))

for i in range(0, len(test_images)):
    res = sess.run(y, {x: [test_images[i]]})
    print("Result: " + str(res) + " Actually: " + str(test_labels[i]))

我是否漏掉了什么重要的点？

- Johnny000

---

面对相同的问题，您能告诉我们您是如何解决上述问题的吗？ - xtluo

@XiaotaoLuo请看答案... - Johnny000

1. 我正在使用默认的权重初始化器glorot_uniform_initializer，根据这个问题。
2. cross_entropy是正确的。
3. 减小批量大小，但仍然对所有输入得到相同的预测。

- xtluo

Answer 1

你的代码有三个潜在问题：

1. The weights, W, are initialized to zero. This question from stats.stackexchange.com has a good discussion of why this can lead to poor training outcomes (such as getting stuck in a local minimum). Instead, you should initialize them randomly, for example as follows:

   W = tf.Variable(tf.truncated_normal([pixels, labels],
                                       stddev=1./math.sqrt(pixels)))
   

2. The cross_entropy should be aggregated to a single, scalar value before minimizing it, using for example tf.reduce_mean():

   cross_entropy = tf.reduce_mean(
       tf.nn.softmax_cross_entropy_with_logits(y_prime, y_))
   

3. You may get faster convergence if you train on mini-batches (or even single examples) rather than training on the entire dataset at once:

   for i in range(10):
           for j in range(4000):
               res = train_step.run({x: train_images[j:j+1],
                                     y_: train_labels[j:j+1]})
   

- mrry

---

谢谢你的提示，帮了我大忙。 - Johnny000

Answer 2

你可能面临的另一个问题是类别不平衡。如果有一个类比其他类数量多很多，你的函数可能会收敛到那个值上。尝试平衡训练样本中的类别，并使用较小的批次。例如，如果您的标签是二进制的，请确保训练样本中零和一的标签数量相等。

- user2498105