在TensorFlow中出现IndexError: list index out of range错误

Question

我遇到了一个错误，IndexError: list index out of range。Traceback 显示：

Run id: P0W5X0
Log directory: /tmp/tflearn_logs/
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.15/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/local/Cellar/python@2/2.7.15/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/Users/xxx/anaconda/xxx/lib/python2.7/site-packages/tflearn/data_flow.py", line 201, in fill_batch_ids_queue
    ids = self.next_batch_ids()
  File "/Users/xxx/anaconda/xxx/lib/python2.7/site-packages/tflearn/data_flow.py", line 215, in next_batch_ids
    batch_start, batch_end = self.batches[self.batch_index]
IndexError: list index out of range

I wrote codes,

# coding: utf-8
import tensorflow as tf
import tflearn

from tflearn.layers.core import input_data,dropout,fully_connected
from tflearn.layers.conv import conv_2d, max_pool_2d
from tflearn.layers.normalization import local_response_normalization
from tflearn.layers.estimator import regression

tf.reset_default_graph()
net = input_data(shape=[None,20000, 4, 42])
net = conv_2d(net, 4, 16, activation='relu')
net = max_pool_2d(net, 1)
net = tflearn.activations.relu(net)
net = dropout(net, 0.5)
net = tflearn.fully_connected(net, 2, activation='softmax')
net = tflearn.regression(net, optimizer='adam', learning_rate=0.5, loss='categorical_crossentropy')

model = tflearn.DNN(net)

model.fit(np.array(trainDataSet).reshape(1,20000, 4, 42), np.array(trainLabel), n_epoch=400, batch_size=32, validation_set=0.1, show_metric=True)


pred = np.array(model.predict(np.array(testDataSet).reshape(1,20000, 4, 42)).argmax(axis=1))

label = np.array(testLabel).argmax(axis=0)
accuracy = np.mean(pred == label, axis=0)

print(accuracy)

我真的不明白为什么会出现这样的错误。我尝试将 <p> 重写成

model.fit(np.array(trainDataSet).reshape(1,20000, 4, 42), np.array(trainLabel), n_epoch=400, batch_size=1, validation_set=0.1, show_metric=True) 

因为bach引起了这个错误，但是同样的错误仍然存在。我改写了这部分的另一个数字，但是仍然出现了相同的错误。我的代码有什么问题？我该如何修复它？

- user10492592

---

请告诉我关于训练数据集的情况。它是什么，为什么形状是这样的？从输入数据（shape=[None,20000, 4, 42]）来看，您似乎期望一些形状为20000x4x42的批次数量，但在model.fit中却只提供了一个20000x4x42的样本。 - Anton Codes

我有点听不懂你在说什么。np.array(trainDataSet).shape 是 (20000, 4, 42)。 - user10492592

Answer 1

问题

如何解决列表索引超出范围的错误？

最初的回答

从您的代码中可以看出，您传递到神经网络中的训练和测试集只有一个元素，由reshape(1,20000, 4, 42)得到形状为20000x4x42。我认为您的意思是要有20000个4x42的元素。

我们可以使用reshape(20000, 4, 42, 1)代替reshape(1,20000, 4, 42)，也必须将input_data(shape=[None, 20000, 4, 42])更改为input_data(shape=[None, 4, 42, 1])

如果这样做，您的代码就可以正常工作了。

可行的代码

# coding: utf-8
import tensorflow as tf
import tflearn

from tflearn.layers.core import input_data,dropout,fully_connected
from tflearn.layers.conv import conv_2d, max_pool_2d
from tflearn.layers.normalization import local_response_normalization
from tflearn.layers.estimator import regression

tf.reset_default_graph()
net = input_data(shape=[None, 4, 42, 1])
net = conv_2d(net, 4, 16, activation='relu')
net = max_pool_2d(net, 1)
net = tflearn.activations.relu(net)
net = dropout(net, 0.5)
net = tflearn.fully_connected(net, 2, activation='softmax')
net = tflearn.regression(net, optimizer='adam', learning_rate=0.5, loss='categorical_crossentropy')

model = tflearn.DNN(net)

model.fit(np.array(trainDataSet).reshape(20000, 4, 42, 1), np.array(trainLabel), n_epoch=400, batch_size=32, validation_set=0.1, show_metric=True)


pred = np.array(model.predict(np.array(testDataSet).reshape(20000, 4, 42, 1)).argmax(axis=1))

label = np.array(testLabel).argmax(axis=0)
accuracy = np.mean(pred == label, axis=0)

print(accuracy)

输出

为了让上述代码正常工作，我们需要包含一些训练和测试数据。Numpy随机数的使用方式如下所示：

import numpy as np

trainDataSet = np.random.rand(20000, 4, 42)
trainLabel = ( np.random.rand(20000,2) > .5 ) *1.0

testDataSet = np.random.rand(20000, 4, 42)
testLabel = ( np.random.rand(20000,2) > .5 ) *1.0

最初的回答：

这是输出结果

Run id: JDSG88
Log directory: /tmp/tflearn_logs/
---------------------------------
Training samples: 18000
Validation samples: 2000
--
Training Step: 563  | total loss: 12.13387 | time: 5.312s
| Adam | epoch: 001 | loss: 12.13387 - acc: 0.7138 | val_loss: 11.90437 - val_acc: 0.7400 -- iter: 18000/18000
--
Training Step: 1126  | total loss: 11.58909 | time: 5.184s
| Adam | epoch: 002 | loss: 11.58909 - acc: 0.7496 | val_loss: 11.90437 - val_acc: 0.7400 -- iter: 18000/18000
--
Training Step: 1689  | total loss: 11.93482 | time: 5.174s
| Adam | epoch: 003 | loss: 11.93482 - acc: 0.7357 | val_loss: 11.90437 - val_acc: 0.7400 -- iter: 18000/18000
--
...

- Anton Codes

Answer 2

我也曾经遇到了你同样的问题。我的解决方法是将 n_epoch 的数量设为数据集行数。例如，如果我的数组形状为461*5，则n_epoch的值为461。你也可以将这个值稍微大一点或小一点，以适应你的数据集。在我的代码中，500或400也是有用的。

- taylor