我正在尝试将一个包含浮点数的csv文件写入tfrecords文件,然后再读取出来。所有我看到的示例都是将csv列打包,然后直接将它们馈送到sess.run()中,但我不知道如何将特征列和标签列写入tfrecord。我该怎么做呢?
我正在尝试将一个包含浮点数的csv文件写入tfrecords文件,然后再读取出来。所有我看到的示例都是将csv列打包,然后直接将它们馈送到sess.run()中,但我不知道如何将特征列和标签列写入tfrecord。我该怎么做呢?
您需要一份单独的脚本将您的CSV文件转换为TFRecords。
假设您有一个包含以下标题的CSV:
feature_1, feature_2, ..., feature_n, label
您需要使用类似于pandas
的工具读取CSV文件,手动构建tf.train.Example
,然后使用TFRecordWriter
将其写入文件。
csv = pandas.read_csv("your.csv").values
with tf.python_io.TFRecordWriter("csv.tfrecords") as writer:
for row in csv:
features, label = row[:-1], row[-1]
example = tf.train.Example()
example.features.feature["features"].float_list.value.extend(features)
example.features.feature["label"].int64_list.value.append(label)
writer.write(example.SerializeToString())
import pandas as pd
import tensorflow as tf
def create_tf_example(features, label):
tf_example = tf.train.Example(features=tf.train.Features(feature={
'Time': tf.train.Feature(bytes_list=tf.train.BytesList(value=[features[1].encode('utf-8')])),
'Height':tf.train.Feature(int64_list=tf.train.Int64List(value=[features[2]])),
'Width':tf.train.Feature(int64_list=tf.train.Int64List(value=[features[3]])),
'Mean':tf.train.Feature(float_list=tf.train.FloatList(value=[features[4]])),
'Std':tf.train.Feature(float_list=tf.train.FloatList(value=[features[5]])),
'Variance':tf.train.Feature(float_list=tf.train.FloatList(value=[features[6]])),
'Non-homogeneity':tf.train.Feature(float_list=tf.train.FloatList(value=[features[7]])),
'PixelCount':tf.train.Feature(int64_list=tf.train.Int64List(value=[features[8]])),
'contourCount':tf.train.Feature(int64_list=tf.train.Int64List(value=[features[9]])),
'Class':tf.train.Feature(bytes_list=tf.train.BytesList(value=[label.encode('utf-8')])),
}))
return tf_example
csv = pd.read_csv("dataset.csv").values
with tf.python_io.TFRecordWriter("dataset.tfrecords") as writer:
for row in csv:
features, label = row[:-1], row[-1]
print features, label
example = create_tf_example(features, label)
writer.write(example.SerializeToString())
writer.close()
更多详细信息请点击这里。这对我很有效,希望对你也有帮助。
TFRecordWriter
已经被移动到tf.io.TFRecordWriter
。 - Nitindef convert_to():
filename = os.path.join(wdir, 'ml-100k' + '.tfrecords')
print('Writing', filename)
with tf.python_io.TFRecordWriter(filename) as writer:
with open("/Users/shishir/Documents/botconnect_Playground/tfRecords/ml-100k.train.rating", "r") as f:
line = f.readline()
while line != None and line != "":
arr = line.split("\t")
u, i, l = int(arr[0]), int(arr[1]), int(arr[2])
u_arr = np.reshape(u,[1]).astype('int64')
i_arr = np.reshape(i,[1]).astype('int64')
l_arr = np.reshape(l,[1]).astype('int64')
example = tf.train.Example()
example.features.feature["user"].int64_list.value.extend(u_arr)
example.features.feature["item"].int64_list.value.extend(i_arr)
example.features.feature["label"].int64_list.value.append(int(l_arr))
writer.write(example.SerializeToString())
line = f.readline()