在Python 3中在线从CSV文件读取数据

Question

作为练习，我想测试一些东西。我有一个在线的巨大CSV文件https://raw.github.com/datasets/gdp/master/data/gdp.csv ，我想读取所有数据并将其放入表格中，以便分析和制作表格。 到目前为止，我所拥有的代码是使用其他StackOverflow问题和其他网站组合而成。但是当它被读取并立即再次打印时，结果是逐字逐句地输出，因此我得到：

['C']
['o']
['u']
['n']
['t']
['r']
['y']
[' ']
['N']
['a']
['m']
['e']
['', '']
['C']
['o']
['u']
['n']
['t']
['r']
['y']
[' ']
['C']
['o']
['d']
['e']
['', '']
['Y']
['e']
['a']
['r']
['', '']
['V']
['a']
['l']
['u']
['e']
[]
[]
['A']
['r']
['a']
['b']
[' ']
['W']
['o']
['r']
['l']
['d']
['', '']

我目前的代码如下：

import csv
import urllib.request

url = "https://raw.github.com/datasets/gdp/master/data/gdp.csv"
webpage = urllib.request.urlopen(url)
datareader = csv.reader(webpage.read().decode('utf-8'))
data = []
for row in datareader:
    data.append(row)

for row in data:
    print(row)

如何更改代码使其逐行读取并将每一行拆分为不同的变量。以前我用过以下代码：

payRollNumber, salary, jobTitle, otherNames, \
               surname = line.strip().split(',')

我得到行后可以应用这个。有什么想法吗？

- DonnellyOverflow

Answer 1

在传递给csv.reader()之前，您需要按行拆分读取的 CSV 数据：

datareader = csv.reader(webpage.read().decode('utf-8').splitlines())

csv.reader() 能够为你处理剩下的部分。

你也可以使用 io.TextIOWrapper() 来帮你处理读取、解码和按行处理等内容。

import csv
import io
import urllib.request

url = "https://raw.github.com/datasets/gdp/master/data/gdp.csv"
webpage = urllib.request.urlopen(url)
datareader = csv.reader(io.TextIOWrapper(webpage))

循环读取器并将行添加到列表中意义不大；你可以直接这样做：

data = list(datareader)

如果你只是想打印出列，可以直接遍历reader并这样做：

datareader = csv.reader(io.TextIOWrapper(webpage))
for row in datareader:
    print(row)

不管怎样，无论是手动分割行还是使用TextIOWrapper，代码现在会生成：

['Country Name', 'Country Code', 'Year', 'Value']
['Arab World', 'ARB', '1968', '32456179321.45']
['Arab World', 'ARB', '1969', '35797666653.6002']
['Arab World', 'ARB', '1970', '39062044200.4362']
['Arab World', 'ARB', '1971', '45271917893.3429']
['Arab World', 'ARB', '1972', '54936622019.8224']
['Arab World', 'ARB', '1973', '69564884441.8264']
['Arab World', 'ARB', '1974', '132123836511.468']
['Arab World', 'ARB', '1975', '147666389454.913']
['Arab World', 'ARB', '1976', '182208407088.856']
# ... etc. ...

- Martijn Pieters

Answer 2

更好的处理CSV的方法。

根据您的方便，您可以使用csv reader或DictReader。

CSV Reader

['Country Name', 'Country Code', 'Year', 'Value'] ['Arab World', 'ARB', '1968', '32456179321.45']... ...

DictReader

{"Country Name": 'Arab World', 'Country Code': 'ARB', 'Year':'1968', 'Value':'32456179321.45'} ... ...

import requests

download = requests.get(CSV_URL)
decoded_content = download.content.decode('utf-8')
file = decoded_content.splitlines()

# cr = csv.reader(file, delimiter=',')
cr = csv.DictReader(file, delimiter=',')
my_list = list(cr)
for row in my_list:
    print(row)

- Amrit Prasad