如何在Python 3中使用https URL下载图片？

Question

我尝试使用 Python 编写一个简短的批量下载脚本，用来将图片列表存储到本地。

它对于 http 图片的 url 工作得非常好，但是对于任何带有 https 的 url 却无法下载。涉及到问题的代码行如下：

import urllib.request
urllib.request.urlretrieve(url, filename)

例如，https://cdn.discordapp.com/attachments/299398003486097412/303580387786096641/FB_IMG_1490534565948.jpg 会导致HTTP Error 403: Forbidden，以及任何其他https图像。

这让我有两个问题：

1. 我如何使用Python下载这样的图像？
2. 为什么图像甚至有https URL，如果它们基本上只是文件？

编辑：

以下是堆栈跟踪：

Traceback (most recent call last):
  File "img_down.py", line 52, in <module>
    main()
  File "img_down.py", line 38, in main
    save_img(d, l)
  File "img_down.py", line 49, in save_img
    stream = read_img(url)
  File "img_down.py", line 42, in read_img
    with urllib.request.urlopen(url) as response:
  File "D:\Users\Jan\AppData\Local\Programs\Python\Python36-32\lib\urllib\request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "D:\Users\Jan\AppData\Local\Programs\Python\Python36-32\lib\urllib\request.py", line 532, in open
    response = meth(req, response)
  File "D:\Users\Jan\AppData\Local\Programs\Python\Python36-32\lib\urllib\request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "D:\Users\Jan\AppData\Local\Programs\Python\Python36-32\lib\urllib\request.py", line 570, in error
    return self._call_chain(*args)
  File "D:\Users\Jan\AppData\Local\Programs\Python\Python36-32\lib\urllib\request.py", line 504, in _call_chain
    result = func(*args)
  File "D:\Users\Jan\AppData\Local\Programs\Python\Python36-32\lib\urllib\request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

- Jan Schultke

---

你尝试过这篇帖子上的解决方案吗？https://dev59.com/33A75IYBdhLWcg3w6Ndj - jose_bacoy

@anonyXmous 我已经尝试过了。当面对这个URL时，它也会引发403 HTTP错误。 - Jan Schultke

Answer 1

希望这有所帮助。

import requests
with open('FB_IMG_1490534565948.jpg', 'wb') as f:
    f.write(requests.get('https://url/to/image.jpg').content)

- jose_bacoy

Answer 2

或许能帮到你...

我编写了这个脚本，但是从未完成（最终目的是使其每天自动运行）。

但是为了不拖延回答，这里是你感兴趣的代码片段：

    def downloadimg(self):
        import datetime
        imgurl = self.getdailyimg();
        imgfilename = datetime.datetime.today().strftime('%Y%m%d') + '_' + imgurl.split('/')[-1]
        with open(IMGFOLDER + imgfilename, 'wb') as f:
            f.write(self.readimg(imgurl))

希望这些对你有所帮助！

编辑后：

附：使用Python3的完整脚本

#!/usr/bin/python
# -*- coding: utf-8 -*-

import os
IMGFOLDER = os.getcwd() + '/images/'


class BingImage(object):
    """docstring for BingImage"""
    BINGURL = 'http://www.bing.com/'
    JSONURL = 'HPImageArchive.aspx?format=js&idx=0&n=1&mkt=pt-BR'
    LASTIMG = None

    def __init__(self):
        super(BingImage, self).__init__()
        try:
            self.downloadimg()
        except:
            pass

    def getdailyimg(self):
        import json
        import urllib.request
        with urllib.request.urlopen(self.BINGURL + self.JSONURL) as response:
            rawjson = response.read().decode('utf-8')
            parsedjson = json.loads(rawjson)
            return self.BINGURL + parsedjson['images'][0]['url'][1:]

    def downloadimg(self):
        import datetime
        imgurl = self.getdailyimg();
        imgfilename = datetime.datetime.today().strftime('%Y%m%d') + '_' + imgurl.split('/')[-1]
        with open(IMGFOLDER + imgfilename, 'wb') as f:
            f.write(self.readimg(imgurl))
        self.LASTIMG = IMGFOLDER + imgfilename

    def checkfolder(self):
        d = os.path.dirname(IMGFOLDER)
        if not os.path.exists(d):
            os.makedirs(d)

    def readimg(self, url):
        import urllib.request
        with urllib.request.urlopen(url) as response:
            return response.read()


def DefineBackground(src):
    import platform
    if platform.system() == 'Linux':
        MAINCMD = "gsettings set org.gnome.desktop.background picture-uri"
        os.system(MAINCMD + ' file://' + src)


def GetRandomImg():
    """Return a random image already downloaded from the images folder"""
    import random
    f = []
    for (dirpath, dirnames, filenames) in os.walk(IMGFOLDER):
        f.extend(filenames)
        break
    return IMGFOLDER + random.choice(f)


if __name__ == '__main__':
    # get a new today's image from Bing
    img = BingImage()
    # check whether a new image was get or not
    if(img.LASTIMG):
        DefineBackground(img.LASTIMG)
    else:
        DefineBackground(GetRandomImg())
    print('Background defined')

- Thiago Cardoso

---

你肯定没有读我上面发布的链接...这是我创建的一个类的一部分，用来下载必应网站上的今日图片...真正有趣的是如何打开文件（注意'wb'的部分，以使其不仅可写，还要在二进制模式下），并将原始内容保存在其中。 - Thiago Cardoso

Answer 3

这是关于那个问题的最新答案，我使用了openCV来存储图像，并结合请求模块。此外，它还可以处理批量操作并作为通用代码添加。

import numpy as np
from urllib.request import urlopen
import cv2
import os
current_path = os.getcwd()
try: os.mkdir(current_path + "\\Downloaded\\")
except:pass

def downloadImage(url):
    try:
        print("Downloading %s" % (url))
        image_name = str(url).split('/')[-1]
        resp = urlopen(url)
        image = np.asarray(bytearray(resp.read()), dtype="uint8")
        image = cv2.imdecode(image, cv2.IMREAD_COLOR)
        cv2.imwrite(current_path + "\\Downloaded\\" + image_name, image)
    except Exception as error:
        print(error)

if __name__ == '__main__':
    urls = ["https://www.google.com/logos/doodles/2019/st-georges-day-2019-6234830302871552.20-2x.png"]
    for url in urls:
        downloadImage(url)

- Dhamodharan

Answer 4

参考以下链接这里，即使使用@Thiago Cardoso的解决方案，您可能仍会遇到“HTTP错误403：禁止”错误，因为服务器不知道请求来自哪里。一些网站将验证UserAgent以防止异常访问。因此，您应该提供虚假浏览器访问的信息。

因此，我修改了读取方法的代码如下：

def readimg(self, img_url):
    from urllib.request import urlopen, Request
    headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.3'}
    req = Request(url=img_url, headers=headers) 
    with urlopen(req) as response:
        return response.read()

- Ramy Ibrahim

Answer 5

你需要创建一个userAgent。 这可能是服务器安全功能，阻止未知的用户代理。

如果你设置已知的浏览器用户代理，将起作用。

def download_img(img_url, img_name):
    request = Request(img_url, headers={'User-Agent': 'Mozilla/5.0'})
    response = urlopen(request)
    with open(img_name, "wb") as f:
       f.write(response.read())

- Lucas Orso