scrapy [boto] ERROR: Caught exception reading instance data URLError: <urlopen error [Errno 10051


20150909 11:13:26 [boto] DEBUG: Retrieving credentials from metadata server. 20150909 11:13:27 [boto] ERROR: Caught exception reading instance data
Traceback (most recent call last):
File “”, line 210, in retry_url
r =, timeout=timeout)
File “”, line 431, in open response = self._open(req, data)
File “”, line 449, in _open _open, req)
File “”, line 409, in _call_chain result = func(*args)
File “”, line 1227, in http_open return self.do_open(httplib.HTTPConnection, req)
File “”, line 1197, in do_open
raise URLError(err)
URLError:20150909 11:13:27 [boto] ERROR: Unable to read instance data, giving up

在setting.py中禁用s3 download就能够了


That particular error message is being generated by boto (boto 2.38.0 py27_0), which is used to connect to Amazon S3. Scrapy doesn’t have this enabled by default.

EDIT: In reply to the comments, this appears to be a bug with Scrapy when boto is present (bug here).

In response “how to disable the Download handler”, add the following to your file:


Your file should be in the root of your Scrapy project folder, (one level deeper than your scrapy.cfg file).

If you’ve already got DOWNLOAD_HANDLERS in your file, just add a new entry for ‘s3’ with a None value.

EDIT 2: I’d highly recommend looking at setting up virtual environments for your projects. Look into virtualenv, and it’s usage. I’d make this recommendation regardless of packages used for this project, but doubly so with your extreme number of packages.

波比源码 – 精品源码模版分享 |
1. 本站所有资源来源于用户上传和网络,如有侵权请邮件联系站长!
2. 分享目的仅供大家学习和交流,您必须在下载后24小时内删除!
3. 不得使用于非法商业用途,不得违反国家法律。否则后果自负!
4. 本站提供的源码、模板、插件等等其他资源,都不包含技术服务请大家谅解!
5. 如有链接无法下载、失效或广告,请联系管理员处理!
6. 本站资源售价只是赞助,收取费用仅维持本站的日常运营所需!
7. 如遇到加密压缩包,请使用WINRAR解压,如遇到无法解压的请联系管理员!

波比源码 » scrapy [boto] ERROR: Caught exception reading instance data URLError: <urlopen error [Errno 10051


Hi, 如果你对这款模板有疑问,可以跟我联系哦!

赞助VIP 享更多特权,建议使用 QQ 登录