Scrapy pipeline extension for qiniu.com
Project description
scrapy_qiniu
=====
适用于Scrapy的Pipeline插件。扩展了Scrapy自带的FilesPipeline,可以将静态资源上传到七牛云
存储上面。
* 支持缓存,可以避免静态资源的重复下载
* 采用fetch模式,让七牛服务器代为下载,而不用像默认的FilesPipeline那样,先下载到爬虫所在
主机,然后再上传到七牛服务器
=====
适用于Scrapy的Pipeline插件。扩展了Scrapy自带的FilesPipeline,可以将静态资源上传到七牛云
存储上面。
* 支持缓存,可以避免静态资源的重复下载
* 采用fetch模式,让七牛服务器代为下载,而不用像默认的FilesPipeline那样,先下载到爬虫所在
主机,然后再上传到七牛服务器
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
scrapy_qiniu-0.1.0.tar.gz
(3.6 kB
view hashes)
Built Distribution
Close
Hashes for scrapy_qiniu-0.1.0-py2-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2d4949b0f9db81fe2bea7074fbb9bfe79f401caff467c7dbc906e7741949ff7e |
|
MD5 | 8ce79a4e7725509aad4e447ef7307cfe |
|
BLAKE2b-256 | 7f97d70e837b91312531ba3e32e1d8abb16be3c5bf9ed3133761c70f3a3cbca2 |