Client library to process URLs through Zyte API
Project description
Installation
pip install scrapy-zyte-apiThis package requires Python 3.7+.
How to configure
Replace the default http and https in Scrapy’s DOWNLOAD_HANDLERS in the settings.py of your Scrapy project.
You also need to set the ZYTE_API_KEY.
DOWNLOAD_HANDLERS = {
"http": "scrapy_zyte_api.handler.ScrapyZyteAPIDownloadHandler",
"https": "scrapy_zyte_api.handler.ScrapyZyteAPIDownloadHandler"
}
# Having the following in the env var would also work.
ZYTE_API_KEY = "<your API key>"Also, make sure to install the asyncio-based Twisted reactor in the settings.py file as well:
TWISTED_REACTOR = "twisted.internet.asyncioreactor.AsyncioSelectorReactor"How to use
Set the zyte_api Request.meta key to download a request using Zyte API. Full list of parameters is provided in the Zyte API Specification.
yield scrapy.Request(
"http://books.toscrape.com/",
callback=self.parse,
meta={
"zyte_api": {
"browserHtml": True,
"geolocation": "US",
"javascript": True,
"echoData": {"something": True}
}
}
)Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
scrapy-zyte-api-0.1.0.tar.gz
(4.1 kB
view hashes)
Built Distribution
Close
Hashes for scrapy_zyte_api-0.1.0-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | f91c36c6243ab310e5e38e4a8b6b927da8cc975671be61a17a4b75c7d50e21cf |
|
| MD5 | 43b3f6b5cf162872d9b175145246dffd |
|
| BLAKE2b-256 | 453e7305cba1a5499fd59d545a95b291d66b4ef2dcbfd6354768e4c68e81b0ac |
