Skip to main content

Scrapy entrypoint for Scrapinghub job runner

Project description

Scrapy entrypoint for Scrapinghub job runner.

The package implements a base wrapper layer to extract job data from environment, parse/prepare it properly and execute job using Scrapy or custom executor.

Features

  • parsing job data from environment

  • processing job args and settings

  • running a job with Scrapy

  • collecting stats

  • advanced logging & error handling

  • full hubstorage support

  • custom scripts support

Install

pip install scrapinghub-entrypoint-scrapy

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scrapinghub-entrypoint-scrapy-0.10.1.tar.gz (25.7 kB view hashes)

Uploaded Source

Built Distribution

scrapinghub_entrypoint_scrapy-0.10.1-py2.py3-none-any.whl (34.8 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page