Scrapy item exporter for the Algolia API
Project description
[![PyPI](https://img.shields.io/pypi/v/scrapy_algolia_exporter.svg)](https://pypi.python.org/pypi/scrapy_algolia_exporter) [![Build Status](https://travis-ci.org/clemfromspace/scrapy-algolia-exporter.svg?branch=master)](https://travis-ci.org/clemfromspace/scrapy-algolia-exporter)
## Scrapy Algolia Exporter
Directly populate a given **Algolia** index from a scrapy spider.
### Usage
**Your crawled items must contains a unique `ObjectID` key.**
Add the mandatory scrapy settings to configure the **Algolia** API:
```
ALGOLIA_API_ID='my_algolia_api_id'
ALGOLIA_API_KEY='my_algolia_api_key'
ALGOLIA_INDEX_NAME='my_algolia_index_name'
```
Add the `AlgoliaItemPipeline` in the `ITEM_PIPELINES` scrapy setting:
```
ITEM_PIPELINES = {
'scrapy_algolia_exporter.pipelines.AlgoliaItemPipeline': 10
}
```
The `ALGOLIA_ITEM_BULK_NBR` setting control how many items will be send to Algolia at the same time.
If not provided, the items will be send by group of `100`.
### Install
```
pip install scrapy_algolia_exporter
```
## Scrapy Algolia Exporter
Directly populate a given **Algolia** index from a scrapy spider.
### Usage
**Your crawled items must contains a unique `ObjectID` key.**
Add the mandatory scrapy settings to configure the **Algolia** API:
```
ALGOLIA_API_ID='my_algolia_api_id'
ALGOLIA_API_KEY='my_algolia_api_key'
ALGOLIA_INDEX_NAME='my_algolia_index_name'
```
Add the `AlgoliaItemPipeline` in the `ITEM_PIPELINES` scrapy setting:
```
ITEM_PIPELINES = {
'scrapy_algolia_exporter.pipelines.AlgoliaItemPipeline': 10
}
```
The `ALGOLIA_ITEM_BULK_NBR` setting control how many items will be send to Algolia at the same time.
If not provided, the items will be send by group of `100`.
### Install
```
pip install scrapy_algolia_exporter
```
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapy_algolia_exporter-0.0.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 24c3a2dc0a0f113b7a1acc343ef42d9459e4d5ac6f0d37ccd9d3297187689984 |
|
MD5 | 36df52a9e5a0e1f9e1586201690ed3c1 |
|
BLAKE2b-256 | 24a10bcaab1a2f47373c889800668c45d20a44d2e4f3052943a56280791a2899 |
Close
Hashes for scrapy_algolia_exporter-0.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | adb67f67a4597a9c61432d94f567e1e54d0233f442b780cc9f4e5b33b611421b |
|
MD5 | 3d64775fc6c8f61f61be9ca93b858b81 |
|
BLAKE2b-256 | 1e64df1f436174bc0fce020a231655e6b08064a244241a283c4d25754d031739 |