Scrapy item exporter for the Algolia API
Project description
[![PyPI](https://img.shields.io/pypi/v/scrapy_algolia_exorter.svg)](https://pypi.python.org/pypi/scrapy_algolia_exporter) [![Build Status](https://travis-ci.org/clemfromspace/scrapy_algolia_exporter.svg?branch=master)](https://travis-ci.org/clemfromspace/scrapy_algolia_exporter)
## Scrapy Algolia Exporter
Directly populate a given **Algolia** index from a scrapy spider.
### Usage
Add a unique `ObjectID` key to your crawled items.
Add the mandatory scrapy settings to configure the **Algolia** API:
```
ALGOLIA_API_ID='my_algolia_api_id'
ALGOLIA_API_KEY='my_algolia_api_key'
ALGOLIA_INDEX_NAME='my_algolia_index_name'
```
Add the `AlgoliaItemPipeline` in the `ITEM_PIPELINES` scrapy setting:
```
ITEM_PIPELINES = {
'scrapy_algolia_exporter.pipelines.AlgoliaItemPipeline': 10
}
```
The `ALGOLIA_ITEM_BULK_NBR` setting control how many items will be send to Algolia at the same time.
If not provided, the items will be send by group of `100`.
### Install
```
pip install scrapy_algolia_exporter
```
## Scrapy Algolia Exporter
Directly populate a given **Algolia** index from a scrapy spider.
### Usage
Add a unique `ObjectID` key to your crawled items.
Add the mandatory scrapy settings to configure the **Algolia** API:
```
ALGOLIA_API_ID='my_algolia_api_id'
ALGOLIA_API_KEY='my_algolia_api_key'
ALGOLIA_INDEX_NAME='my_algolia_index_name'
```
Add the `AlgoliaItemPipeline` in the `ITEM_PIPELINES` scrapy setting:
```
ITEM_PIPELINES = {
'scrapy_algolia_exporter.pipelines.AlgoliaItemPipeline': 10
}
```
The `ALGOLIA_ITEM_BULK_NBR` setting control how many items will be send to Algolia at the same time.
If not provided, the items will be send by group of `100`.
### Install
```
pip install scrapy_algolia_exporter
```
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapy_algolia_exporter-0.0.1.dev1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7177aa5fb02294d8117a71a2fe746bfaf8cf4d8f7ca66fbdb0f2ecac337c93a1 |
|
MD5 | 514f8a869a38f84f06425db24d924cfd |
|
BLAKE2b-256 | bac2477929094ef7bac4e71da24537e48688d457323b99fbc63b600e871b9a68 |
Close
Hashes for scrapy_algolia_exporter-0.0.1.dev1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7d95a55d172a7ccc99275ff712e75950dea1d72a4cb3cbac2bb6841283afa342 |
|
MD5 | eda57252b095efab1e7764765035b68f |
|
BLAKE2b-256 | 6b9a0d2f24c5db6f7a60b1dd53635d9ef4fbc1b626b7a2f1097a3df5c7d44d6a |