Timestamp-based Kafka topic replayer
Project description
kafka_replayer is a library that helps consume time ranges of messages from Kafka topics. While the standard Kafka consumer API allows seeking to a specific offset and replaying from there, using offsets as the replay abstraction is cumbersome and potentially error-prone. This library does the translation from timestamps to offsets transparently.
This library is written in Python, and leverages kafka-python’s consumer to poll Kafka for messages.
Installing
$ pip install python-kafka-replayerUsing
import json
import kafka_replayer
des_fn = lambda x: json.loads(x) if x else None
replayer = kafka_replayer.KafkaReplayer('my-topic',
bootstrap_servers=['localhost:9092'],
key_deserializer=des_fn,
value_deserializer=des_fn)
# Replay all records between the start and end millis timestamps
for record in replayer.replay(1469467314341, 1469467907549):
print recordLicence
See LICENSE.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
kafka_replayer-1.0.1.tar.gz
(4.7 kB
view hashes)
