kiox: A composable experience replay buffer library

kiox is a composable experience replay buffer library.

from kiox.kiox import Kiox
from kiox.transition_buffer import FIFOTransitionBuffer
from kiox.transition_factory import SimpleTransitionFactory

kiox = Kiox(FIFOTransitionBuffer(1000), SimpleTransitionFactory())

# collect experiences
kiox.collect(<obsrvation>, <action>, <reward>, <terminal>)

# sample batch
batch = kiox.sample(256)

key features

:zap: Composable experience replay buffer

kiox is composable and fully Pythonic library. You can add your own sampling algorithms and inject sampling-time logics (e.g. loading image data from disk just before sampling).

:beginner: User-friendly API

kiox provides user-friendly API so that you can instantly incorporate kiox with your RL algorithms.

:rocket: Distributed RL training support

kiox supports distributed RL training by using ProtocolBuffer and gRPC. Your custom modules will work without any code changes.

installation

kiox supports Linux, macOS and Windows.

$ pip install kiox

examples

Many extensive examples are available.

distributed training

In actor process:

from kiox.distributed.step_sender import StepSender
sender = StepSender("localhost", 8000, 1)
sender.collect(<obsrvation>, <action>, <reward>, <terminal>)

In trainer process:

# trainer process
from kiox.distributed.server import KioxServer

def transition_buffer_builder():
    return FIFOTransitionBuffer(1000)

def transition_factory_builder():
    return SimpleTransitionFactory()

# setup server
server = KioxServer(
    host="localhost",
    port=8000,
    observation_shape=(4,),
    action_shape=(1,),
    reward_shape=(1,),
    batch_size=8,
    transition_buffer_builder=transition_buffer_builder,
    transition_factory_builder=transition_factory_builder,
)
server.start()

# sample batch
batch = server.sample()

from offline data

# from offline data
from kiox.offline import create_simple_kiox_from_data
kiox = create_simple_kiox_from_data(
  observations=<observations>,
  actions=<actions>,
  rewards=<rewards>,
  terminals=<terminals>,
)

build

$ pip install grpcio-tools
$ scripts/build-protobuf
$ pip install -e .

contributions

Any kind of contribution to kiox would be highly appreciated! Please check the contribution guide.