AEStream - Address Event streaming library

AEStream efficiently reads sparse events from an input source and streams it to an output sink. AEStream supports reading from files, USB cameras, as well as network via UDP and can stream events to files, network over UDP, and peripherals such as GPUs and neuromorphic hardware.

Read more in the AEStream publication.

Installation

AEStream is usable both as a command-line binary or Python tool.

Contributions to support AEStream on additional platforms are always welcome.

Usage: read event-address files in Python

AEStream can process fixed input sources like files like so:

FileInput("file", (640, 480)).load()

Usage: stream real-time data in Python

AEStream also supports streaming data in real-time without strict guarantees on orders. This is particularly useful in real-time scenarios, for instance when operating with USBInput or UDPInput

# Stream events from a DVS camera over USB
with USBInput((640, 480)) as stream:
    while True:
        frame = stream.read() # Provides a (640, 480) tensor
        ...

# Stream events from UDP port 3333 (default)
with UDPInput((640, 480), port=3333) as stream:
    while True:
        frame = stream.read() # Provides a (640, 480) tensor
        ...

More examples can be found in our example folder. Please note the examples may require additional dependencies (such as Norse for spiking networks or PySDL for rendering). To install all the requirements, simply stand in the aestream root directory and run pip install -r example/requirements.txt

Example: real-time edge detection with spiking neural networks

We stream events from a camera connected via USB and process them on a GPU in real-time using the spiking neural network library, Norse using fewer than 50 lines of Python. The left panel in the video shows the raw signal, while the middle and right panels show horizontal and vertical edge detection respectively. The full example can be found in example/usb_edgedetection.py

Usage (CLI)

Installing AEStream also gives access to the command-line interface (CLI) aestream. To use aestraem, simply provide an input source and an optional output sink (defaulting to STDOUT):

aestream input <input source> [output <output sink>]

Supported Inputs and Outputs

CLI examples

Custom installation (C++)

Metavision SDK and libcaer are optional dependencies, but are needed for connecting to Prophesee and Inivation cameras respectively.

AEStream is based on C++20. Since C++20 is not yet fully supported by all compilers, we recommend using GCC >= 10.2.

To build the binaries of this repository, run the following code:

# Optional: Ensure paths to libcaer or Metavision are in place
mkdir build/
cd build/
cmake -GNinja ..
ninja

If your default C++ compiler doesn't support C++ 20, you will have to install an up-to-date compiler and provide the environmental variable CXX. For instance like this: CXX=/path/to/g++-10 cmake -GNinja ..

Inivation cameras

For Inivation cameras, the libcaer library needs to be available, either by a -DCMAKE_PREFIX_PATH flag to cmake or included in the PATH environmental variable. For examble: cmake -GNinja -DCMAKE_PREFIX_PATH=/path/to/libcaer. Inivation made the library available for most operating systems, but you may have to build it yourself.

Prophesee cameras

For Prophesee cameras, a version of the Metavision SDK needs to be present. The open-source version the SDK openeb is available with installation instructions at https://github.com/prophesee-ai/openeb. Using openeb, it should be sufficient to install it using cmake && make && make install to put it in your path. Otherwise, you can point to it using the -DCMAKE_PREFIX_PATH option in cmake.

Acknowledgments

AEStream is developed by (in alphabetical order):

• Cameron Barker (@GitHub cameron-git)
• Alexander Hadjivanov (@Github cantordust)
• Jens E. Pedersen (@GitHub jegp)
• Christian Pehle (@GitHub cpehle)

The work has received funding from the EC Horizon 2020 Framework Programme under Grant Agreements 785907 and 945539 (HBP) and by the Deutsche Forschungsgemeinschaft (DFG, German Research Fundation) under Germany's Excellence Strategy EXC 2181/1 - 390900948 (the Heidelberg STRUCTURES Excellence Cluster).

Thanks to Philipp Mondorf for interfacing with Metavision SDK and preliminary network code.

Citation

Please cite aestream if you use it in your work:

@misc{aestream,
  doi = {10.48550/ARXIV.2212.10719},
  url = {https://arxiv.org/abs/2212.10719},
  author = {Pedersen, Jens Egholm and Conradt, Jörg},
  title = {AEStream: Accelerated event-based processing with coroutines},
  publisher = {arXiv},
  year = {2022},
}