Kappe

Kappe is an efficient data migration tool designed to seamlessly convert and split MCAP files.

Installation

pip install kappe

or

rye tools install kappe

Usage

Create a yaml config file containing the migrations you want to perform.

Example:

config.yaml

topic:
  mapping:
    /points: /sensor/points

Run the converter:

kappe convert --config config.yaml ./input.mcap

Convert

kappe convert [-h] [--config CONFIG] [--overwrite] input output

Converts a single file or a directory of files to the MCAP format.

Topic

Rename a topic

topic:
  mapping:
    /points: /sensor/points

Remove a topic

topic:
  remove:
    - /points

Topic Times

The time_offset config manipulates the mcap message time and/or the ROS header timestamp. When using default as topic name, the config will be applied to all messages.

Update the ROS header time

Adds 8 second and 300 nanosec to the ROS header.

time_offset:
  /sensor/points:
    sec: 8
    nanosec: 300

Change ROS Timestamp to publish time

Change the time of the ROS Timestamp to the time the message was published.

time_offset:
  /sensor/points:
    pub_time: True

Change MCAP pub/log time from ROS header

Update the log/pub time from the ROS header. If pub_time is set, pub time will be used as source. If sec and/or nanosec is set, the offset is used.

time_offset:
  /sensor/points:
    update_publish_time: True
    update_log_time: True

Add a time offset to ROS Timestamp

Add 15 seconds to the ROS Timestamp.

time_offset:
  /sensor/points:
    sec: 15
    nanosec: 0

Pointcloud

Remove zero points from PointCloud2

point_cloud:
  /sensor/points:
    remove_zero: true

Rotate a pointcloud

point_cloud:
  /sensor/points:
    rotation:
      euler_deg:
        - 180
        - 0
        - 0

Rename PointCloud2 field name

Changes the field name of the PointCloud2 message, from AzimuthAngle to azimuth_angle.

point_cloud:
  /sensor/points:
    field_mapping:
      AzimuthAngle: azimuth_angle

TF

To update a static transform you need to remove the old one and insert a new one.

Remove Transform

Removes all transform where the child_frame_id is test_data_frame or other_frame.

tf_static:
  remove:
    - test_data_frame
    - other_frame

Insert Static Transform

Rotation can be specified in euler_deg or quaternion

tf_static:
  insert:
    - frame_id: base
      child_frame_id: also_base

    - frame_id: base
      child_frame_id: sensor
      translation:
        x: -0.1
        y: 0
        z: 0.1
      rotation:
        euler_deg:
          - 0
          - 90
          - 0

Schema Mapping

If the new schema is not already in the mcap, kappe will try to load it either from your ROS2 environment or from ./msgs.

msg_schema:
  mapping:
    std_msgs/Int32: std_msgs/Int64

Trim

Trim the mcap file to a specific time range.

time_start:  1676549454.0
time_end:    1676549554.0

Plugins

Kappe can be extended with plugins, for example to compress images. Source code for plugins can be found in the plugins, additional plugins can be loaded from ./plugins.

plugins:
  - name: image.CompressImage
    input_topic: /image
    output_topic: /compressed/image
    settings:
      quality: 50

ROS1 to ROS2 conversion

Kappe automatic converts ROS1 messages to ROS2 messages. It will not reuse ROS1 definitions, all schemas will be loaded either from your ROS2 environment or from ./msgs. If the ROS2 schema name has changed use the msg_schema.mapping to map the old schema to the new schema.

The msg field will be mapped exactly, ROS1 time and duration will be converted to ROS2 time and duration (secs -> sec & nsecs -> nanosec).

To download the common ROS2 schemas run:

git clone --depth=1 --branch=humble https://github.com/ros2/common_interfaces.git ./msgs/common_interfaces
git clone --depth=1 --branch=humble https://github.com/ros2/geometry2.git ./msgs/geometry2

Reproducibility

Kappe saves the input/output path, the time and the version into a MCAP metadata field, called convert_metadata. The config will be saved as an attachment named convert_config.yaml.

Cut

usage: kappe cut [-h] --config CONFIG [--overwrite] input [output_folder]

Cuts a directory of mcaps into smaller mcaps, based on timestamp or topic.

When keep_tf_tree is set to true all splits will have the same /tf_static messages.

Split on time

The start and end times define the range to extract into each split file. They are specified in seconds which is compared against the log time (UNIX Timestamp).

keep_tf_tree: true
splits:
  - start:  1676549454.0
    end:    1676549554.0
    name:   beginning.mcap
  - start:  1676549554.0
    end:    1676549654.0
    name:   end.mcap

kappe cut --config config.yaml ./input.mcap ./output_folder

Results in a folder with the following structure:

output_folder
├── beginning.mcap
└── end.mcap

Split on topic

Splits the mcap file into multiple files, every time a message on the topic /marker is read. The file will be split before the message is read. debounce is the time in seconds that the cutter will wait before splitting the file again, default is 0.

keep_tf_tree: true
split_on_topic:
  topic: "/marker"
  debounce: 10