grepros

grep for ROS bag files and live topics.

Searches through ROS messages and matches any message field value by regular expression patterns or plain text, regardless of field type. Can also look for specific values in specific message fields only.

By default, matches are printed to console. Additionally, matches can be written to a bagfile, or published to live topics.

Supports both ROS1 and ROS2. ROS environment variables need to be set.

Grepping from or publishing to ROS1 live topics requires ROS master to be running.

Example usage

Search for "my text" in all bags under current directory and subdirectories:

grepros -r "my text"

Print 30 lines of the first message from each live ROS topic:

grepros ".*" --max-per-topic 1 --lines-per-message 30 --live

Find first message containing "future" (case-insensitive) in my.bag:

grepros future -I --max-count 1 --name my.bag

Find 10 messages, from geometry_msgs package, in "map" frame, from bags in current directory:

grepros frame_id=map --type geometry_msgs/* --max-count 10

Pipe all diagnostics messages with "CPU usage" from live ROS topics to my.bag:

grepros "CPU usage" --type *DiagnosticArray --no-console-output --write my.bag

Find messages with field "key" containing "0xA002", in topics ending with "diagnostics", in bags under "/tmp":

grepros key=0xA002 --topic *diagnostics --path /tmp

Find diagnostics_msgs messages in bags in current directory, containing "navigation" in fields "name" or "message", print only header stamp and values:

grepros --type diagnostic_msgs/* --select-field name message \
        --print-field header.stamp status.values -- navigation

Print first message from each lidar topic on host 1.2.3.4:

ROS_MASTER_URI=http://1.2.3.4::11311 \
grepros ".*" --live --topic *lidar* --max-per-topic 1

Patterns use Python regular expression syntax, message matches if all match. '*' wildcards in other arguments use simple globbing as zero or more characters, target matches if any value matches.

When publishing matches to live topics, the published topic name will default to /grepros/ORIGINALNAME. Topic prefix and suffix can be changed via command-line parameter, or topic set to a single output topic name.

Command-line arguments

positional arguments:
  PATTERN               pattern(s) to find in message field values,
                        can specify message field as NAME=PATTERN
                        (name may be a nested.path)

optional arguments:
  -h, --help            show this help message and exit
  -F, --fixed-strings   PATTERNs are ordinary strings, not regular expressions
  -I, --no-ignore-case  use case-sensitive matching in PATTERNs
  -v, --invert-match    select non-matching messages
  --version             display version information and exit
  --live                read messages from live ROS topics instead of bagfiles
  --publish             publish matched messages to live ROS topics
  --write OUTFILE       write matched messages to specified output file
  --write-format {bag,html}
                        output format (default "bag"),
                        appended to if bag already exists

Filtering:
  -t TOPIC [TOPIC ...], --topic TOPIC [TOPIC ...]
                        ROS topics to scan if not all (supports * wildcards)
  -nt TOPIC [TOPIC ...], --no-topic TOPIC [TOPIC ...]
                        ROS topics to skip (supports * wildcards)
  -d TYPE [TYPE ...], --type TYPE [TYPE ...]
                        ROS message types to scan if not all (supports * wildcards)
  -nd TYPE [TYPE ...], --no-type TYPE [TYPE ...]
                        ROS message types to skip (supports * wildcards)
  -t0 TIME, --start-time TIME
                        earliest timestamp of messages to scan
                        as relative seconds if signed,
                        or epoch timestamp or ISO datetime
                        (for bag input, relative to bag start time
                        if positive or end time if negative,
                        for live input relative to system time,
                        datetime may be partial like 2021-10-14T12)
  -t1 TIME, --end-time TIME
                        latest timestamp of messages to scan
                        as relative seconds if signed,
                        or epoch timestamp or ISO datetime
                        (for bag input, relative to bag start time
                        if positive or end time if negative,
                        for live input relative to system time,
                        datetime may be partial like 2021-10-14T12)
  -n0 INDEX, --start-index INDEX
                        message index within topic to start from
                        (1-based if positive, counts back from bag total if negative)
  -n1 INDEX, --end-index INDEX
                        message index within topic to stop at
                        (1-based if positive, counts back from bag total if negative)
  -sf [FIELD [FIELD ...]], --select-field [FIELD [FIELD ...]]
                        message fields to use in matching if not all
                        (supports nested.paths and * wildcards)
  -ns [FIELD [FIELD ...]], --no-select-field [FIELD [FIELD ...]]
                        message fields to skip in matching
                        (supports nested.paths and * wildcards)
  -m NUM, --max-count NUM
                        number of matched messages to emit (per file if bag input)
  --max-per-topic NUM   number of matched messages to emit from each topic
                        (per file if bag input)
  --max-topics NUM      number of topics to emit matches from
  --unique-only         only emit matches that are unique in topic,
                        taking --select-field and --no-select-field into account
                        (per file if bag input)

Output control:
  -pf [FIELD [FIELD ...]], --print-field [FIELD [FIELD ...]]
                        message fields to print in console output if not all
                        (supports nested.paths and * wildcards)
  -np [FIELD [FIELD ...]], --no-print-field [FIELD [FIELD ...]]
                        message fields to skip in console output
                        (supports nested.paths and * wildcards)
  -B NUM, --before-context NUM
                        emit NUM messages of leading context before match
  -A NUM, --after-context NUM
                        emit NUM messages of trailing context after match
  -C NUM, --context NUM
                        emit NUM messages of leading and trailing context
                        around match
  -mo, --matched-fields-only
                        print only the fields where PATTERNs find a match
  -la NUM, --lines-around-match NUM
                        print only matched fields and NUM message lines
                        around match
  -lf NUM, --lines-per-field NUM
                        maximum number of lines to print per field
  -l0 NUM, --start-line NUM
                        message line number to start printing from
                        (1-based if positive, counts back from total if negative)
  -l1 NUM, --end-line NUM
                        message line number to stop printing at
                        (1-based if positive, counts back from total if negative)
  -lm NUM, --lines-per-message NUM
                        maximum number of lines to print per message
  --match-wrapper [STR [STR ...]]
                        string to wrap around matched values,
                        both sides if one value, start and end if more than one,
                        or no wrapping if zero values
                        (default "**" in colorless output)
  --wrap-width NUM
                        width to wrap console output at, 0 disables wrapping
                        (defaults to detected terminal width)
  --write-format-template OUTFILE_TEMPLATE
                        path to custom template to use for HTML output
  --color {auto,always,never}
                        use color output in console (default "always")
  --no-meta             do not print bag and topic and message metainfo to console
  --no-filename         do not print bag filename prefix on each console message line
  --no-console-output   do not print matches to console
  --verbose             print status messages during console output
                        for publishing and bag writing

Bag input control:
  -n [FILE [FILE ...]], --filename [FILE [FILE ...]]
                        names of ROS bagfiles to scan if not all in directory
                        (supports * wildcards)
  -p [PATH [PATH ...]], --path [PATH [PATH ...]]
                        paths to scan if not current directory
                        (supports * wildcards)
  -r, --recursive       recurse into subdirectories when looking for bagfiles
  --order-bag-by {topic,type}
                        order bag messages by topic or type first and then by time

Live topic control:
  --publish-prefix PREFIX
                        prefix to prepend to input topic name on publishing match
                        (default "/grepros")
  --publish-suffix SUFFIX
                        suffix to append to input topic name on publishing match
  --publish-fixname TOPIC
                        single output topic name to publish all matches to,
                        overrides prefix and suffix
  --queue-size-in SIZE  live ROS topic subscriber queue size (default 10)
  --queue-size-out SIZE
                        output publisher queue size (default 10)
  --ros-time-in         use ROS time instead of system time for incoming message
                        timestamps from subsribed live ROS topics

Installation

Using pip

pip install grepros

This will add the grepros command to your path.

Requires ROS Python packages (ROS1: rospy, roslib, rosbag, genpy; ROS2: rclpy, rosidl_runtime_py).

If you don't want to install the ROS1 stack, and are only interested in using bag files, not grepping from or publishing to live topics, minimal ROS1 Python packages can also be installed separately with:

pip install rospy rosbag roslib roslz4 \
--extra-index-url https://rospypi.github.io/simple/

Using catkin

In your ROS1 workspace, under the source directory:

git clone https://github.com/suurjaak/grepros.git
cd grepros
catkin build --this

This will add the grepros command to your local ROS1 workspace path.

Using colcon

In your ROS2 workspace, at the workspace root:

git clone https://github.com/suurjaak/grepros.git src/grepros
colcon build --packages-select grepros

This will add the grepros command to your local ROS2 workspace path.

Attribution

Includes a modified version of step, Simple Template Engine for Python, (c) 2012, Daniele Mazzocchio, https://github.com/dotpy/step, released under the MIT license.

License

Copyright (c) by Erki Suurjaak. Released as free open source software under the BSD License, see LICENSE.md for full details.