Development kit for the Pit30M large scale localization dataset
Project description
Pit30M Development Kit
Overview
This is the Python software development kit for the Pit30M benchmark for large-scale global localization. The devkit is currently in a pre-release state and many features are coming soon!
Consider checking out the original paper. If you would like to, you could also follow some of the authors' social media, e.g., Andrei's in order to be among the first to hear of any updates!
Getting Started
The recommended way to interact with the dataset is with the pip
package, which you can install with:
pip install pit30m
The devkit lets you efficiently access data on the fly. Here is a "hello world" command which renders a demo video from a random log segment. Note that it assumes ffmpeg
is installed:
python -m pit30m.cli multicam_demo --out-dir .
To preview data more interactively, check out the tutorial notebook.
More tutorials coming soon.
Torch Data Loading
We provide basic log-based PyTorch dataloaders. Visual-localization-specific ones are coming soon. To see an
example on how to use one of these dataloaders, have a look at demo_dataloader
in torch/dataset.py
.
An example command:
python -m pit30m.torch.dataset --root-uri s3://pit30m/ --logs 00682fa6-2183-4a0d-dcfe-bc38c448090f,021286dc-5fe5-445f-e5fa-f875f2eb3c57,1c915eda-c18a-46d5-e1ec-e4f624605ff0 --num-workers 16 --batch-size 64
Features
- Framework-agnostic multiprocessing-safe log reader objects
- PyTorch dataloaders
In-progress
- More lightweight package with fewer dependencies.
- Very efficient native S3 support through AWS-authored PyTorch-optimized S3 DataPipes.
- Support for non-S3 data sources, for users who wish to copy the dataset, or parts of it, to their own storage.
- Tons of examples and tutorials. See
examples/
for more information.
Development
Package development, testing, and releasing is performed with poetry
. If you just want to use the pit30m
package, you don't need to care about this section; just have a look at "Getting Started" above!
- Install poetry
- Setup/update your
dev
virtual environments withpoetry install --with=dev
in the project root- If you encounter strange keyring/credential errors, you may need
PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring poetry install --with=dev
- If you encounter strange keyring/credential errors, you may need
- Develop away
- run commands like
poetry run python -m pit30m.cli
- run commands like
- Test with
poetry run pytest
- Remember to run
poetry install
after pulling and/or updating dependencies.
Note that in the pre-release time, torch
will be a "dev" dependency, since it's necessary for all tests to pass.
Publishing
- Configure poetry with a PyPI account which has access to edit the package. You need to make sure poetry is configured with your API key.
poetry publish --build
Citation
@inproceedings{martinez2020pit30m,
title={Pit30m: A benchmark for global localization in the age of self-driving cars},
author={Martinez, Julieta and Doubov, Sasha and Fan, Jack and B{\^a}rsan, Ioan Andrei and Wang, Shenlong and M{\'a}ttyus, Gell{\'e}rt and Urtasun, Raquel},
booktitle={{IROS}},
pages={4477--4484},
year={2020},
organization={IEEE}
}
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.