ASAM MDF measurement data file parser
Project description
asammdf is a fast parser and editor for ASAM (Associtation for Standardisation of Automation and Measuring Systems) MDF (Measurement Data Format) files.
asammdf supports MDF versions 2 (.dat), 3 (.mdf) and 4 (.mf4).
asammdf works on Python >= 3.6 (for Python 2.7, 3.4 and 3.5 see the 4.x.y releases)
Status
! | Travis CI | Appveyor | CoverAlls | Codacy | ReadTheDocs |
---|---|---|---|---|---|
master |
PyPI | conda-forge |
---|---|
Project goals
The main goals for this library are:
- to be faster than the other Python based mdf libraries
- to have clean and easy to understand code base
- to have minimal 3-rd party dependencies
Features
-
create new mdf files from scratch
-
append new channels
-
read unsorted MDF v3 and v4 files
-
read CAN and LIN bus logging files
-
extract CAN and LIN signals from anonymous bus logging measurements
-
filter a subset of channels from original mdf file
-
cut measurement to specified time interval
-
convert to different mdf version
-
export to HDF5, Matlab (v4, v5 and v7.3), CSV and parquet
-
merge multiple files sharing the same internal structure
-
read and save mdf version 4.10 files containing zipped data blocks
-
space optimizations for saved files (no duplicated blocks)
-
split large data blocks (configurable size) for mdf version 4
-
full support (read, append, save) for the following map types (multidimensional array channels):
-
mdf version 3 channels with CDBLOCK
-
mdf version 4 structure channel composition
-
mdf version 4 channel arrays with CNTemplate storage and one of the array types:
- 0 - array
- 1 - scaling axis
- 2 - look-up
-
-
add and extract attachments for mdf version 4
-
handle large files (for example merging two fileas, each with 14000 channels and 5GB size, on a RaspberryPi)
-
extract channel data, master channel and extra channel information as Signal objects for unified operations with v3 and v4 files
-
time domain operation using the Signal class
- Pandas data frames are good if all the channels have the same time based
- a measurement will usually have channels from different sources at different rates
- the Signal class facilitates operations with such channels
-
graphical interface to visualize channels and perform operations with the files
Major features not implemented (yet)
-
for version 3
- functionality related to sample reduction block: the samples reduction blocks are simply ignored
-
for version 4
- experiemental support for MDF v4.20 column oriented storage
- functionality related to sample reduction block: the samples reduction blocks are simply ignored
- handling of channel hierarchy: channel hierarchy is ignored
- full handling of bus logging measurements: currently only CAN and LIN bus logging are implemented with the ability to get signals defined in the attached CAN/LIN database (.arxml or .dbc). Signals can also be extracted from an anonymous bus logging measurement by providing a CAN or LIN database (.dbc or .arxml)
- handling of unfinished measurements (mdf 4): finalization is attempted when the file is loaded, however the not all the finalization steps are supported
- full support for remaining mdf 4 channel arrays types
- xml schema for MDBLOCK: most metadata stored in the comment blocks will not be available
- full handling of event blocks: events are transfered to the new files (in case of calling methods that return new MDF objects) but no new events can be created
- channels with default X axis: the defaukt X axis is ignored and the channel group's master channel is used
- attachment encryption/decryption using user provided encryption/decryption functions; this is not part of the MDF v4 spec and is only supported by this library
Usage
from asammdf import MDF
mdf = MDF('sample.mdf')
speed = mdf.get('WheelSpeed')
speed.plot()
important_signals = ['WheelSpeed', 'VehicleSpeed', 'VehicleAcceleration']
# get short measurement with a subset of channels from 10s to 12s
short = mdf.filter(important_signals).cut(start=10, stop=12)
# convert to version 4.10 and save to disk
short.convert('4.10').save('important signals.mf4')
# plot some channels from a huge file
efficient = MDF('huge.mf4')
for signal in efficient.select(['Sensor1', 'Voltage3']):
signal.plot()
Check the examples folder for extended usage demo, or the documentation http://asammdf.readthedocs.io/en/master/examples.html
https://canlogger.csselectronics.com/canedge-getting-started/log-file-tools/asammdf-api/
Documentation
http://asammdf.readthedocs.io/en/master
And a nicely written tutorial on the CSS Electronics site
Contributing & Support
Please have a look over the contributing guidelines
If you enjoy this library please consider making a donation to the numpy project or to danielhrisca using liberapay <a href="https://liberapay.com/danielhrisca/donate"><img alt="Donate using Liberapay" src="https://liberapay.com/assets/widgets/donate.svg"></a>
Contributors
Thanks to all who contributed with commits to asammdf:
- Julien Grave JulienGrv
- Jed Frey jed-frey
- Mihai yahym
- Jack Weinstein jackjweinstein
- Isuru Fernando isuruf
- Felix Kohlgrüber fkohlgrueber
- Stanislav Frolov stanifrolov
- Thomas Kastl kasuteru
- venden venden
- Marat K. kopytjuk
- freakatzz freakatzz
- Martin Falch MartinF
- dxpke dxpke
- Nick James driftregion
- tobiasandorfer tobiasandorfer
Installation
asammdf is available on
- github: https://github.com/danielhrisca/asammdf/
- PyPI: https://pypi.org/project/asammdf/
- conda-forge: https://anaconda.org/conda-forge/asammdf
pip install asammdf
# for the GUI
pip install asammdf[gui]
# or for anaconda
conda install -c conda-forge asammdf
In case a wheel is not present for you OS/Python versions and you lack the proper compiler setup to compile the c-extension code, then you can simply copy-paste the pacakge code to your site-packages. In this way the python fallback code will be used instead of the compiled c-extension code.
Dependencies
asammdf uses the following libraries
- numpy : the heart that makes all tick
- numexpr : for algebraic and rational channel conversions
- wheel : for installation in virtual environments
- pandas : for DataFrame export
- canmatrix : to handle CAN/LIN bus logging measurements
- natsort
- lxml : for canmatrix arxml support
- lz4 : to speed up the disk IO peformance
optional dependencies needed for exports
- h5py : for HDF5 export
- scipy : for Matlab v4 and v5 .mat export
- hdf5storage : for Matlab v7.3 .mat export
- fastparquet : for parquet export
other optional dependencies
- PyQt5 : for GUI tool
- pyqtgraph : for GUI tool and Signal plotting
- matplotlib : as fallback for Signal plotting
- cChardet : to detect non-standard unicode encodings
- chardet : to detect non-standard unicode encodings
- pyqtlet : for GPS window
Benchmarks
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for asammdf-6.4.2.dev4-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 79435e999596caf84810020ce8e49cebc282d2c7d1d1643b5bf61e26581ea6f6 |
|
MD5 | 4ef23c5159803de1574fc2ab0b3c0db7 |
|
BLAKE2b-256 | 74399d98b69bb8fbdfce8daeede24c4a550a30f9ffeababb6af60902aa134077 |
Hashes for asammdf-6.4.2.dev4-cp39-cp39-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f7a30628e5d88a3fa63bfbc8d18c23a02f5c58fef3893c6c069d302e8cd3d23f |
|
MD5 | 7e49d72c9da071b0a03b49f6572b1030 |
|
BLAKE2b-256 | f8454ce8032ab86a0baa0fde12fbf3e743e6269aef3fa04b1d53b56106fbe0b6 |
Hashes for asammdf-6.4.2.dev4-cp39-cp39-macosx_10_15_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 459783ac919925c80109646d3d9d843a39b8be7eb2255f1000235868160bc372 |
|
MD5 | 5448e33804f3d483a2ca07850c202886 |
|
BLAKE2b-256 | 00169adb128b4d05535a9385afc20123d06eaa4ba6259f21ce3a960e96d54d0e |
Hashes for asammdf-6.4.2.dev4-cp39-cp39-macosx_10_14_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1aa42e2b0be8b85629ebff71b767bf59aa2a090262776926c1734a8f025188db |
|
MD5 | d34584f4c54631526e235fe48740fdc6 |
|
BLAKE2b-256 | 8cf087ca4222db141ebff8e1820fd0ce62a059727a211bbb206e80f9c7a5abd3 |
Hashes for asammdf-6.4.2.dev4-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1193184581489cf3aeea2174d190435da36e05aac18bca5aa54bc698c4ed66c9 |
|
MD5 | d26b667f4f98f1e25342a25d7c804ad7 |
|
BLAKE2b-256 | 2601d14aee33a2ccf4ae309473c7bb548c2f9802ce9842955ab8b31c80524b97 |
Hashes for asammdf-6.4.2.dev4-cp38-cp38-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fccced052de9dd2976330b4ffef2e1016c7ea1e44a752ff74c7b797e86ec2af7 |
|
MD5 | ae54fb5b7bb0e8dc6e00c021e2bec728 |
|
BLAKE2b-256 | 355fbaf0e3495db398a80ca8c7f57b4bf9ec637f6ba633be7a4a8b054dde976e |
Hashes for asammdf-6.4.2.dev4-cp38-cp38-macosx_10_15_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 81069aa809cb74a591e331923a18f6afc4229d5d3facacba4018212aed9163b0 |
|
MD5 | 1d3b0348d06666a5beece7cc8ca280bf |
|
BLAKE2b-256 | af6b723a5c90fcfae337ffc1e757711fced751da3bd2632a84445498981ef557 |
Hashes for asammdf-6.4.2.dev4-cp38-cp38-macosx_10_14_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4bacf6ab5cc9cc746dbd81b1f435d20d0df46babdfddbab8c5058b834f745a20 |
|
MD5 | 4d0f9b902277d7e6dfb11fadcda7a707 |
|
BLAKE2b-256 | 74b193a36e4a88ce79052ca6bb1e5b36dea86f6914ab823d3bd1513438279e54 |
Hashes for asammdf-6.4.2.dev4-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f37766636711895ca50ff9c1ba86a40d793613b76d4893200862e89f2901ff25 |
|
MD5 | afb1a17304e90c2c64a51ec36de8b51a |
|
BLAKE2b-256 | 050bdd604de49fb1b8b2679976b11565879d036cdff92269a8a710d21ccac47d |
Hashes for asammdf-6.4.2.dev4-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 154b29662dcf5038ae716a6c054f0707c196757ad07f53439cf21deb4683096e |
|
MD5 | 5bcddc704fa3bb552831e2a46c419468 |
|
BLAKE2b-256 | 9f844def0b220f2e95c15cf60587799d3aaccec212c6937e8d52186db92a3c79 |
Hashes for asammdf-6.4.2.dev4-cp37-cp37m-macosx_10_15_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f804312d3417ce5afc1a6878e93b88f81995b7cc95f3502281f7be3705a7aa24 |
|
MD5 | c842c88201acd6821b8eee8906c44f60 |
|
BLAKE2b-256 | 15afacebcfc2b03b5285e5ba10f3c4fcf6e633426edfe988fe157a9c80c7337d |
Hashes for asammdf-6.4.2.dev4-cp37-cp37m-macosx_10_14_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cacfe7d0c444bb06c1b95e5512ed33cf12f52fbbe44d447ece237549d2b6bb4c |
|
MD5 | 1851cbb7216900c20c67ddf5b4deb542 |
|
BLAKE2b-256 | 6414686ec8bad2821b8afdef369cd8fe0ef87546065f579ec6d78ae24f1fa919 |
Hashes for asammdf-6.4.2.dev4-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 570a02dd30b639dccee8e37e2571118af0ff5e78139120b9d91357e53b257608 |
|
MD5 | 7882bc5b18eb456023a3cbdb404f571d |
|
BLAKE2b-256 | b28708954faadad48a7c9bdb0d43c79f791e09cb9bdd640ebbb261401d55096d |
Hashes for asammdf-6.4.2.dev4-cp36-cp36m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d675308bf53d284d35db81600cc915f2ecea2a5fca94c43382c331ff15aa33f8 |
|
MD5 | 0a3502d48e1cea6b4de12664d2adfb4a |
|
BLAKE2b-256 | 33c403cb2cf2c4663497f7efe8c9a3537525420a783885bc7262b3875e3b4dc5 |
Hashes for asammdf-6.4.2.dev4-cp36-cp36m-macosx_10_15_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 15a541737b96ad51792567245657b046c3b4266eacb7fd6ff582f89206bd175c |
|
MD5 | c8983a15ce66dd81e59acc087b2d462e |
|
BLAKE2b-256 | df31346c78d299933edcac783d857790548062ac767c64e271bfa4a8ab3a5919 |
Hashes for asammdf-6.4.2.dev4-cp36-cp36m-macosx_10_14_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f9cfffdbd09f1620b8c47a36c6ca9ad44ceb3bd6d27285936378545919e1ff3e |
|
MD5 | 5340c0190f285c6fcbb9d4c8b6ad6351 |
|
BLAKE2b-256 | d72d127febafc86b177814d33672141610141eb90a870dce433ad2182a02a203 |