ASAM MDF measurement data file parser
Project description
asammdf is a fast parser and editor for ASAM (Associtation for Standardisation of Automation and Measuring Systems) MDF (Measurement Data Format) files.
asammdf supports MDF versions 2 (.dat), 3 (.mdf) and 4 (.mf4).
asammdf works on Python >= 3.6 (for Python 2.7, 3.4 and 3.5 see the 4.x.y releases)
Status
! | Travis CI | Appveyor | CoverAlls | Codacy | ReadTheDocs |
---|---|---|---|---|---|
master |
PyPI | conda-forge |
---|---|
Project goals
The main goals for this library are:
- to be faster than the other Python based mdf libraries
- to have clean and easy to understand code base
- to have minimal 3-rd party dependencies
Features
-
create new mdf files from scratch
-
append new channels
-
read unsorted MDF v3 and v4 files
-
read CAN and LIN bus logging files
-
extract CAN and LIN signals from anonymous bus logging measurements
-
filter a subset of channels from original mdf file
-
cut measurement to specified time interval
-
convert to different mdf version
-
export to HDF5, Matlab (v4, v5 and v7.3), CSV and parquet
-
merge multiple files sharing the same internal structure
-
read and save mdf version 4.10 files containing zipped data blocks
-
space optimizations for saved files (no duplicated blocks)
-
split large data blocks (configurable size) for mdf version 4
-
full support (read, append, save) for the following map types (multidimensional array channels):
-
mdf version 3 channels with CDBLOCK
-
mdf version 4 structure channel composition
-
mdf version 4 channel arrays with CNTemplate storage and one of the array types:
- 0 - array
- 1 - scaling axis
- 2 - look-up
-
-
add and extract attachments for mdf version 4
-
handle large files (for example merging two fileas, each with 14000 channels and 5GB size, on a RaspberryPi)
-
extract channel data, master channel and extra channel information as Signal objects for unified operations with v3 and v4 files
-
time domain operation using the Signal class
- Pandas data frames are good if all the channels have the same time based
- a measurement will usually have channels from different sources at different rates
- the Signal class facilitates operations with such channels
-
graphical interface to visualize channels and perform operations with the files
Major features not implemented (yet)
-
for version 3
- functionality related to sample reduction block: the samples reduction blocks are simply ignored
-
for version 4
- experiemental support for MDF v4.20 column oriented storage
- functionality related to sample reduction block: the samples reduction blocks are simply ignored
- handling of channel hierarchy: channel hierarchy is ignored
- full handling of bus logging measurements: currently only CAN and LIN bus logging are implemented with the ability to get signals defined in the attached CAN/LIN database (.arxml or .dbc). Signals can also be extracted from an anonymous bus logging measurement by providing a CAN or LIN database (.dbc or .arxml)
- handling of unfinished measurements (mdf 4): finalization is attempted when the file is loaded, however the not all the finalization steps are supported
- full support for remaining mdf 4 channel arrays types
- xml schema for MDBLOCK: most metadata stored in the comment blocks will not be available
- full handling of event blocks: events are transfered to the new files (in case of calling methods that return new MDF objects) but no new events can be created
- channels with default X axis: the defaukt X axis is ignored and the channel group's master channel is used
Usage
from asammdf import MDF
mdf = MDF('sample.mdf')
speed = mdf.get('WheelSpeed')
speed.plot()
important_signals = ['WheelSpeed', 'VehicleSpeed', 'VehicleAcceleration']
# get short measurement with a subset of channels from 10s to 12s
short = mdf.filter(important_signals).cut(start=10, stop=12)
# convert to version 4.10 and save to disk
short.convert('4.10').save('important signals.mf4')
# plot some channels from a huge file
efficient = MDF('huge.mf4')
for signal in efficient.select(['Sensor1', 'Voltage3']):
signal.plot()
Check the examples folder for extended usage demo, or the documentation http://asammdf.readthedocs.io/en/master/examples.html
https://canlogger.csselectronics.com/canedge-getting-started/log-file-tools/asammdf-api/
Documentation
http://asammdf.readthedocs.io/en/master
And a nicely written tutorial on the CSS Electronics site
Contributing & Support
Please have a look over the contributing guidelines
If you enjoy this library please consider making a donation to the numpy project.
Contributors
Thanks to all who contributed with commits to asammdf:
- Julien Grave JulienGrv
- Jed Frey jed-frey
- Mihai yahym
- Jack Weinstein jackjweinstein
- Isuru Fernando isuruf
- Felix Kohlgrüber fkohlgrueber
- Stanislav Frolov stanifrolov
- Thomas Kastl kasuteru
- venden venden
- Marat K. kopytjuk
- freakatzz freakatzz
- Martin Falch MartinF
- dxpke dxpke
- Nick James driftregion
- tobiasandorfer tobiasandorfer
Installation
asammdf is available on
- github: https://github.com/danielhrisca/asammdf/
- PyPI: https://pypi.org/project/asammdf/
- conda-forge: https://anaconda.org/conda-forge/asammdf
pip install asammdf
# for the GUI
pip install asammdf[gui]
# or for anaconda
conda install -c conda-forge asammdf
Dependencies
asammdf uses the following libraries
- numpy : the heart that makes all tick
- numexpr : for algebraic and rational channel conversions
- wheel : for installation in virtual environments
- pandas : for DataFrame export
- canmatrix : to handle CAN/LIN bus logging measurements
- natsort
- lxml : for canmatrix arxml support
- lz4 : to speed up the disk IO peformance
optional dependencies needed for exports
- h5py : for HDF5 export
- scipy : for Matlab v4 and v5 .mat export
- hdf5storage : for Matlab v7.3 .mat export
- fastparquet : for parquet export
other optional dependencies
- PyQt5 : for GUI tool
- pyqtgraph : for GUI tool and Signal plotting
- matplotlib : as fallback for Signal plotting
- cChardet : to detect non-standard unicode encodings
- chardet : to detect non-standard unicode encodings
Benchmarks
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for asammdf-6.0.1-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 30c4b407a51ad01147af4fe26697d243575d5b655bf4f01577571baac218be4e |
|
MD5 | 860b938ba9aca95fefa9926c143da305 |
|
BLAKE2b-256 | 5a5ba7ca051985d6d368d78b9678de90068b96f744976b0176fdeaff34e4cdb5 |
Hashes for asammdf-6.0.1-cp38-cp38-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1e7aa2658f67af306590e2160985a29b5bb8d59b715caa1bc0b9724fcf8ca928 |
|
MD5 | 2f8c5bfdc46cc700aa1c216d0a3b63d1 |
|
BLAKE2b-256 | 1e400b512028234d3be4e115995b5d95becc319bedcb565d1cce9b1e2e177078 |
Hashes for asammdf-6.0.1-cp38-cp38-macosx_10_15_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c6de73479c8a873aaf80c14149af3caf900af843e2ca8afcfafecf900eb48723 |
|
MD5 | 3a2c7495aede344ccba464cf27a2e1fb |
|
BLAKE2b-256 | 0e463b0c8d72bc5be63fa084cfbd54cbd26407ba223ae7f534e8825cf10fb824 |
Hashes for asammdf-6.0.1-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 41bbce0ca33638ff31a2c9d15fbfaa68faa587aa93f4e7171a269a8ab8b9021c |
|
MD5 | ec7927813dab499219b883698679e2be |
|
BLAKE2b-256 | 6ce97175f07293fbd31a869904b1dcb69fe84b43394082a0540fec624587cb83 |
Hashes for asammdf-6.0.1-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 712b301e0f9192e31bd907c20fc16598e38d6202e4df34f3462df853dd0437ba |
|
MD5 | 98af736c7ab7faff0b6a02e90a2a0dac |
|
BLAKE2b-256 | 304ad6dee0d672cdcc2407c7d37956d54332afedc71044da4cf6f6e1af916f40 |
Hashes for asammdf-6.0.1-cp37-cp37m-macosx_10_15_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ebac01fd37f4da5ddd50407fa4672d7f45e436beb3c591c68e1b27275fb4c72e |
|
MD5 | 8788642139930d0acf64f19711b70220 |
|
BLAKE2b-256 | 32f56f888ea9fa2424ebf8cc75eb6a6d72ad77e92d60cf38c43b138a32ff88bf |
Hashes for asammdf-6.0.1-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d0f6e55b1d605ffbfacc84426104a9b87401d31b014bf1bc8a4b180ceffcc493 |
|
MD5 | 7b212af7bff8a0ae78454e33c0cefc9c |
|
BLAKE2b-256 | 2d4edad39142c27c291abcc05bdde44f724e93cfcbc3b82dc942af3d387a824c |
Hashes for asammdf-6.0.1-cp36-cp36m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3a5a9a1c2ff1f0ef9f622f127e8bb862c9506a221210c899676d3557374ba5fa |
|
MD5 | 87d202696feabf54dc24561218a76e7b |
|
BLAKE2b-256 | f1ae7676eb7c82353c9d63bc7687c2a39bb7ae7ccaaffe2dfc8edee3bb96bb85 |
Hashes for asammdf-6.0.1-cp36-cp36m-macosx_10_15_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f4ee563d236c2d006b78a5af2228ae0b57b2dc43c4ca2fd6d834e9d000b17fff |
|
MD5 | e08760072a3f9da0d3e35c4d49b95631 |
|
BLAKE2b-256 | 80e3beeaa3576138a18f060da0452285255e1a9c63639d4b65e6f134a53ddedf |