ASAM MDF measurement data file parser
Project description
asammdf is a fast parser and editor for ASAM (Associtation for Standardisation of Automation and Measuring Systems) MDF (Measurement Data Format) files.
asammdf supports MDF versions 2 (.dat), 3 (.mdf) and 4 (.mf4).
asammdf works on Python >= 3.6 (for Python 2.7, 3.4 and 3.5 see the 4.x.y releases)
Status
! | Travis CI | Appveyor | CoverAlls | Codacy | ReadTheDocs |
---|---|---|---|---|---|
master |
PyPI | conda-forge |
---|---|
Project goals
The main goals for this library are:
- to be faster than the other Python based mdf libraries
- to have clean and easy to understand code base
- to have minimal 3-rd party dependencies
Features
-
create new mdf files from scratch
-
append new channels
-
read unsorted MDF v3 and v4 files
-
read CAN and LIN bus logging files
-
extract CAN and LIN signals from anonymous bus logging measurements
-
filter a subset of channels from original mdf file
-
cut measurement to specified time interval
-
convert to different mdf version
-
export to HDF5, Matlab (v4, v5 and v7.3), CSV and parquet
-
merge multiple files sharing the same internal structure
-
read and save mdf version 4.10 files containing zipped data blocks
-
space optimizations for saved files (no duplicated blocks)
-
split large data blocks (configurable size) for mdf version 4
-
full support (read, append, save) for the following map types (multidimensional array channels):
-
mdf version 3 channels with CDBLOCK
-
mdf version 4 structure channel composition
-
mdf version 4 channel arrays with CNTemplate storage and one of the array types:
- 0 - array
- 1 - scaling axis
- 2 - look-up
-
-
add and extract attachments for mdf version 4
-
handle large files (for example merging two fileas, each with 14000 channels and 5GB size, on a RaspberryPi)
-
extract channel data, master channel and extra channel information as Signal objects for unified operations with v3 and v4 files
-
time domain operation using the Signal class
- Pandas data frames are good if all the channels have the same time based
- a measurement will usually have channels from different sources at different rates
- the Signal class facilitates operations with such channels
-
graphical interface to visualize channels and perform operations with the files
Major features not implemented (yet)
-
for version 3
- functionality related to sample reduction block: the samples reduction blocks are simply ignored
-
for version 4
- experiemental support for MDF v4.20 column oriented storage
- functionality related to sample reduction block: the samples reduction blocks are simply ignored
- handling of channel hierarchy: channel hierarchy is ignored
- full handling of bus logging measurements: currently only CAN and LIN bus logging are implemented with the ability to get signals defined in the attached CAN/LIN database (.arxml or .dbc). Signals can also be extracted from an anonymous bus logging measurement by providing a CAN or LIN database (.dbc or .arxml)
- handling of unfinished measurements (mdf 4): finalization is attempted when the file is loaded, however the not all the finalization steps are supported
- full support for remaining mdf 4 channel arrays types
- xml schema for MDBLOCK: most metadata stored in the comment blocks will not be available
- full handling of event blocks: events are transfered to the new files (in case of calling methods that return new MDF objects) but no new events can be created
- channels with default X axis: the defaukt X axis is ignored and the channel group's master channel is used
Usage
from asammdf import MDF
mdf = MDF('sample.mdf')
speed = mdf.get('WheelSpeed')
speed.plot()
important_signals = ['WheelSpeed', 'VehicleSpeed', 'VehicleAcceleration']
# get short measurement with a subset of channels from 10s to 12s
short = mdf.filter(important_signals).cut(start=10, stop=12)
# convert to version 4.10 and save to disk
short.convert('4.10').save('important signals.mf4')
# plot some channels from a huge file
efficient = MDF('huge.mf4')
for signal in efficient.select(['Sensor1', 'Voltage3']):
signal.plot()
Check the examples folder for extended usage demo, or the documentation http://asammdf.readthedocs.io/en/master/examples.html
https://canlogger.csselectronics.com/canedge-getting-started/log-file-tools/asammdf-api/
Documentation
http://asammdf.readthedocs.io/en/master
And a nicely written tutorial on the CSS Electronics site
Contributing & Support
Please have a look over the contributing guidelines
If you enjoy this library please consider making a donation to the numpy project or to danielhrisca using liberapay <a href="https://liberapay.com/danielhrisca/donate"><img alt="Donate using Liberapay" src="https://liberapay.com/assets/widgets/donate.svg"></a>
Contributors
Thanks to all who contributed with commits to asammdf:
- Julien Grave JulienGrv
- Jed Frey jed-frey
- Mihai yahym
- Jack Weinstein jackjweinstein
- Isuru Fernando isuruf
- Felix Kohlgrüber fkohlgrueber
- Stanislav Frolov stanifrolov
- Thomas Kastl kasuteru
- venden venden
- Marat K. kopytjuk
- freakatzz freakatzz
- Martin Falch MartinF
- dxpke dxpke
- Nick James driftregion
- tobiasandorfer tobiasandorfer
Installation
asammdf is available on
- github: https://github.com/danielhrisca/asammdf/
- PyPI: https://pypi.org/project/asammdf/
- conda-forge: https://anaconda.org/conda-forge/asammdf
pip install asammdf
# for the GUI
pip install asammdf[gui]
# or for anaconda
conda install -c conda-forge asammdf
In case a wheel is not present for you OS/Python versions and you lack the proper compiler setup to compile the c-extension code, then you can simply copy-paste the pacakge code to your site-packages. In this way the python fallback code will be used instead of the compiled c-extension code.
Dependencies
asammdf uses the following libraries
- numpy : the heart that makes all tick
- numexpr : for algebraic and rational channel conversions
- wheel : for installation in virtual environments
- pandas : for DataFrame export
- canmatrix : to handle CAN/LIN bus logging measurements
- natsort
- lxml : for canmatrix arxml support
- lz4 : to speed up the disk IO peformance
optional dependencies needed for exports
- h5py : for HDF5 export
- scipy : for Matlab v4 and v5 .mat export
- hdf5storage : for Matlab v7.3 .mat export
- fastparquet : for parquet export
other optional dependencies
- PyQt5 : for GUI tool
- pyqtgraph : for GUI tool and Signal plotting
- matplotlib : as fallback for Signal plotting
- cChardet : to detect non-standard unicode encodings
- chardet : to detect non-standard unicode encodings
Benchmarks
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for asammdf-6.1.2-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 75211f5766fa4b4b05d326cf10638d5cfd358199019d362cde26d88cabec2d38 |
|
MD5 | b73b542afe0907b343423467fceceb82 |
|
BLAKE2b-256 | 9918d20476eb9861f38562ae8c31f17efa72adbe32ec3c7eed08439e5e1cad76 |
Hashes for asammdf-6.1.2-cp39-cp39-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e47ee9ff558cb22dfe5ad162bf7760705ebbbf12837fc4804933b48739e9446c |
|
MD5 | 7e03027c5fe713c809af24bdf0713ba2 |
|
BLAKE2b-256 | 266f61c9e223487aa6bb03ecdf359dcca01c953b0c8807dd3f927378e6d946fb |
Hashes for asammdf-6.1.2-cp39-cp39-macosx_10_15_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bd66295b862bcccb3bb95035de13c056f7e9a0cad09106f69d5085dcb4aeaa5b |
|
MD5 | c99644c6583c6e332139e4941f4a3377 |
|
BLAKE2b-256 | 740833e80afef37530988ea4bb0cd3ef0f21b56f0aa40113d08366e66289ba1d |
Hashes for asammdf-6.1.2-cp39-cp39-macosx_10_14_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4eafd1abd4026e896b292d0c1305fc97afe1c44036fb6e5bcc4c614a0b876041 |
|
MD5 | ed6b0a7764a314b4b5fb9948d5e19efd |
|
BLAKE2b-256 | e4494a41023f30ab22a6cbc39ebe2e9dcb1edc5bdf0b4204a3891af0a1cb9829 |
Hashes for asammdf-6.1.2-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dddc4f80fc194a62fbc364a5a6d71854b225977336e1e1032f75acf0a3b3a9c6 |
|
MD5 | 9807e090b44b3e95b16a6a77f8ef6a69 |
|
BLAKE2b-256 | f09de92f75cb8eb03a5aab2c598a0994d05ffbd98767ec2dc94a9a2bbb5f00e0 |
Hashes for asammdf-6.1.2-cp38-cp38-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b6899b9d6008f653e11225f05e61658bb20e0b3ad956488b16ef3888ba187693 |
|
MD5 | 43e29ff6817cd64fcbfb93082dc6eaca |
|
BLAKE2b-256 | 2f3f1cc95118e4da45e82c56ff2ea10142061da62337d65029481d8e696ea6ad |
Hashes for asammdf-6.1.2-cp38-cp38-macosx_10_15_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6405a24d70548c99edf106ac9f9899959ac76a27f34c18a22f6a661baa6253c3 |
|
MD5 | 67f6e81f76f21b874f27f7d280c276d8 |
|
BLAKE2b-256 | 7a3835cdad40ed7c7a5fe67e63e2ebe746067296ad1ea59c4926f7e19c540791 |
Hashes for asammdf-6.1.2-cp38-cp38-macosx_10_14_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4d631b346c758ef4b01a5b8812a0f26a0da82cda5a70c9667eda37d344106d87 |
|
MD5 | cad15bc14f5f09d5328d318e5576e2aa |
|
BLAKE2b-256 | e46c8988f8e60b18aface4f8bd03fff0345d1933a7d885dd0404efed72e7a8ea |
Hashes for asammdf-6.1.2-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c7fdd86fa0cb7ad09710e6c116c124148d6c29091792c890396434b5ab74860a |
|
MD5 | e4bf9bc7001b190637ffb47936748b40 |
|
BLAKE2b-256 | 5891b6e459e0f9dd6524eec108a24ec01e543543a47ec913439aa21e7add1cc3 |
Hashes for asammdf-6.1.2-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e9293df7062d7f08a7db8a2e3d066e04b3e036fa1e3986c5bc93869dbb0c1709 |
|
MD5 | 9d2fb6c151f5905f1ffd9f4b6963136c |
|
BLAKE2b-256 | ce45620c3e19c90f359f85453a19d33ae5c79cf3bebcde81ea8722a432c25471 |
Hashes for asammdf-6.1.2-cp37-cp37m-macosx_10_15_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1ef7d1a97a5899eca33590fdf58d616695550527174a5aeedd47b6d39142d9a6 |
|
MD5 | b37709788c4b5aab30eb3017fc055952 |
|
BLAKE2b-256 | f6e107f463999e36db1f09b485b27f820855347b870882cd53e083764a8fec2d |
Hashes for asammdf-6.1.2-cp37-cp37m-macosx_10_14_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0650c33ce23c2ad2eec4eee07da6cfa271541b2f53eee709a53793ffea8e6d57 |
|
MD5 | b5e1019f510a074f4e2301d06cae049a |
|
BLAKE2b-256 | c384a470d2b72de363950f09338c1bf81422a068cd50b4f851b465f6b5db01d5 |
Hashes for asammdf-6.1.2-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a83b7779d3f05498bbd84142f5c7585dca4222e9cf7182915796911299601231 |
|
MD5 | 72a9f509ed35d847409d7ff1aa598dbe |
|
BLAKE2b-256 | 1f143aa1ae70e7ed05ef963c39bd2c8acec4ad7d7cfffab77c027cbfcb05c640 |
Hashes for asammdf-6.1.2-cp36-cp36m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | aa4e8aac6a0ab2bdfa6f88a4efd4c9e2f1e55480bb42100b1bc3960f3bacbeab |
|
MD5 | 337b7cf3d76281f829962eeefff2ee16 |
|
BLAKE2b-256 | 6b618189f1031eb431f99b17a246e6279a52c26d5a12124bb65b73ab0c18e5c9 |
Hashes for asammdf-6.1.2-cp36-cp36m-macosx_10_15_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b7f15e9bd02c2cb93ba08689b01ce1ce4fd0727edba1bd18a4c7dfcbdf9bf8a5 |
|
MD5 | ee7a8f51bcc2752a3f424ab2461b5a81 |
|
BLAKE2b-256 | 0dbe467cedac3703b25280d35c5b9dba33b5ff5cb9afe48c82c259362ced2fd4 |
Hashes for asammdf-6.1.2-cp36-cp36m-macosx_10_14_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7a643e0460f3a471d4a372df224fa5d15df355f01723eb5e26c8aa48fb5add2b |
|
MD5 | 53ba3092d8731f003fd52a868816b262 |
|
BLAKE2b-256 | 56e30193a82a87ea4bb117a3d2185d29f01eb2d73b056364e39f5ec44d567650 |