TimeSide 0.5.6.3

Build status

• Branch master :

• Branch dev :

Goals

• Do asynchronous and fast audio processing with Python,

• Decode audio frames from any audio or video media format into numpy arrays,

• Analyze audio content with some state-of-the-art audio feature extraction libraries like Aubio, Yaafe and VAMP as well as some pure python processors

• Visualize sounds with various fancy waveforms, spectrograms and other cool graphers,

• Transcode audio data in various media formats and stream them through web apps,

• Organize, serialize and save feature analysis data through various portable formats,

• Playback and interact on demand through a smart high-level HTML5 extensible player,

• Index, tag and annotate audio archives with semantic metadata (see Telemeta which embed TimeSide).

Architecture

The streaming architecture of TimeSide relies on 2 main parts: a processing engine including various plugin processors written in pure Python and a user interface providing some web based visualization and playback tools in pure HTML5.

Processors

News

0.5.6

• Bugfix release

• Fix analyzer instanciation as parent for some graphers

0.5.5

• All processor folders (decoder, analyzer, grapher, encoder) are now real plugin repositories : you can just drop processors in it and play!

• TimeSide can be installed without Aubio, Yaafe nor Vamp : it should be easier to install on old distributions for which those librairies are difficult or impossible to compile

• Encoder : add an Opus encoder

• Experimental : add a django web server with a REST API (see “Web server”)

• AubioPitch: prevent NaN in result by converting them to zero

• Yaafe analyzer: simplify adaptation of process frames from TimeSide to Yaafe

• LimsiSad: add a default value for parameter sad_model

• Fix various NaN and Inf and PEP8 issues also many PyFlake warnings

• Full Travis integration with tests and test coverage through coveralls.io

• Thanks to all contributors!

• WARNING: some of the processor paths used in your app could have moved between 0.5.4 and 0.5.5. Check them with timeside.core.processors(). Note that it is now advised to use the timeside.core.get_processor() method to instantiate the processors with their respective id as argument.

• UPGRADING from the sources: please remove all .pyc files from your repository.

0.5.4

• Encoder : transcoded streams where broken. Now fixed with some smart thread controls.

• Analyzer : update VAMP plugin example in sandbox

• Analyzer : new experimental plugin : Limsi Speech Activity Detection Systems (limsi_sad)

• Decoder : process any media in streaming mode giving its URL

• Install : fix some setup requirements

0.5.3

• Make Analyzer rendering more generic and easy to implement

• Analyzer : implement rendering capability for event and segment + add some more analyzer graphers

• Analyzer : refactoring the results rendering method. + Capability to use matplotlib in environnement with no display

• Decoder : Add a Live decoder to get data from the soundcard

• Decoder : add support for 96kHz sampling rate

• Encoder: live AudioSink encoder, encoder that plays the audio stream through the soundcard

• Grapher : add a generic Class to display Analyzers through their ‘render’ method. Add the new grapher file

• Grapher : add a generic Class to display Analyzers through their ‘render’ method. For now, it only support FrameValueResult analyzer

• Core : add a condition to catch signal only if a LiveDecoder source is used

• Various bugfixes

0.5.2

• Add a general launch script “timeside-launch” (see “Shell interface”)

• Add some decorators to filter the inputs of processes (see analyzer.waveform for ex)

• Add a “stack” option to the FileDecoder to accumulate audio data allowing multipass processes

• Add beat confidence to aubio_temporal

• Add AAC encoder (gstreamer voaacenc plugin needed)

• Add UUIDs to the file URI and to all processors

• Add a Debian repository with all dependencies for i386 and amd64 architectures

• Fix buggy WebM encoder

• Fix buggy MP3 muxing

• Fix various minor bugs

0.5.1

• Add parent processor list to Processor

• Simplify and optimize the grapher system

• Add Grapher abstract generic class

• Add a UUID property to Processor

• Add a SpectrogramLinear grapher

• Add WaveformTransparent grapher

• Fix some assignment issues regarding immutable type in for Analyzer Result

• Simplify analyzer results implementation by introducing a Factory and multiple classes and subclasses to handle the 8 different kinds of results

• Add doctests and improve the unit tests

• Add a OnsetDetectionFunction analyzer

• Update documentation

• Various cleanups

• Various bugfixes

0.5.0

• Deep refactoring of the analyzer API to handle various new usecases, specifically audio feature extraction

• Add serializable global result container (NEW dependency to h5py, json, yaml)

• Add new audio feature extraction analyzers thanks to the Aubio library providing beat & BPM detection, pitch dectection and other cool stuff (NEW dependency on aubio)

• Add new audio feature extraction analyzers thanks to the Yaafe library (NEW dependency on yaafe)

• Add new IRIT speech detection analyzers (NEW dependency on scipy)

• EXPERIMENTAL : add new audio feature extraction thanks to the VAMP plugin library (NEW dependency on some vamp toold)

• Add new documentation : http://files.parisson.com/timeside/doc/

• New Debian repository for instant install

• Various bugfixes

• Comptatible with Python >=2.7

• WARNING : no longer compatible with Telemeta 1.4.5

Dive in

To list all available plugins:

>>> import timeside
>>> timeside.core.list_processors()

Define some processors:

>>> from timeside.core import get_processor
>>> decoder  =  get_processor('gst_dec')('sweep.wav')
>>> grapher  =  get_processor('waveform_simple')
>>> analyzer =  get_processor('level')
>>> encoder  =  get_processor('gst_vorbis_enc')('sweep.ogg')

Then run the magic pipeline:

>>> (decoder | grapher | analyzer | encoder).run()

Render the grapher results:

>>> grapher.render(output='waveform.png')

Show the analyzer results:

>>> print 'Level:', analyzer.results

The encoded OGG file should also be there…

Note you can also instanciate each processor with its own class:

>>> decoder  =  timeside.decoder.file.FileDecoder('sweep.wav')
>>> grapher  =  timeside.grapher.waveform_simple.Waveform()
>>> analyzer =  timeside.analyzer.level.Level()
>>> encoder  =  timeside.encoder.ogg.VorbisEncoder('sweep.ogg')

For more extensive examples, please see the full documentation.

API / Documentation

• General : http://files.parisson.com/timeside/doc/

• Tutorial : http://files.parisson.com/timeside/doc/tutorial/index.html

• API : http://files.parisson.com/timeside/doc/api/index.html

• Player / UI : https://github.com/yomguy/TimeSide/wiki/Ui-Guide (see also “Web player”)

• Examples:

  • http://nbviewer.ipython.org/github/thomasfillon/AES53-timeside-demos/tree/master/

  • https://github.com/yomguy/TimeSide/blob/master/tests/sandbox/example_CMMR.py

  • https://github.com/yomguy/TimeSide/blob/master/tests/sandbox/exempleCMMR_vamp.py

Install

The TimeSide engine is intended to work on all Linux and Unix like platforms.

It depends on several other python modules and compiled librairies like GStreamer.

$ sudo pip install timeside

$ sudo pip install timeside

python (>=2.7) python-setuptools python-numpy python-scipy python-h5py python-matplotlib pillow
python-simplejson python-yaml python-mutagen libhdf5-serial-dev python-gst0.10
gstreamer0.10-gnonlin gstreamer0.10-plugins-good gstreamer0.10-plugins-bad gstreamer0.10-plugins-ugly

aubio (>=0.4.1) yaafe python-aubio python-yaafe vamp-examples
django (>=1.4) django-south djangorestframework django-extensions

User Interfaces

$ timeside-launch -h
Usage: scripts/timeside-launch [options] -c file.conf file1.wav [file2.wav ...]
 help: scripts/timeside-launch -h

Options:
 -h, --help            show this help message and exit
 -v, --verbose         be verbose
 -q, --quiet           be quiet
 -C <config_file>, --conf=<config_file>
                       configuration file
 -s <samplerate>, --samplerate=<samplerate>
                       samplerate at which to run the pipeline
 -c <channels>, --channels=<channels>
                       number of channels to run the pipeline with
 -b <blocksize>, --blocksize=<blocksize>
                       blocksize at which to run the pipeline
 -a <analyzers>, --analyzers=<analyzers>
                       analyzers in the pipeline
 -g <graphers>, --graphers=<graphers>
                       graphers in the pipeline
 -e <encoders>, --encoders=<encoders>
                       encoders in the pipeline
 -R <formats>, --results-formats=<formats>
                       list of results output formats for the analyzers
                       results
 -I <formats>, --images-formats=<formats>
                       list of graph output formats for the analyzers results
 -o <outputdir>, --ouput-directory=<outputdir>
                       output directory

$ cd timeside/server/sandbox
$ ./manage.py syncdb
$ ./manage.py migrate
$ ./manage.py runserver

Development

For versions >=0.5 on Debian 7 Wheezy:

$ echo "deb http://debian.parisson.com/debian/ stable main" | sudo tee -a /etc/apt/sources.list
$ echo "deb-src http://debian.parisson.com/debian/ stable main" | sudo tee -a /etc/apt/sources.list
$ sudo apt-get update
$ sudo apt-get install git
$ sudo apt-get build-dep python-timeside

$ git clone https://github.com/yomguy/TimeSide.git
$ cd TimeSide
$ git checkout dev
$ sudo pip install -e .
$ python tests/run_all_tests

Sponsors and Partners

• Parisson

• CNRS (National Center of Science Research, France)

• Huma-Num (big data equipment for digital humanities, ex TGE Adonis)

• CREM (french National Center of Ethomusicology Research, France)

• Université Pierre et Marie Curie (UPMC Paris, France)

• ANR (CONTINT 2012 project : DIADEMS)

• MNHN : Museum National d’Histoire Naturelle (Paris, France)

Related projects

• Telemeta : open web audio platform

• Sound archives of the CNRS, CREM and the “Musée de l’Homme” in Paris, France.

• The DIADEMS project sponsored by the ANR.

Copyrights

• Copyright (c) 2006, 2014 Parisson SARL

• Copyright (c) 2006, 2014 Guillaume Pellerin

• Copyright (c) 2010, 2014 Paul Brossier

• Copyright (c) 2013, 2014 Thomas Fillon

• Copyright (c) 2013, 2014 Maxime Lecoz

• Copyright (c) 2013, 2014 David Doukhan

• Copyright (c) 2006, 2010 Samalyse SARL

License

TimeSide is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 2 of the License, or (at your option) any later version.

TimeSide is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

See LICENSE for more details.