skip to navigation
skip to content

Not Logged In

phase.timeserieslogger 2.0.10

Python script that store timeseries in TSV files.

Time series logger module for phase project.

This module is one of the main module of phase project. It collects every timeseries messages sent on the RabbitMQ network like devices consumptions informations, devices events or devices connections errors and store this timleseries in TSV files for further analysis.

For example, this module let us collecting timeseries from phase currencost module and store them into TSV files.

For more information, full documentation is available on readthedocs.org

Installation

Requirements:

  • Platform: Unix (only tested on Ubuntu 12.04)
  • Python 2.7
  • Pip (last version)
  • RabbitMQ (last version)
  • MongoDB (last version)

To install this software, you need to install python 2.7 and pip

sudo apt-get install python python-pip
sudo pip install --upgrade pip

Then, you have to install and configure RabbitMQ (change “admin” and “password” by your credential)

USERNAME="admin"
PASSWORD="password"
echo "deb http://www.rabbitmq.com/debian/ testing main" > sudo /etc/apt/sources.list.d/rabbitmq.list
wget http://www.rabbitmq.com/rabbitmq-signing-key-public.asc
sudo apt-key add rabbitmq-signing-key-public.asc
sudo apt-get update
sudo apt-get install rabbitmq-server -y
sudo service rabbitmq-server start
sudo rabbitmq-plugins enable rabbitmq_management
# If you get a 'command not found' error for this line, use:
# sudo /usr/lib/rabbitmq/bin/rabbitmq-plugins enable rabbitmq_management
# sudo /usr/lib/rabbitmq/bin/rabbitmq-activate-plugins enable rabbitmq_management
sudo rabbitmqctl add_user $USERNAME $PASSWORD
sudo rabbitmqctl set_permissions -p / $USERNAME ".*" ".*" ".*"
sudo rabbitmqctl delete_user guest
sudo service rabbitmq-server restart

Finally, you can install phase.timeserieslogger

sudo pip install phase.timeserieslogger

To verify version of phase.timeserieslogger installed package

pip show phase.timeserieslogger

Usage

$ sudo phase-timeserieslogger -h
usage: phase-timeserieslogger [-h] [-l LOG_CONF] [-v] [-d DATA_FOLDER]
                              rabbitmq_credential

positional arguments:
  rabbitmq_credential   credential for rabbitMQ. Format: username:password.

optional arguments:
  -h, --help            show this help message and exit
  -l LOG_CONF, --log-conf LOG_CONF
                        path to log configuration
  -v, --verbose         activate verbose mode
  -d DATA_FOLDER, --data-folder DATA_FOLDER
                        Name of the folder where to store data

By default:

  • Log configuration file is located in /opt/phase/phase-timeserieslogger.conf and log file is in /var/logs/phase/phase-timeserieslogger.log. You can set log configuration file using -l option with a path to your log.conf file.
  • Data folder is located in /opt/phase/data. You can set this configuration using -d option with the path to the your folder where you want to store TSV files generated by this script.

Examples

To listen on RabbitMQ port and see message in terminal:

$ sudo phase-timeserieslogger admin:password -v
{"variableID": "electric_meter", "dstTimezone": "UTC", "siteID": "liogen_home", "date": "2014-04-25T12:00:17.754959", "value": "CurrentCost electric_meter in liogen_home: TTY connection problem: /dev/currentcost is unreachable. Retry connection in 5 seconds.", "nonDstTimezone": "UTC"}
{"variableID": "global", "dstTimezone": "UTC", "siteID": "liogen_home", "date": "2014-04-25T12:00:22.769256", "value": "12", "nonDstTimezone": "UTC"}
{"variableID": "living_room_tmpr", "dstTimezone": "UTC", "siteID": "liogen_home", "date": "2014-04-25T12:00:22.769256", "value": "20.3", "nonDstTimezone": "UTC"}

Message receive through RabbitMQ

Two kinds of messages are actually waited by phase-timeserieslogger:

  • Error messages with “error” topic
  • Timeseries messages with “timeseries” topic

This table summarize message receive and according action expected.

Topic Message Actions
timeseries all kinds of timeseries Store value and date of timeseries message in TSV file in <siteID> folder in <variableID>.tsv.<date> file.
error all kinds of errors messages Store this error in TSV file in <siteID folder> in error.tsv.<date> file.

Message send through RabbitMQ

None

Contribute

I am more than happy to accept external contributions like feedback, bug reports and pull requests.

Do not hesitate to post an issue if you have any problem to install or to use this software.

You can also use this way to ask for features request. I am also available to answer you on Stack Overflow

Here is the development process to test and validate your features.

  1. Prepare your development environment:

    $ sudo apt-get install vim git-core
    # Install virtualenv and virtualenvwrapper if it's not done.
    $ sudo pip install virtualenvwrapper
    $ mkdir ~/.virtualenvs
    $ vim ~/.bashrc
    # Modify your ~/.bashrc and add this 2 lines:
    # export WORKON_HOME=~/.virtualenvs
    # source /usr/local/bin/virtualenvwrapper.sh
    $ bash
    
  2. Go on github and fork this project.

  3. Clone it on your conputer:

    $ cd <your_workspace>
    $ git clone git@github.com:<username>/phase.timeserieslogger.git
    $ git checkout develop
    
  4. Prepare your virtualenv

    $ mkvirtualenv phase
    (phase)$ pip install paver
    (phase)$ paver prepare
    
  5. Add functional or unit tests

  6. Code your features

  7. To validate your implementation, launch:

    (phase)$ paver validate
    
  8. Modify Todo, Changelog and update documentation

  9. Commit and push on github:

    (phase)$ git add .
    (phase)$ git commit -a -m "<your commit message>"
    (phase)$ git push origin develop
    
  10. Propose a pull request on github

License

The MIT License (MIT)

Copyright (c) 2014 Pierre Leray

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

 
File Type Py Version Uploaded on Size
phase.timeserieslogger-2.0.10.tar.gz (md5) Source 2014-11-12 9KB
  • Downloads (All Versions):
  • 53 downloads in the last day
  • 476 downloads in the last week
  • 1701 downloads in the last month