skip to navigation
skip to content

Not Logged In

mmstats 0.7.0

Stat, metric, and diagnostic publishing and consuming tools

Latest Version: 0.8.0

Documentation | Package | Code

https://secure.travis-ci.org/schmichael/mmstats.png?branch=master

About

Mmstats is a way to expose and read diagnostic values and metrics for applications.

Think of mmstats as /proc for your application and the readers as procps utilities.

This project is a Python implementation, but compatible implementations can be made in any language (see Goals).

Discuss at https://groups.google.com/group/python-introspection

Goals

  • Separate publishing/writing from consuming/reading tools
  • Platform/language independent (a Java writer can be read by a Python tool)
  • Predictable performance impact for writers via:
    • No locks (1 writer per thread)
    • No syscalls (after instantiation)
    • All in userspace
    • Reading has no impact on writers
  • Optional persistent (writer can sync anytime)
  • 1-way (Publish/consume only; mmstats are not management extensions)

Usage

Requirements

CPython 2.6 or 2.7 (Windows is untested)

PyPy (only tested in 1.7, should be faster in 1.8)

Using

  1. python setup.py install
  2. import mmstats
  3. Create a subclass of mmstats.MmStats like:
class WebStats(mmstats.MmStats):
    status2xx = mmstats.CounterField(label='status.2XX')
    status3xx = mmstats.CounterField(label='status.3XX')
    status4xx = mmstats.CounterField(label='status.4XX')
    status5xx = mmstats.CounterField(label='status.5XX')
    last_hit = mmstats.DoubleField(label='timers.last_hit')
  1. Instantiate it once per process: (instances are automatically thread local)
webstats = WebStats(label_prefix='web.stats.')
  1. Record some data:
if response.status_code == 200:
    webstats.status2xx.inc()

webstats.last_hit = time.time()
  1. Run slurpstats to read it
  2. Run mmash to create a web interface for stats
  3. Run pollstats -p web.stats.status 2XX,3XX,4XX,5XX /tmp/mmstats-* for a vmstat/dstat like view.
  4. Did a process die unexpectedly and leave around a stale mmstat file? cleanstats /path/to/mmstat/files will check to see which files are stale and remove them.

History

0.7.0 "Local Artisanal Stats" released 2012-10-02

  • Per-thread model instances are created automatically - no need to manually create one per thread
  • Backward incompatible change to naming templates; they now use str.format style substitutions:
    • New: {CMD} - Current process name
    • %PID% -> {PID}
    • %TID% -> {TID}

0.6.2 "Graphtastic" released 2012-03-23

  • Added live graphing of numeric metrics thanks to @haard's work at PyCon
  • Documentation improvements

0.6.1 "MANIFEST.out" released 2012-03-08

  • Fix packaging issue

0.6.0 "PyCon 2012" released 2012-03-08

  • [API CHANGE] - MovingAverageField's kwarg changed from window_size => size
  • Refactored __init__.py into fields, models, and default (and imported public bits into __init__)
  • Added TimerField (MovingAverageField + context manager)
  • Added docs (don't get too excited, just a start)

0.5.0 "100% More Average" released 2012-02-25

  • [API CHANGE] - RunningAverage field is now AverageField
  • Added MovingAverageField with window_size=100 parameter
  • Tests can now be run via "python setup.py test"

0.4.1 "Derpstats" released 2012-01-31

  • Fixed pollstats
  • Updated README slightly

0.4.0 "On the Road to Pycon" released 2012-01-17

  • Added clean module and cleanstats script to clean stale mmstat files
  • Added path kwarg to MmStats class to allow easy path overriding
  • Added StringField for UTF-8 encoded strings
  • Added StaticFloatField & StaticDoubleField
  • Added created UNIX timestamp (sys.created) to default MmStats class
  • Moved all modules into mmstats package
  • Fixed mmash template packaging
  • Fixed test mmstat file cleanup
  • Refactored reading code into mmstats.reader module

0.3.12 "Meow" released 2011-11-29

  • Use ctypes.get_errno() instead of Linux specific wrapper

0.3.11 "Rawr" released 2011-11-29

  • Fix libc loading on OSX

0.3.10 "π²" released 2011-11-28

  • PyPy support (switched from ctypes._CData.from_buffer to .from_address)
  • Multiple calls to MmStats().remove() no longer error (makes testing easier)

0.3.9 "MLIT" released 1970-01-01

  • Mistag of 0.3.8

0.3.8 "Hapiness" released 2011-11-20

  • Allow filename templating with %PID% and %TID% placeholders
  • Allow setting filename template via MMSTATS_FILES environment variable
  • Improved docs slightly
  • Fixed Ctrl-Cing run_flask_example script
  • Fixed 64 bit integer fields on 32 bit platforms
  • Fixed StaticInt64Field (was actually a uint64 field before)
  • Fixed slurpstats debug output (always showed first 40 bytes of file)
  • Strip newlines from org.python.version

0.3.7 "Depressive Realism is for Winners" released 2011-11-17

  • Add pollstats utility (similar to dstat/vmstat)
  • Cleanup development/testing section of the README
  • Slight improvements to basic flask integration example

0.3.6 "The M is for Mongo" released 2011-11-09

  • Allow setting the value of CounterFields

0.3.5 "Ornery Orangutan" released 2011-10-20

  • Added a running average field
  • Made mmash more configurable and added a console entry point
  • Updated TODO

TODO

There's always bugs to fix: https://github.com/schmichael/mmstats/issues/

  • Add API to dynamically add fields to MmStat classes
  • Percentiles
  • Time based windows for moving averages (eg last 60 seconds)
  • Multiple exposed fields (average, mean, and percentiles) from 1 model field
  • Add alternative procedural writer API (vs existing declarative models)
  • Test severity of race conditions (especially: byte value indicating write buffer)
  • Test performance
  • Vary filename based on class name
  • Improve mmash (better live graphing, read from multiple paths, etc)
  • Include semantic metadata with field types (eg to differentiate an int that's a datetime from an int that's a counter)
  • Logo
 
File Type Py Version Uploaded on Size
mmstats-0.7.0.tar.gz (md5) Source 2012-10-02 100KB
  • Downloads (All Versions):
  • 32 downloads in the last day
  • 134 downloads in the last week
  • 1061 downloads in the last month