Helper for pushing AWS CloudWatch metrics to Graphite
Project description
Cloudwatch-to-Graphite (leadbutt) is a small utility to take metrics from CloudWatch to Graphite.
Installation
Install using pip:
pip install cloudwatch-to-graphite
Configuring boto
Cloudwatch-to-Graphite uses boto, so make sure to follow its configuration instructions. The easiest way to do this is to set up the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables.
Usage
Configuration Files
If you have a simple setup, the easiest way to get started is to set up a config.yaml. You can copy the included config.yaml.example. Then just run:
leadbutt
If you have several configs you want to switch between, you can specify a custom configuration file:
leadbutt --config-file=production.yaml -n 20
You can even generate configs on the fly and send them in via stdin by setting the config file to ‘-‘:
generate_config_from_inventory | leadbutt --config-file=-
There’s a helper to generate configuration files called plumbum. Use it like:
plumbum [-r REGION] [-f FILTER] [--token TOKEN] template namespace
Namespace is the CloudWatch namespace for the resources of interest; for example AWS/RDS. The template is a Jinja2 template. You can add arbitrary replacement tokens, eg {{ replace_me }}, and then pass in values on the CLI via --token. For example, if you called:
plumbum --token replace_me='hello, world' sample_templates/rds.yml.j2 AWS/RDS
You would get all instances of {{ replace_me }} in the templace replaced with hello, world.
Filters
You can pass simple key=value filters in to plumbum; be aware of the limitations:
the filters run against whatever the AWS API has returned; if you have a lot of objects of whatever type, expect the API request to take a while.
they work only against object attributes and tags returned by the API. For example, RDS and ELB objects can be tagged, but as getting the tags is a per-object subrequest; plumbum does not do those, so you can only filter on the object attributes.
Example: plumbum -f Name=my-dev-instance sample_templates/ec2.yml.j2 ec2
Sending Data to Graphite
If your graphite server is at graphite.local, you can send metrics by chaining with netcat:
leadbutt | nc -q0 graphite.local 2003
Or if you want to use UDP:
leadbutt | nc -uw0 graphite.local 2003
If you need to namespace your metrics for a hosted Graphite provider, you could provide a custom formatter, but the easiest way is to just run the output through awk:
leadbutt | \ awk -v namespace="$HOSTEDGRAPHITE_APIKEY" '{print namespace"."$0}' | \ nc -uw0 my-graphite-provider.xxx 2003
Customizing Your Graphite Metric Names
Set the Formatter option to set the template used to generate Graphite metric names. I wasn’t sure what should be default, so I copied cloudwatch2graphite’s. Here’s what it looks like:
cloudwatch.%(Namespace)s.%(dimension)s.%(MetricName)s.%(statistic)s.%(Unit)s
TitleCased variables come directly from the YAML configuration, while lowercase variables are derived:
statistic – the current statistic since Statistics can be a list
dimension – the dimension value, e.g. “i-r0b0t” or “my-load-balancer”
The format string is Python’s %-style.
config.yaml
What metrics are pulled is in a YAML configuration file. See the example config.yaml.example for an idea of what you can do.
Developing
See: : Contributing.
Useful References
Prior Art
Cloudwatch-to-Graphite was inspired by edasque’s cloudwatch2graphite. I was looking to expand it, but I wanted to use boto.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for cloudwatch-to-graphite-0.11.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | cdf1a5b9718b610f123e853e633ae22d472ce8494528c48e60b5210b86a949d7 |
|
MD5 | 903cec0da2f4a9e1a5166ad44aae4e3c |
|
BLAKE2b-256 | f0c44ea77f6c25b9e110b3c5d0f7b559784ef7c4099e01aeeed119bb9207cde8 |
Hashes for cloudwatch_to_graphite-0.11.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 99b91d392dbabbfb540ee570049eced299da6f2a8e1306d77413f0550469f044 |
|
MD5 | ead3e6fc4ac0b62b73163c81767cb5fe |
|
BLAKE2b-256 | a56a8d5b196378c7a4890bd1a2b6ff7a9985a5843188102f8259afebe7e55656 |