Skip to main content

Compare directories, efficiently sync changes to AWS, and invalidate CDNs.

Project description

https://img.shields.io/pypi/status/wood.svg https://img.shields.io/pypi/v/wood.svg https://img.shields.io/pypi/pyversions/wood.svg https://travis-ci.org/gebn/wood.svg?branch=master https://coveralls.io/repos/github/gebn/wood/badge.svg?branch=master

Wood is a toolbox for representing directory trees, calculating comparisons between them, and effecting those changes in S3, CloudFront and Cloudflare. It started as a set of deploy scripts for my personal website, but grew into something that I was too chuffed with to keep hidden away in the bowls of another website repository. It’s not as refined as, say, requests, but should be useful to anyone looking to efficiently deploy a website to AWS, and/or using Cloudflare as a CDN.

Notably, unlike the AWS CLI tools or boto3, wood does not rely on file timestamps, instead using MD5 checksums to make the minimum number of changes possible in S3. It also has an algorithm for aggregating invalidation paths to make the CloudFront free tier go as far as possible. Syncers and invalidators are implemented in a generic way, allowing easy extension to additional services as needed.

Features

  • Tree representation

  • Tree comparison

    • Action that comparison in S3

    • Invalidate paths as necessary in CloudFront and Cloudflare

Demo

import wood

# low-level comparison of two local directories
comparison = wood.compare('~/dir', '~/.snapshot/hourly.1/dir')
comparison.new()       # files added since the snapshot
comparison.modified()  # files modified since the snapshot
comparison.deleted()   # files deleted since the snapshot


import pathlib
import boto3

local_base = pathlib.Path('/path/to/web/root')
bucket = boto3.resource('s3').Bucket('example.com')

# create representations of the local and remote trees
to_deploy = wood.root(local_base)
deployed = wood.s3.objects_to_root(bucket.objects.all())

# compare the two as if they were local directories
comparison = wood.compare(to_deploy, deployed)

# write all changes (additions, modifications, deletions) to the S3 bucket
syncer = wood.s3.Syncer(local_base, bucket)
syncer.sync(comparison)

# invalidate the minimum amount in CloudFront to ensure the changes are
# visible, using prefix grouping where possible
cloudfront = boto3.client('cloudfront')
invalidator = wood.cloudfront.Invalidator(cloudfront, '{distribution}',
                                          '{reference}')
invalidator.invalidate(comparison)

# do the same for Cloudflare in the case of a second CDN
invalidator = wood.cloudflare.Invalidator(sess, email, key, zone,
                                         'https://example.com/')
invalidator.invalidate(comparison)

Why “wood”?

Because it started as a tree comparison tool, and grew out from there. It’s also a little rough around the edges.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

wood-1.0.3.tar.gz (15.6 kB view hashes)

Uploaded Source

Built Distribution

wood-1.0.3-py2.py3-none-any.whl (20.5 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page