Skip to main content

Implement for AWS ECS and Docker Compose what SAM is to Serverless for AWS Lambda

Reason this release was yanked:

obsolete

Project description

PYPI_VERSION GitHub

CodeStyle TDD with pytest BDD with Behave

BUILD

Build your infrastructure and deploy your services to AWS services using docker-compose file format.

Introduction

Docker Compose has been around for a long while and enabled developers to perform local integration testing between their microservices as well as with other dependencies their application have (i.e. a redis or MySQL server).

However, for developers to translate their docker compose file into an AWS infrastructure can be a lot of work. And for the cloud engineers (or DevOps engineers) it can very quickly become something overwhelming to manage at very large scale to ensure best-practices are in place, for example, ensuring least privileges access from a service to another.

This is where ECS ComposeX comes into play.

Translate Docker services into AWS ECS

First ECS ComposeX translates the services definition in the docker compose file into the ECS definitions to allow the service to run on AWS. It will, doing so, create all the necessary elements to ensure a successful and feature rich deployment into ECS.

Provision other AWS resources your services need

So you have the definitions of your services and they are running on ECS. But what about these other services that you need for your application to work? DBs, notifications, streams etc. Are you going to run your MySQL server onto ECS too or are you going to want to use AWS RDS? How are you going to define the IAM roles and policies for each service? Access Secrets? Configuration settings?

That is the second focus of ECS ComposeX: defining extra sections in the YAML document of your docker compose file, you can define, for your databases, queues, secrets etc.

ECS ComposeX will parse every single one of these components. These components can exist on their own but what is of interest is to allow the services to access these.

That is where ECS ComposeX will automatically take care of all of that for you.

For services like SQS or SNS, it will create the IAM policies and assign the permissions to your ECS Task Role so the service gets access to these via IAM and STS. Credentials will be available through the metadata endpoint, which your SDK will pick immediately.

For services such as RDS or ElasticCache, it will create the security groups ingress rules as needed, and when applicable, will handle to generate secrets and expose these via ECS Secrets to your services.

Implementing least privileges at the heart of ECS ComposeX

One of the most important value add for a team of Cloud/DevOps engineers who have to look after an environment to use ECS ComposeX is the persistent implementation of best practices:

  • All microservices are using different sets of credentials

  • All microservices are isolated by default and allowed traffic only when explicitly permitted

  • All microservices must be defined as the consumer of a resource (DB, Queue, Table) to be granted access to it.

There have been to many instances of breaches on AWS due to a lack of strict IAM definitions and permissions. Automation can solve that problem and with ECS ComposeX the effort is to constantly abide by the least privileges access principle.

Plug-And-Play

ECS ComposeX allows to create not only the resources your application stack needs, but also the underlying infrastrcuture, for example, your networking layer (VPC, subnets etc.) as well as the compute (using SpotFleet by default).

This is to allow developers to deploy in their development accounts without having to concern themselves with network design and capacity planning.

Fargate First

However the original deployments and work on this project was done using EC2 instances (using SpotFleet mostly), everything is now implemented to work on AWS Fargate First (2020-06-06).

License and documentation

Blog

Follow the news and technical articles on using ECS ComposeX on the Blog BLOG_RELEASE

GitHub project

To follow the progress of ECS ComposeX and raise issues/feature requests, you can go to to the ECS ComposeX Project

What is next for ECS ComposeX ?

  • Add more resources supports (DynamoDB tables, SNS Topics).

  • Enable definition of service mesh and service discovery

First, move this into a CFN Macro, with a simple root template that would take a few settings in and the URL to the Compose file and render all templates within CFN itself via Lambda. Then, with the newly released CFN Private Registries, mutate this system to have fully integrated to CFN objects which will resolve all this.

Credits

This package would not have been possible without the amazing job done by the AWS CloudFormation team! This package would not have been possible without the amazing community around Troposphere! This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ecs_composex-0.3.0.tar.gz (123.0 kB view hashes)

Uploaded Source

Built Distribution

ecs_composex-0.3.0-py2.py3-none-any.whl (150.8 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page