Automatic Heat template generation
Project description
Heat
OpenStack Orchestration project Heat implements an orchestration engine to launch multiple composite cloud applications based on templates. A Heat template describes infrastructure resources (servers, networks, floating ips, etc) and the relationships between these resources, allowing Heat to deploy the resources in a correct order and to manage whole infrastructure lifecycle.
Today Heat provides compatibility with the AWS CloudFormation template format and has its own, native format called Heat Orchestration Template (HOT).
Flame
Flame, a tool that generates HOT Heat template from already existing infrastructure. Currently this project is developed by Thomas Herve (Heat core developer) and myself and provides support for Nova (key pairs and servers), Cinder (volumes) and Neutron (router, networks, subnets, security groups and floating IPs) resources.
Flame works as follows: using provided credentials (user name, project name, password, authentication url), the tool will list supported resources deployed in the project and will generate corresponding, highly customized HOT template.
First example : Router, network, instance from image
The easiest way to understand how Flame works is to show some examples. So here they are.
Suppose that in your project you deployed a router, a network with corresponding subnet and an instance that is booted from an image. A public key was imported in the project and used to access the instance, a floating IP was associated with the instance and the default security group was modified in order to allow incoming ssh connection.
For this infrastructure, Flame will generate a template that will look like this:
description: Generated template
heat_template_version: 2013-05-23
parameters:
external_network_for_floating_ip_0:
constraints:
- custom_constraint: neutron.network
description: Network to allocate floating IP from
type: string
flavor_server_0:
default: m1.small
description: Flavor to use for instance my_instance
type: string
image_server_0:
description: Image to use to boot instance my_instance
type: string
router_0_external_network:
constraints:
- custom_constraint: neutron.network
description: Router external network
type: string
resources:
floatingip_0:
properties:
floating_network_id:
get_param: external_network_for_floating_ip_0
type: OS::Neutron::FloatingIP
key_0:
properties:
name: arezmerita
public_key: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC8u8FIZjmhO+hM/f+2J9qYKgJPG16pQmBfQeUvFlC5u9xxf57eGKuq7xYMIoW63gGM8dnsXcQp9Lmp/+TacwPkis5Q8LKriJSxZUgwczM2ppwwJ/SOraRDHy+2bgbrrO2ZYNdoD5zBaiC5jh6YemrB+y5TtkiEo+llNZw+6e5TlZxEEGD4Zgid/Tfz4qwkKvoGwx34ltQ+XvT2Tv6kE7JWc8rR37wkCbLVQd3G3vAJFI3bWrYan3XNP5+wsVydWn3APF2l8FtLkSpE5Fkai7OWACPRZ9zNlQSBk6pRNlxfZ8jQL6Kuk3MU2tTrqw5g/jG7Hlu3vCeDIYOiFI2a8GUX
type: OS::Nova::KeyPair
network_0:
properties:
admin_state_up: true
name: network
shared: false
type: OS::Neutron::Net
network_subnet_0:
properties:
allocation_pools:
- end: 10.0.48.254
start: 10.0.48.242
cidr: 10.0.48.240/28
dns_nameservers: []
enable_dhcp: true
host_routes: []
ip_version: 4
name: network_subnet
network_id:
get_resource: network_0
type: OS::Neutron::Subnet
newdefault_0:
properties:
description: default
name: newdefault
rules:
- direction: egress
ethertype: IPv6
- direction: ingress
ethertype: IPv6
remote_mode: remote_group_id
- direction: ingress
ethertype: IPv4
port_range_max: 22
port_range_min: 22
protocol: tcp
remote_ip_prefix: 0.0.0.0/0
- direction: egress
ethertype: IPv4
- direction: ingress
ethertype: IPv4
remote_mode: remote_group_id
type: OS::Neutron::SecurityGroup
router_0:
properties:
admin_state_up: true
name: router
type: OS::Neutron::Router
router_0_gateway:
properties:
network_id:
get_param: router_0_external_network
router_id:
get_resource: router_0
type: OS::Neutron::RouterGateway
router_0_interface_0:
properties:
router_id:
get_resource: router_0
subnet_id:
get_resource: network_subnet_0
type: OS::Neutron::RouterInterface
server_0:
properties:
block_device_mapping: []
config_drive: ''
diskConfig: AUTO
flavor:
get_param: flavor_server_0
image:
get_param: image_server_0
key_name:
get_resource: key_0
name: my_instance
networks:
- network:
get_resource: network_0
security_groups:
- get_resource: newdefault_0
type: OS::Nova::Server
It’s not so easy to write a template like this manually, right?
There are two major sections in this generated template: parameters and resources. The parameters section is used to customize each deployment, by specifying input parameters for template instantiation. In the resources section, are defined actual resources that will compose a Heat stack deployed from the HOT template.
In order to understand this generated template and to be able to modify it, I will explain here for each resource type, the possible parameters and its relationship with other resources.
Floating IP
floatingip_0:
properties:
floating_network_id:
get_param: external_network_for_floating_ip_0
type: OS::Neutron::FloatingIP
Each resource declaration block is headed by the resource ID: floatingip_0 for this resource. Every resource ID must be unique within the resource section. Type attribute specifies the type of resource: OS::Neutron::FloatingIP in this case.
Since a floating IP is allocated from a network, we must indicate the network id. To be fully customisable, instead of hard coding a network id, we add a parameter external_network_for_floating_ip_0, that will be provided by the user during Heat stack creation.
Key Pair used to access the instance
key_0:
properties:
name: arezmerita
public_key: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC8u8FIZjmhO+hM/f+2J9qYKgJPG16pQmBfQeUvFlC5u9xxf57eGKuq7xYMIoW63gGM8dnsXcQp9Lmp/+TacwPkis5Q8LKriJSxZUgwczM2ppwwJ/SOraRDHy+2bgbrrO2ZYNdoD5zBaiC5jh6YemrB+y5TtkiEo+llNZw+6e5TlZxEEGD4Zgid/Tfz4qwkKvoGwx34ltQ+XvT2Tv6kE7JWc8rR37wkCbLVQd3G3vAJFI3bWrYan3XNP5+wsVydWn3APF2l8FtLkSpE5Fkai7OWACPRZ9zNlQSBk6pRNlxfZ8jQL6Kuk3MU2tTrqw5g/jG7Hlu3vCeDIYOiFI2a8GUX
type: OS::Nova::KeyPair
Like floating IP resource, this key pair resource has an ID and a type. In addition, the name and the value of the public key are specified.
Network
network_0:
properties:
admin_state_up: true
name: network
shared: false
type: OS::Neutron::Net
network_subnet_0:
properties:
allocation_pools:
- end: 10.0.48.254
start: 10.0.48.242
cidr: 10.0.48.240/28
dns_nameservers: []
enable_dhcp: true
host_routes: []
ip_version: 4
name: network_subnet
network_id:
get_resource: network_0
type: OS::Neutron::Subnet
Declaration of the network resource does not differ much from from two previous resources. However, the subnet resource that belongs to the network_0 resource requires its network ID. Since we do not know it, we will just reference network_0 resource using get_resource: network_0. At runtime, this reference will be resolved to reference ID of the network resource.
Router, router gateway, router interface
router_0:
properties:
admin_state_up: true
name: router
type: OS::Neutron::Router
router_0_gateway:
properties:
network_id:
get_param: router_0_external_network
router_id:
get_resource: router_0
type: OS::Neutron::RouterGateway
router_0_interface_0:
properties:
router_id:
get_resource: router_0
subnet_id:
get_resource: network_subnet_0
type: OS::Neutron::RouterInterface
These three resources are closely related. The router_0 resource declares a router. The router_0_gateway declares external network gateway for this router and expects a parameter from the user ( get_param: router_0_external_network ), that corresponds to the ID of the external network for the gateway.
The router_0_interface_0 resource declares an internal network interface to the router_0.
Security group
newdefault_0:
properties:
description: default
name: newdefault
rules:
- direction: egress
ethertype: IPv6
- direction: ingress
ethertype: IPv6
remote_mode: remote_group_id
- direction: ingress
ethertype: IPv4
port_range_max: 22
port_range_min: 22
protocol: tcp
remote_ip_prefix: 0.0.0.0/0
- direction: egress
ethertype: IPv4
- direction: ingress
ethertype: IPv4
remote_mode: remote_group_id
type: OS::Neutron::SecurityGroup
The default security group is created automatically for each project. The user can add new rules in this group, but the user is not allowed to delete this group or create another security group with the name default. For this reason, when we export this group, we rename it to _default.
Instance
server_0:
properties:
block_device_mapping: []
config_drive: ''
diskConfig: AUTO
flavor:
get_param: flavor_server_0
image:
get_param: image_server_0
key_name:
get_resource: key_0
name: my_instance
networks:
- network:
get_resource: network_0
security_groups:
- get_resource: newdefault_0
type: OS::Nova::Server
The last resource corresponds to the instance. The flavor and image used to boot this server must be specified by the user : get_param: flavor_server_0 and get_param: image_server_0. The keypair name, network and security group will be automatically resolved.
Second example : Router, network, two instances, volumes
The second example will focus on volumes. Like in previous example, in your project you deployed a router, a network with corresponding subnet and two instances. The first instance was booted from a volume that was created from an image. The second instance was booted from an image and a volume is attached to this instance.
For this infrastructure, generated template will look like this (only the major differences are showed):
description: Generated template
heat_template_version: 2013-05-23
parameters:
......
flavor_server_0:
default: m1.small
description: Flavor to use for instance instance
type: string
flavor_server_1:
default: m1.small
description: Flavor to use for instance instance_from_volume
type: string
image_server_0:
description: Image to use to boot instance instance
type: string
volume_image_1:
description: Image to create volume volume_from_image
type: string
volume_type_0:
default: iscsi
description: Volume type for volume resource volume_0
type: string
volume_type_1:
default: iscsi
description: Volume type for volume resource volume_1
type: string
resources:
......
server_0:
properties:
block_device_mapping:
- device_name: /dev/vdb
volume_id:
get_resource: volume_0
diskConfig: AUTO
flavor:
get_param: flavor_server_0
image:
get_param: image_server_0
key_name:
get_resource: key_0
name: instance
networks:
- network:
get_resource: network_0
security_groups:
- get_resource: _default_0
type: OS::Nova::Server
server_1:
properties:
block_device_mapping:
- device_name: vda
volume_id:
get_resource: volume_1
diskConfig: AUTO
flavor:
get_param: flavor_server_1
key_name:
get_resource: key_0
name: instance_from_volume
networks:
- network:
get_resource: network_0
security_groups:
- get_resource: _default_0
type: OS::Nova::Server
volume_0:
properties:
metadata:
attached_mode: rw
readonly: 'False'
name: volume
size: 10
volume_type:
get_param: volume_type_0
type: OS::Cinder::Volume
volume_1:
properties:
image:
get_param: volume_image_1
metadata:
attached_mode: rw
readonly: 'False'
name: volume_from_image
size: 10
volume_type:
get_param: volume_type_1
type: OS::Cinder::Volume
As in the previous example, each resource is identified by an ID. Since in this example we have two volumes and two instances, two OS::Cinder::Volume and two OS::Nova::Server resources are added.
In this example, for two server resources, the properties section block_device_mapping is used to express the fact that
the volume resource volume_0 is attached to server_0 on /dev/vdb
that volume resource volume_1 is used as boot source for server_1 on vda device.
And since an image is used to create bootable volume volume_1, Flame will add volume_image_1 parameter in template parameters section.
Conclusion
In this article we saw how to use Flame to automatically generate Heat template from existing infrastructure. Generated template is highly customized, can be easily modified and reused on every OpenStack installation (with Heat of cause).
Since Flame is still in development, there are some interesting features that we will add to improve it: enrich the set of supported resources, add stack data file generation, that will help users to “adopt” Heat stacks from already existing resources, and the last, but not the least, we would like to improve resource selection i.e. give the user the possibility to specify one by one what resource he/she wants to export in template.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
