Cloud Zone is brought to you in partnership with:

Build a Cloud.org is a resource for those users who want to build cloud computing software with both open source and proprietary software. Mark is a DZone MVB and is not an employee of DZone and has posted 73 posts at DZone. You can read more from them at their website. View Full User Profile

Puppet and CloudStack

03.12.2013
| 2027 views |
  • submit to reddit

Curator's Note: The content of this article was originally written by David Nalley over at the Build a Cloud blog .

Efficient use of CloudStack really demands configuration management (among other things). I've been a puppet user for many years predating my involvement with CloudStack, and thus I have a slight bias for puppet.

Thus it thrilled me to no end when I saw folks like Jason Hancock doing work around automating configuration of CloudStack instances. Jason really knows Puppet, and even operates a new Hosted Puppetmaster startup. Jason showed this off a few times at both PuppetCamp LA 2012, and at the CloudStack Collaboration Conference in 2012.

It's awesome work, and you should take the time to watch both of his videos and check out his blog.

The gist of what he was presenting is configuring instance metadata within CloudStack at deployment, having the instance read that metadata and set it as a fact, and then using case statements to apply different roles to the instances.

And then there was a knife.....plugin


Next I learned that the good folks at Edmunds.com had written a CloudStack plugin for knife. That was exciting in and of itself, especially as a CloudStack person. It wasn't just knife, which is a command-line tool for Chef, another configuration management tool. knife is commonly used to provision machines, and the folks at Edmunds.com had baked in some extra awesomeness. They had the ability to define an application stack based on a JSON definition of what the stack looked like.

So one could define a Hadoop Cluster like this in JSON, complete with network and firewall configuration:
"name": "hadoop_cluster_a",
"description": "A small hadoop cluster with hbase",
"version": "1.0",
"environment": "production",
"servers": [
  {
    "name": "zookeeper-a, zookeeper-b, zookeeper-c",
    "description": "Zookeeper nodes",
    "template": "rhel-5.6-base",
    "service": "small",
    "port_rules": "2181",
    "run_list": "role[cluster_a], role[zookeeper_server]",
    "actions": [
      { "knife_ssh": ["role:zookeeper_server", "sudo chef-client"] }
    ]
  },
  {
    "name": "hadoop-master",
    "description": "Hadoop master node",
    "template": "rhel-5.6-base",
    "service": "large",
    "networks": "app-net, storage-net",
    "port_rules": "50070, 50030, 60010",
    "run_list": "role[cluster_a], role[hadoop_master], role[hbase_master]"
  },
  {
    "name": "hadoop-worker-a hadoop-worker-b hadoop-worker-c",
    "description": "Hadoop worker nodes",
    "template": "rhel-5.6-base",
    "service": "medium",
    "port_rules": "50075, 50060, 60030",
    "run_list": "role[cluster_a], role[hadoop_worker], role[hbase_regionserver]",
    "actions": [
      { "knife_ssh": ["role:hadoop_master", "sudo chef-client"] },
      { "http_request": "http://${hadoop-master}:50070/index.jsp" }
    ]
  }
And then deploying a Hadoop Cluster is as simple as:
    knife cs stack create hadoop_cluster_a
As a CloudStack guy I thought this was awesome, complex applications were suddenly deployable with ease, this is exactly the kind of automation that CloudStack is supposed to enable.

JEALOUSY


As a puppet afficionado though, it made me a bit sad, nothing existed like this for folks using puppet, and I was jealous.

Then I went to FOSDEM


At FOSDEM some folks from a mapping company were talking about their use of CloudStack, and commented that they wanted CloudStack resource types. At first, I was confused. They then showed me some work by Ken Barber on OpenNebula resource and types, and I realized that this was exactly what would fill the role of knife-cloudstack's stack functionality. It also would in many ways, truly be infrastructure as code. It would not just be configuring the OS and software on a machine, but it would be configuring the underlying machine itself, and indeed many machines. You could literally have most of your infrastructure as code.

That was awesome, but it was for OpenNebula, and not CloudStack - I started bugging folks like Teyo Tyree and Dan Bode about this.

Puppetconf


Then I attended Puppetconf, and saw the inimitable Dan Bode showing a similar (albeit evolved) resource type for Google Compute Engine. I was even more envious at this point.

And then just after Thanksgiving - Dan Bode unveiled resources and types for Apache CloudStack.

Now you can use puppet not only to query CloudStack, but you can also define your infrastructure.
    cloudstack_instance { 'foo1':
      ensure     => present,
      flavor     => 'M1.Medium',
      zone       => 'FMT-ACS-001',
      image      => 'CentOS 6.3(64-bit)',
      network    => 'davidsnetwork',
    }
As with other resource types, you can define defaults:
Cloudstack_instance{
  image   => 'CentOS 6.3(64bit)',
  flavor  => 'M1.Medium',
  zone    => 'FMT-ACS-001',
  network => 'davidsnetwork',
  keypair => 'davidskeys',
}
Which makes defining instances much easier, and looking something like this:
cloudstack_instance { 'bar':
  ensure  => $::ensure,
  group   => 'role=db'
}
Of course, that's still a single machine, so if we want to have a multi-machine LAMP stack.
class my_web_stack {
  cloudstack_instance { 'httpd':
    ensure => present,
    group  => 'role=httpd',
  }

  cloudstack_instance { 'mysql':
    ensure => present,
    group  => 'role=mysql',
  }
}
Hopefully this leaves you as excited, as you can now puppetize ALL of your CloudStack instances.








Published at DZone with permission of Mark Hinkle, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)