DevOps: Vagrant with Chef-Server

In my last blog post, The DevOps Tools We Use & How We Use Them, I talked about how we use Chef with Vagrant for managing, maintaining and monitoring our servers. (If you haven’t read it yet, I suggest you have a quick look at it for reference.) This is great for a development environment where you spin up servers locally in Virtualbox, however in doesn’t help much when trying to maintain your production environment. In this blog post I will focus on the differences between chef-solo and chef-server and setting up your own chef server.

Vagrant with Chef-Server

Chef-solo is an open source version of the chef-client that allows using cookbooks with nodes without requiring access to a server. This basically allows you to load your cookbooks, databags, etc. onto a filesystem and run Chef locally, similar to how we did it in the last post. Chef-solo does have some limitations, the main one being the lack of information and flexibility. With chef-solo we do not know about any other nodes, except itself. When it comes to configuring components like Nagios or Graphite this can be very difficult, especially where discovery of your other web or db nodes are non existent. This is where chef-server comes in to play.

Chef-server is the ability to run Chef in a client/server model. The Chef server stores all information about your environments, roles, cookbooks, nodes and more. This means we can dynamically update our servers, assign specific monitoring checks to specific nodes and update configs easily and more often. Chef-server comes in two forms, self hosted chef-server or Enterprise Chef.

In this post we will focus on how to get set up with hosted version of Chef server. Opscode offers a free version of Chef server which supports up to 5 nodes. Once you have signed up we need to download some files from the chef server. Navigate to https://manage.opscode.com/organizations and click the “Generate knife config” link. Then click the “regenerate validation key” link. The knife config is information about where your cookbook lives, your server location and keys to use to authenticate with. The validation key is a key used to authenticate yourself to your Chef server.

Logentries_Try_It_Free_Promo_W

Let’s have a quick look to understand your knife.rb file:

current_dir = File.dirname(__FILE__)
log_level                :info
log_location             STDOUT
node_name                "logentries"
client_key               "#{current_dir}/logentries.pem"
validation_client_name   "logentries-validator"
validation_key           "#{current_dir}/logentries-validator.pem"
chef_server_url          "https://api.opscode.com/organizations/logentries"
cache_type               'BasicFile'
cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )
cookbook_path            ["/location/of/your/cookbooks"]

Make sure your .chef directory and cookbook directory matches what is specified in the config above.

Knife, a command line tool assumes your chef files are in .chef, therefore you will need to create a .chef file and move the previously downloaded files there:

mkdir ~/.chef
mv ~/Downloads/knife.rb ~/.chef
mv ~/Downloads/*.pem ~/.chef

Once completed, let’s check if your configuration is working by running a knife command:

knife client list

This should return with the name of your organisation you signed up for followed by “-validator.”

Let’s upload your cookbooks next, this is done with another knife command:

knife cookbook upload -a

This will read your cookbook_path specified in your knife file and send your cookbooks up to your Chef server.

Knife cookbook list will display a list of the cookbooks you uploaded.

Now, let’s follow on from the last post and use Vagrant with chef-server to provision a VM. Create another file called Vagrantfile with the following contents:

# -*- mode: ruby -*-
# vi: set ft=ruby :
 Vagrant.configure('2') do |config|
 config.vm.box = 'precise'
 config.vm.box_url = 'http://grahamc.com/vagrant/ubuntu-12.04.2-i386-chef-11-omnibus.box'
 config.vm.network :private_network, ip: '192.168.33.10'

config.vm.provider :virtualbox do |vb|
     vb.customize ['modifyvm', :id, '--memory', 1024, '--cpus', '1']
 end
 config.vm.provision :chef_client do |chef|
   chef.node_name = 'java'
   chef.chef_server_url = 'https://api.opscode.com/organizations/logentries'
   chef.validation_key_path = '~/.chef/logentries-validator.pem'
   chef.validation_client_name = 'logentries-validator'
   chef.run_list = [ "recipe[java_wrapper::default]" ]
 end
end

The main difference is the vm provisioner. Before, we used chef_solo, this time we will use chef_client. This basically specifies that you want to use chef server to provision your VM. The options above specify the name of your VM, location of your Chef server and your runlist.

Running VagrantUp will provision your VM with Java. Next navigate to https://manage.opscode.com/nodes/java in your browser to see the information about the node you just provisioned. This is where chef stores all the information about your nodes and what roles, recipes, etc. related to them. And, with that done, you’ve set up your own chef server and have the basis of your production environment up and running, with knowledge of other nodes and the ability to dynamically update our servers, assign specific monitoring checks to specific nodes and update configs easily.



    Share This Post


    Posted in Cloud, DevOps, How To
    4 comments on “DevOps: Vagrant with Chef-Server
    1. Mukund Manikarnike says:

      Hi,
      I understand that Vagrant cannot spin up virtual machines on a remote machine. This means running vagrant up on a particular machine will spin up VMs on that machine alone.

      First of all, is this understanding right?

      If yes, what’s the point of using a chef server to provision the server? I feel that if I’m using Vagrant, using chef solo is the way to go and chef server isn’t adding any value.

      Do you think I’m missing something?

      • Joe says:

        Hi Mukund,

        Vagrant also has an array of plugins, for example vagrant-aws or vagrant-digitalocean. For spinning up machines on these providers you will require Chef server to connect.

        Any other questions, Ill be happy to help,
        Joe.

      • Joe says:

        Hi Mukund,

        Vagrant also has an array of plugins such as vagrant-aws or vagrant-digitalocean which allows spinning up instances in those providers. For such cases you would require Chef server.

        If you’re just spinning up servers locally chef-solo should suffice.

        Regards,
        Joe

        • Mukund Manikarnike says:

          Hi Joe,

          Thanks for the clarification. This tutorial blog that you’ve written is very helpful. :)

          Regards,
          Mukund Manikarnike

    2 Pings/Trackbacks for "DevOps: Vagrant with Chef-Server"
    1. […] In my last blog post, The DevOps Tools We Use & How We Use Them, I talked about how we use Chef with Vagrant for managing, maintaining and monitoring our servers.  […]

    2. […] In my last blog post, The DevOps Tools We Use & How We Use Them, I talked about how we use Chef with Vagrant for managing, maintaining and monitoring our servers.  […]

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    *

    You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <p>

     

    Subscribe to the Blog