Education and the Cloud

January 1, 2015

Using Rundeck on Ubuntu to automate server deployments into LXC (local or remote) containers

Filed under: Cloud Management, LXC, ubuntu, Virtualization Tools — Tags: , , , , — bmullan @ 11:59 am

Continuing my last posts regarding LXC (linux containers) I realize that managing them from the command line might be a bit tedious when there can be hundreds or thousands of containers between your local PC/laptop and any “remote” (ie cloud) servers & LXC containers you utilize/manage.

I just recently found out about Rundeck while searching for orchestration/mgmt tools.

My use-case was that I was looking for something that could help in managing LXC (linux containers) whether remote or local.

Note:  many people confuse LXC and other container technologies like Docker, LMCTFY, etc.   They are all different solutions that underneath utilize Linux Namespaces.   Here is a good multi-part series describing Linux Namespaces.

LXC (https://linuxcontainers.org/) is an incredible technology.

With the release of 1.x this past year it now supports nested containers, unprivileged containers and much more.

Anyway, I decided to see if I could get Rundeck to work in an LXC container and also be able to create workflows/jobs etc to work with LXC containers.

LXC has a rich set of CLI commands:

  • lxc-create
  • lxc-start
  • lxc-attach
  • lxc-stop
  • lxc-clone
  • lxc-destroy
  • etc

There is also an API that supports Python, Go, Ruby etc.

Stephane Graber (one of the LXC core developers) has a great 10 part Blog series that tells you all about LXC.

For me,  I just wanted to get Rundeck to issue the above lxc-xxxxx commands.

Turns out it only took a couple configuration changes so I thought I’d share my notes here.

Note: all of this was done on Ubuntu 14.04

Steps I took to install Rundeck in an LXC container.

create a new container on the Host.   I called mine “rundeck”

$ sudo lxc-create -t download -n rundeck

start the container which will run detached from the terminal you started it on.

$ sudo lxc-start -n rundeck

attach (re get a console into the container)

$ sudo lxc-attach -n rundeck

Note: at this point your console prompt should change to show you are logged in as Root in the Container whose hostname is “rundeck”.

At this point you can do whatever you would do with any ubuntu server but here were my steps

root@rundeck#  apt-get update && apt-get upgrade -y
root@rundeck# apt-get install wget nano default-jre

then I used wget to download the latest Rundeck .deb file:

root@rundeck# wget http://dl.bintray.com/rundeck/rundeck-deb/rundeck-2.4.0-1-GA.deb

Note:  check on their website for the rundeck version number as it may change often

install the rundeck .deb it

root@rundeck# gdebi ./rundeck-2.4.0-1-GA.deb

When the Rundeck installation is done I needed to do a couple of things.

LXC containers in Ubuntu by default are started in their own 10.0.3.x network.   By default applications in the container have internet access and as I’d mentioned before are like being logged into any other ubuntu server in regards to what you can do.

Because its possible that each time you stop/restart an LXC container it may get a different 10.0.3.x address I wanted to create a solution to where the Rundeck webapp would acquire the “current” IP address of the container Rundeck is running inside of, each time that container starts & rundeck starts.

My script looks like this and I saved it into the containers /usr/bin directory after setting it as executable (chmod +x sethost.sh) …  I called mine “sethost.sh” or whatever you called it.


#!/bin/bash
#
# sethost.sh
# purpose:
#            get ip address while running inside a container
#            Stream Edit (SED) and set the /etc/rundeck/rundesk-config.
properties file so that it substitutes
#            “localhost” with that IP
#
#            $ sudo service rundeskd start
#
# assumptions:   container is using its own eth0 for network connection and its primary address
#
# This script is called from /etc/rc.local during system boot but after network IP is set and rundeckd is started
#
# first set rundeck-config.properties back to original
cp /etc/rundeck/rundeck-config.properties-save /etc/rundeck/rundeck-config.properties

# get eth0 IP address (we assume that’s what the container is using

my_ip=$(ifconfig eth0 | grep “inet addr” | awk -F: ‘{print $2}’ | awk ‘{print $1}’)

# swap the term “localhost” with the real IP of the container in the file rundeck-config.properties
FILE=/etc/rundeck/rundeck-config.properties
sed -i -e “s|localhost|$my_ip|” $FILE
# restart the rundeckd service with the new & now actual IP address
sudo /etc/init.d/rundeckd restart

exit

Run this script by adding it into the rc.local file  inside of the lxc container in which you installed rundeck (my container is called rundeck)

in /etc/rc.local just add the following line at the end of whats already there

/usr/bin/sethost.sh

Next, as you may note above I am simply searching rundeck-config.properties for the word “localhost” and substituting the current IP address of the eth0 of the LXC container Rundeck is running in as again “by default” an LXC container will use that IP address and I am assuming defaults here.

Secondly, to keep this simple, before I ever restart the system for the first time I copied the rundeck-config.properties to rundeck-config.properties-save so I had a virgin copy of the original file with “localhost” still in it.

The first step of the sethost.sh script is to restore the original file, then do the SED substitution that way I could always find/substitute the actual IP of the container.

root@rundeck# cp /etc/rundeck/rundeck-config.properites  /etc/rundeck/rundeck-config.properties-save

My next step was to enable use of SUDO in job command so I could have Rundeck work with Privileged LXC containers

Remember to create/start/stop etc those you have to have SUDO privileges on the Host.

I searched the Rundeck forum and found others were grappling with this problem too.

For me my solution (whether its best or not) worked.

I used visudo to edit the SUDOERS file and set the user “rundeck” so “rundeck” user does NOT require a password to execute a sudo command.

Note:  Again, you are doing this WHILE LOGGED INTO the “rundeck” container – NOT – the Host !

This will enable the rundeck web app to execute commands that require “sudo” in them.

in the rundeck container…

$ sudo visudo

Add the following at the end of the sudoers file

rundeck ALL=(ALL) NOPASSWD: ALL

Ctrl-X to leave, save your changes, and you’re done!

Now while logged INTO the rundeck container reboot it.

root@rundeck# shutdown -r now

Note that this will log you out of the container and return you to the original terminal prompt on your Host OS.

If you want to log back into the container “rundeck” you should be able to almost immediately log back in using the lxc-attach command again

$ sudo lxc-attach -n rundeck

But at this point you should be able to log into Rundeck which is running in the separate and isolated LXC container we also called rundeck by pointing your browser to the IP address of the container.

You can find out the containers IP address using the following LXC command while in a terminal on the Host OS:

$ sudo lxc-ls -f

NAME     STATE    IPV4                            IPV6  GROUPS  AUTOSTART 
———————————————————————————————————
base_cn  STOPPED  –                                  –          –                 NO        
rundeck  RUNNING  10.0.3.136, 10.0.4.1    –          –                 NO        
wings    STOPPED  –                                    –          –                 NO  

so in the above case I point my browser to:     http://10.0.3.136:4440

and log into Rundeck as normal (admin/admin  -or- user/user)

However, now when I create a “job” for the localhost… that job executes inside of the LXC container “rundeck” and NOT on the Host OS …!

If you read the linuxcontainers.org website you will also have noticed a new capability/extension to LXC that is now available called LXD (lex-dee).

LXD is introducing a whole new exciting  capability to LXC that includes the ability to easily create/run/manage LXC containers anywhere on any LXC capable host (LXC is part of the linux kernel) whether that host is remote (re Cloud) or local.

This means that even on your laptop you can have dozens or many dozens (depending on memory, applications, etc) of containers all isolated as much/little as you want from each other, from the Host or from the internet.

So now you can use Rundeck to manage/orchestrate all your local PC LXC containers BUT… you should also be able to use LXC & LXD to do the same with remote (re Cloud) servers/LXC containers.

As I am no expert in Rundeck, LXC or Linux feel free to suggest improvements, changes etc where you think this post requires it as I am sure I probably have made some incorrect assumptions w/Rundeck and/or LXC here.

thanks.

Brian

Advertisements

Create a free website or blog at WordPress.com.