It is similar to other OS-level virtualization technologies on Linux such as OpenVZ and Linux-VServer, as well as those on other operating systems such as FreeBSD jails and Solaris Containers.
LXC relies on the Linux kernel cgroups functionality that became available in version 2.6.29, developed as part of LXC. It also relies on other kinds of namespace-isolation functionality, which were developed and integrated into the mainline Linux kernel.
The main benefit of this type of virtualization technology is the lack of hypervizor/guest overhead. This achieves close to bare-metal performance in most cases.
In this blog I'll show you how to configure LXC container and run Apache in it.
1. Prepare the host.
Install the main lxc package, bootstrap scrips and the bridge utility.
On Redhat/Fedora/Centos run:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@host1 ~]# yum install lxc bridge-utils febootstrap |
On Debian/Ubuntu run:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@host1 ~]# apt-get install lxc bridge-utils debootstrap libcap-dev |
Create the directory for the container:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@host1 ~]# mkdir /lxc | |
| [root@host1 ~]# ln -s /lxc /var/lib/ | |
| [root@host1 /lxc]# cd /lxc | |
| [root@host1 /lxc]# mkdir mycontainer share bin |
2. Enable cgroup.
For more information on cgroups read my previous article - here
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@host1 /lxc]# mkdir /cgroup | |
| [root@host1 /lxc]# echo "none /cgroup cgroup defaults 0 0" >>/etc/fstab | |
| [root@host1 /lxc]# mount /cgroup | |
| [root@host1 /lxc]# mkdir /cgroup/mycontainer |
3. Configure bridging.
On Redhat like systems perform the following:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| On Redhat like systems perform the following: | |
| [root@host1 /lxc]# chkconfig NetworkManager off | |
| [root@host1 /lxc]# chkconfig network on | |
| [root@host1 /lxc]# service NetworkManager stop | |
| [root@host1 /lxc]# service network start | |
| [root@host1 /lxc]# vi /etc/sysconfig/network-scripts/ifcfg-eth0 | |
| DEVICE=eth0 | |
| TYPE=Ethernet | |
| HWADDR=aa:bb:cc:dd:ee:ff:11:22 | |
| ONBOOT=yes | |
| BRIDGE=br0 | |
| USERCTL=no | |
| IPV6INIT=no | |
| [root@host1 /lxc]# vi /etc/sysconfig/network-scripts/ifcfg-br0 | |
| HOSTNANE=host1 | |
| DEVICE=br0 | |
| ONBOOT=yes | |
| BOOTPROTO=static | |
| DELAY=0 | |
| TYPE=Bridge | |
| IPADDR=192.168.0.100 | |
| NETWORK=192.168.0.0 | |
| NETMASK=255.255.255.0 | |
| GATEWAY=192.168.0.1 | |
| MTU=1500 | |
| DNS1=192.168.0.1 | |
| IPV6INIT=no | |
| USERCTL=no | |
| [root@host1 /lxc]# vi /etc/sysconfig/network-scripts/ifup-post | |
| if [ $DEVNAME = "br0" ] | |
| then | |
| /usr/sbin/brctl setfd br0 0 | |
| fi | |
| [root@host1 ~]# service network restart |
On Debian/Ubuntu:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@host1 /lxc]# vi /etc/network/interfaces | |
| auto lo | |
| iface lo inet loopback | |
| auto br0 | |
| iface br0 inet static | |
| address 192.168.0.100 | |
| netmask 255.255.255.0 | |
| gateway 192.168.0.1 | |
| bridge_ports eth0 | |
| bridge_stp off | |
| bridge_maxwait 5 | |
| post-up /usr/sbin/brctl setfd br0 0 |
4. Building the container.
There are several ways to do this:
- Using the lxc tools like lxc-debian or lxc-fedora.
- Using the debootstrap or febootstrap scirpts.
- Converting an existing openvz containers.
- Using libvirt as shown in this post.
To create the LXC container we need a directory containing the file system - rootfs, that we created in step 1 - /lxc - and a configuration file. To create the config file for the container we can use a template file, adopted from the lxc.conf man page:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@host1 /]# cd /lxc | |
| [root@host1 /lxc]# cat mycontainer.cfg | |
| lxc.utsname = mycontainer | |
| lxc.tty = 4 | |
| lxc.network.type = veth | |
| lxc.network.link = br0 | |
| lxc.network.name = eth0 | |
| # on ESX VM veth does not work, you must use: | |
| # lxc.network.type = phys | |
| # lxc.network.name = eth1 | |
| # lxc.network.link = eth1 | |
| lxc.network.flags = up | |
| lxc.network.mtu = 1500 | |
| lxc.network.ipv4 = 192.168.0.101/24 | |
| lxc.rootfs = /lxc/ubuntu | |
| lxc.mount = /lxc/ubuntu.fstab | |
| lxc.cgroup.devices.deny = a | |
| # Allow any mknod (but not using the node) | |
| lxc.cgroup.devices.allow = c *:* m | |
| lxc.cgroup.devices.allow = b *:* m | |
| # /dev/null and zero | |
| lxc.cgroup.devices.allow = c 1:3 rwm | |
| lxc.cgroup.devices.allow = c 1:5 rwm | |
| # consoles | |
| lxc.cgroup.devices.allow = c 5:1 rwm | |
| lxc.cgroup.devices.allow = c 5:0 rwm | |
| lxc.cgroup.devices.allow = c 4:0 rwm | |
| lxc.cgroup.devices.allow = c 4:1 rwm | |
| # /dev/{,u}random | |
| lxc.cgroup.devices.allow = c 1:9 rwm | |
| lxc.cgroup.devices.allow = c 1:8 rwm | |
| lxc.cgroup.devices.allow = c 136:* rwm | |
| lxc.cgroup.devices.allow = c 5:2 rwm | |
| # rtc | |
| lxc.cgroup.devices.allow = c 254:0 rwm | |
| lxc.cgroup.cpuset.cpus = 0 | |
| lxc.cgroup.cpu.shares = 1024 | |
| lxc.cgroup.memory.limit_in_bytes = 512M | |
| lxc.cgroup.memory.memsw.limit_in_bytes = 512M | |
| # restrict capabilities | |
| lxc.cap.drop = audit_control | |
| lxc.cap.drop = audit_write | |
| lxc.cap.drop = mac_admin | |
| lxc.cap.drop = mac_override | |
| lxc.cap.drop = mknod | |
| lxc.cap.drop = setpcap | |
| lxc.cap.drop = sys_admin | |
| lxc.cap.drop = sys_boot | |
| lxc.cap.drop = sys_module | |
| lxc.cap.drop = sys_rawio | |
| lxc.cap.drop = sys_time |
The most important lines are:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| lxc.utsname = container_name | |
| lxc.network.ipv4 = your_ip | |
| lxc.rootfs = /lxc/rootfs.container_name | |
| lxc.fstab = /lxc/fstab.container_name |
We also need the following fstab file to define our chroot like environment:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@host1 /lxc]# cat mycontainer.fstab | |
| none /lxc/mycontainer/dev/pts devpts defaults 0 0 | |
| none /lxc/mycontainer/proc proc defaults 0 0 | |
| none /lxc/mycontainer/sys sysfs defaults 0 0 | |
| none /lxc/mycontainer/var/lock tmpfs defaults 0 0 | |
| none /lxc/mycontainer/var/run tmpfs defaults 0 0 | |
| #/etc/resolv.conf /lxc/mycontainer/etc/resolv.conf none bind 0 0 | |
| #/home /lxc/mycontainer/home none bind 0 0 | |
| /lxc/share /lxc/mycontainer/share none bind 0 0 | |
| /lxc/bin /lxc/mycontainer/lxc/bin none bind,ro 0 0 | |
| /cgroup/ubuntu /lxc/mycontainer/lxc/cgroup none bind,ro 0 0 | |
| /lib/modules /lxc/mycontainer/lib/modules none bind,ro 0 0 |
To install a minimum Ubuntu OS in the container we can use debootstrap or lxc-ubuntu as mentioned earlier:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@host1 /lxc]# cd /lxc/mycontainer | |
| [root@host1 /lxc/mycontainer]# PKGS=gpgv,kbd,binutils,language-pack-en | |
| [root@host1 /lxc/mycontainer]# debootstrap --variant=minbase --arch amd64 --include=$PKGS lucid ubuntu |
or
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@host1 /lxc/mycontainer]# lxc-ubuntu -p /lxc/mycontainer --trim -r lucid |
5. Modify the container OS.
The basic Ubuntu installation needs to be cleaned some in order for it to execute correctly in the container. Things like modules, and start scripts needs to be removed/altered.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@host1 /lxc]# cd /lxc | |
| [root@host1 /lxc]# rm mycontainer/var/lib/dpkg/info/udev.postinst | |
| [root@host1 /lxc]# rm mycontainer/var/lib/dpkg/info/plymouth.postinst | |
| [root@host1 /lxc]# rm -rf mycontainer/lib/modules/* | |
| [root@host1 /lxc]# cp /etc/passwd /etc/shadow /etc/group mycontainer/etc/ | |
| [root@host1 /lxc]# cat <<EOD> mycontainer/etc/apt/sources.list | |
| deb http://archive.ubuntu.com/ubuntu/ lucid main restricted universe multiverse | |
| deb http://archive.ubuntu.com/ubuntu/ lucid-updates main restricted universe multiverse | |
| deb http://archive.ubuntu.com/ubuntu/ lucid-security main restricted universe multiverse | |
| EOD | |
| [root@host1 /lxc]# mkdir -p mycontainer/lxc/routing mycontainer/lxc/cgroup mycontainer/lxc/bin mycontainer/share | |
| [root@host1 /lxc]# gw=$(route -n | awk '/^0.0.0.0/{print $2}') | |
| [root@host1 /lxc]# echo "route add default gw $gw" >mycontainer/lxc/routing/default |
chroot into mycontainer and configure the container:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@host1 /lxc]# chroot /lxc/mycontainer /bin/bash | |
| [root@host1 /]# export LANG=C | |
| [root@host1 /]# apt-get update | |
| [root@host1 /]# apt-get install -y adduser apt-utils iproute iptables netbase rsyslog sudo vim | |
| [root@host1 /]# apt-get install -y ssh lsof wget | |
| [root@host1 /]# apt-get install -y iputils-ping mtr-tiny dnsutils bind9-host | |
| [root@host1 /]# apt-get install -y ia32-libs libterm-readline-gnu-perl dialog | |
| [root@host1 /]# apt-get dist-upgrade -y | |
| [root@host1 /]# cat <<EOD> /var/lib/locales/supported.d/en | |
| en_US UTF-8 | |
| en_US.Latin1 ISO-8859-1 | |
| en_US.Latin9 ISO-8859-15 | |
| en_US.ISO-8859-1 ISO-8859-1 | |
| en_US.ISO-8859-15 ISO-8859-15 | |
| en_US.UTF-8 UTF-8 | |
| en_GB.UTF-8 UTF-8 | |
| EOD | |
| [root@host1 /]# dpkg-reconfigure locales | |
| [root@host1 /]# /usr/sbin/update-rc.d -f umountfs remove | |
| [root@host1 /]# /usr/sbin/update-rc.d -f hwclock.sh remove | |
| [root@host1 /]# /usr/sbin/update-rc.d -f hwclockfirst.sh remove | |
| [root@host1 /]# rm /etc/init.d/hwclock* | |
| [root@host1 /]# cd etc | |
| [root@host1 /etc]# rm -f mtab | |
| [root@host1 /etc]# ln -s ../proc/mounts mtab | |
| [root@host1 /etc]# cd init | |
| [root@host1 /etc/init]# rm -f console* control* hwclock* module* network-interface* | |
| [root@host1 /etc/init]# rm -f plymouth* procps* tty{4,5,6}.conf udev* upstart* ufw* mountall* | |
| [root@host1 /etc/init]# cd /dev | |
| [root@host1 /dev]# rm mixer* *midi* audio* dsp* smpte* mpu* sequencer sndstat |
Exit the chroot and run:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@host1 /dev]# exit | |
| [root@host1 /lxc]# cp -a --parents \ | |
| /root/.bashrc \ | |
| /etc/resolv.conf \ | |
| /etc/profile /etc/profile.d \ | |
| /etc/bash.bashrc \ | |
| /etc/timezone \ | |
| /etc/localtime \ | |
| /lxc/mycontainer/ | |
| [root@host1 /lxc]# echo ubuntulxc > /lxc/mycontainer/etc/hostname | |
| [root@host1 /lxc]# cat <<EOD > /lxc/mycontainer/etc/fstab | |
| tmpfs /dev/shm tmpfs defaults 0 0 | |
| EOD | |
| [root@host1 /lxc]# cat <<EOD > /lxc/mycontainer/etc/hosts | |
| 127.0.0.1 localhost | |
| [root@host1 /lxc]# cat <<EOD > /lxc/mycontainer/lib/init/fstab | |
| # /lib/init/fstab: static file system information. | |
| # | |
| # These are the filesystems that are always mounted on boot, you can | |
| # override any of these by copying the appropriate line from this file into | |
| # /etc/fstab and tweaking it as you see fit. See fstab(5). | |
| # | |
| # <file system> <mount point> <type> <options> <dump> <pass> | |
| /dev/root / rootfs defaults 0 1 | |
| none /proc proc nodev,noexec,nosuid 0 0 | |
| none /proc/sys/fs/binfmt_misc binfmt_misc nodev,noexec,nosuid,optional 0 0 | |
| none /sys sysfs nodev,noexec,nosuid 0 0 | |
| none /sys/fs/fuse/connections fusectl optional 0 0 | |
| none /sys/kernel/debug debugfs optional 0 0 | |
| none /sys/kernel/security securityfs optional 0 0 | |
| none /spu spufs gid=spu,optional 0 0 | |
| none /dev devtmpfs,tmpfs mode=0755 0 0 | |
| none /dev/pts devpts noexec,nosuid,gid=tty,mode=0620 0 0 | |
| none /dev/shm tmpfs nosuid,nodev 0 0 | |
| none /tmp none defaults 0 0 | |
| none /var/run tmpfs mode=0755,nosuid,showthrough 0 0 | |
| none /var/lock tmpfs nodev,noexec,nosuid,showthrough 0 0 | |
| none /lib/init/rw tmpfs mode=0755,nosuid,optional 0 0 | |
| EOD | |
| [root@host1 /lxc]# rm -f /lxc/mycontainer/etc/init/rc-sysinit | |
| [root@host1 /lxc]# cat << 'EOF' > /lxc/mycontainer/init/rc-sysinit | |
| #!/bin/bash | |
| rm -f $(find /var/run -name '*pid' | |
| rm -f /var/lock/apache/* | |
| route add default gw 192.168.0.1 | |
| exit 0 | |
| EOF | |
| [root@host1 /lxc]# chmod a+x /lxc/mycontainer/etc/init/rc-sysinit |
6. Start the container.
To create and start the LXC run:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@host1 /lxc]# lxc-create -f /lxc/mycontainer/mycontainer.cfg -n ubuntu | |
| [root@host1 /lxc]# lxc-start -n ubuntu -d |
To list the containers and see their state:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@host1 /lxc]# lxc-ls | |
| ubuntu | |
| [root@host1 /lxc]# lxc-info -n ubuntu | |
| RUNNING |
You should now be able to access the container with either lxc-console or ssh:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@host1 /lxc]# ssh root@192.168.0.100 |
or
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@host1 /lxc]# lxc-console -n ubuntu |
To stop the container:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@host1 /lxc]# lxc-stop -n ubuntu |
And to destroy the container run:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@host1 /lxc]# lxc-destroy -n ubuntu |
While in the container you can install and start services as usual.
Resources:
http://en.wikipedia.org/wiki/LXC https://help.ubuntu.com/community/LXC http://blog.bodhizazen.net/linux/lxc-configure-ubuntu-lucid-containers/ http://lxc.teegra.net
