Deploying OpenVZ Containers

OpenVZ is container-based virtualization for Linux in the likes of LXC. OpenVZ creates multiple secure, isolated Linux containers on a single physical server. [1]
It consists of an optional custom Linux kernel and command-line tools (mainly vzctl).

In this post I'll demonstrate how to install, configure and run an OpenVZ container, bridged to the outside world.

First, lets install it:

	[root@ovz:~]# cd /etc/yum.repos.d
	[root@ovz:~]# wget http://download.openvz.org/openvz.repo
	[root@ovz:~]# rpm --import http://download.openvz.org/RPM-GPG-Key-OpenVZ
	[root@ovz:~]# yum install vzkernel

view raw gistfile1.sh hosted with ❤ by GitHub

This should automatically update the GRUB bootloader. Now let's install the OpenVZ userspace tools:

[root@ovz:~]# yum install vzctl vzquota

view raw gistfile1.sh hosted with ❤ by GitHub

To enable communication between the container and the OS add the following kernel tunables:

	[root@ovz:~]# echo "net.ipv4.ip_forward = 1
	net.ipv4.conf.default.proxy_arp = 0
	net.ipv4.conf.all.rp_filter = 1
	kernel.sysrq = 1
	net.ipv4.conf.default.send_redirects = 1
	net.ipv4.conf.all.send_redirects = 0
	net.ipv4.icmp_echo_ignore_broadcasts=1
	net.ipv4.conf.default.forwarding=1" >> /etc/sysctl.conf
	[root@ovz:~]# sysctl -p

view raw gistfile1.sh hosted with ❤ by GitHub

Reboot into the new OpenVZ aware kernel:

[root@ovz:~]# reboot

view raw gistfile1.sh hosted with ❤ by GitHub

Before we can create virtual machines with OpenVZ, we need to have a template for the distribution that we want to use in the virtual machines in the /vz/template/cache directory.
You can download one from http://wiki.openvz.org/Download/template/precreated.
Let's download a CentOS 6 template:

	[root@ovz:~]# cd /vz/template/cache
	[root@ovz:~]# wget http://download.openvz.org/template/precreated/centos-6-x86_64.tar.gz

view raw gistfile1.sh hosted with ❤ by GitHub

Setup the container and make it start on boot:

	[root@ovz:~]# vzctl create 10 --ostemplate centos-6-x86_64 --config basic
	[root@ovz:~]# vzctl set 10 --onboot yes --save

view raw gistfile1.sh hosted with ❤ by GitHub

Note that each container must have an unique Id, in this case 10.Now let's setup a hostname and IP address:

	[root@ovz:~]# vzctl set 10 --hostname haproxyvz.example.com --save
	[root@ovz:~]# vzctl set 10 --ipadd 10.24.6.230 --save

view raw gistfile1.sh hosted with ❤ by GitHub

Next lets set the maximum number of non-TCP sockets (local sockets, UDP and other types of sockets) and use Google DNS inside the container:

	[root@ovz:~]# vzctl set 10 --numothersock 100 --save
	[root@ovz:~]# vzctl set 10 --nameserver 8.8.8.8 --nameserver 8.8.4.4 --save

view raw gistfile1.sh hosted with ❤ by GitHub

Let's start the container:

[root@ovz:~]# vzctl start 10

view raw gistfile1.sh hosted with ❤ by GitHub

This will create a virtual interface with a name similar to this:

	[root@ovz:~]# ifconfig
	--snip--
	veth101.2 Link encap:Ethernet HWaddr 00:18:51:11:2D:6E
	inet6 addr: fe80::218:51ff:fe11:2d6e/64 Scope:Link
	UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
	RX packets:0 errors:0 dropped:0 overruns:0 frame:0
	TX packets:0 errors:0 dropped:185139 overruns:0 carrier:0
	collisions:0 txqueuelen:0
	RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
	--snip--

view raw gistfile1.sh hosted with ❤ by GitHub

Now lets create a bridge and add our external interface p2p1 and the interface of the container to the bridge br0, so that we can access it from outside:

	[root@ovz:~]# cat /etc/sysconfig/network-scripts/ifcfg-br0
	DEVICE=br0
	BOOTPROTO=static
	IPADDR=10.24.6.211
	NETMASK=255.255.240.0
	GATEWAY=10.24.0.1
	ONBOOT=yes
	TYPE="Bridge"
	[root@ovz:~]# cat /etc/sysconfig/network-scripts/ifcfg-p2p1
	DEVICE=p2p1
	HWADDR=00:26:6C:FE:F2:80
	TYPE=Ethernet
	UUID=cfe0d3f2-8bc9-45c9-91a5-c0cdb9a5644f
	ONBOOT=yes
	BRIDGE=br0
	[root@ovz:~]# /etc/init.d/network restart
	[root@ovz:~]# brctl addif br0 veth101.2
	[root@ovz:~]# vzctl set 10 --netif_add ethp2p1,,,,br0 --save

view raw gistfile1.sh hosted with ❤ by GitHub

Time to change the root password:

[root@ovz:~]# vzctl exec 10 passwd

view raw gistfile1.sh hosted with ❤ by GitHub

To enter the container use:

[root@ovz:~]# vzctl enter 10

view raw gistfile1.sh hosted with ❤ by GitHub

To stop, restart or destroy the container run:

	[root@ovz:~]# vzctl stop 10
	[root@ovz:~]# vzctl restart 10
	[root@ovz:~]# vzctl destroy 10

view raw gistfile1.sh hosted with ❤ by GitHub

To get a list of all containers execute:

	[root@ovz:~]# vzlist -a
	CTID NPROC STATUS IP_ADDR HOSTNAME
	10 16 running 10.24.6.230 haproxyvz.example.com

view raw gistfile1.ps1 hosted with ❤ by GitHub

All the configuration commands that we executed in the previous steps got saved in the following file:

	[root@ovz:~]# cat /etc/vz/conf/10.conf
	KMEMSIZE="14372700:14790164"
	LOCKEDPAGES="2048:2048"
	PRIVVMPAGES="65536:69632"
	SHMPAGES="21504:21504"
	NUMPROC="240:240"
	PHYSPAGES="0:unlimited"
	VMGUARPAGES="33792:unlimited"
	OOMGUARPAGES="26112:unlimited"
	NUMTCPSOCK="360:360"
	NUMFLOCK="188:206"
	NUMPTY="16:16"
	NUMSIGINFO="256:256"
	TCPSNDBUF="1720320:2703360"
	TCPRCVBUF="1720320:2703360"
	OTHERSOCKBUF="1126080:2097152"
	DGRAMRCVBUF="262144:262144"
	NUMOTHERSOCK="120"
	DCACHESIZE="3409920:3624960"
	NUMFILE="9312:9312"
	AVNUMPROC="180:180"
	NUMIPTENT="128:128"
	# Disk quota parameters (in form of softlimit:hardlimit)
	DISKSPACE="2G:2.2G"
	DISKINODES="200000:220000"
	QUOTATIME="0"
	# CPU fair scheduler parameter
	CPUUNITS="1000"
	VE_ROOT="/vz/root/$VEID"
	VE_PRIVATE="/vz/private/$VEID"
	OSTEMPLATE="centos-6-x86_64"
	ORIGIN_SAMPLE="basic"
	ONBOOT="yes"
	HOSTNAME="haproxyvz.example.com"
	IP_ADDRESS="10.24.6.230"
	NAMESERVER="8.8.8.8 8.8.4.4"
	NETIF="ifname=ethp2p1,bridge=br0,mac=00:18:51:13:AF:9F,host_ifname=veth101.2,host_mac=00:18:51:11:2D:6E"

view raw gistfile1.sh hosted with ❤ by GitHub

You can create this configuration file manually instead of using the OpenVZ userspace tools as well.

Experimenting with Ploop:

	# Upgraded /var/lib/vz from ext3 to ext4 with extents, as this is a requirement for ploop to work.
	root@db-566424-1:~# /etc/init.d/vz stop
	root@db-566424-1:~# umount /dev/sdb1
	root@db-566424-1:~# tune2fs -O extents,uninit_bg,dir_index /dev/sdb1
	root@db-566424-1:~# e2fsck -fDC0 /dev/sdb1
	root@db-566424-1:~# /etc/init.d/vz start
	root@db-566424-1:~# vzctl create 9999 --layout ploop --diskspace 5G --ostemplate f7382996-c96f-476e-b153-3935e84e6232
	root@db-566424-1:~# vzctl start 9999
	root@db-566424-1:~# ssh dbaas@172.16.255.100 -p16022 "createVolume clusterName=584027-cl volumeName=ploop_volume thinProvision=0 output=XML size=1GB"
	root@db-566424-1:~# ssh dbaas@172.16.255.100 -p16022 "assignVolumeToServer volumeName=ploop_volume serverName=db-566424-1 output=XML"
	root@db-566424-1:~# iscsiadm -m discovery -t sendtargets -p 172.16.255.100
	root@db-566424-1:~# iscsiadm -m node -T iqn.2003-10.com.lefthandnetworks:ords03:49813:ploop-volume -p 172.16.255.100:3260 --login
	root@db-566424-1:~# vzctl exec 9999 "mknod /dev/vdb b 8 176"
	root@db-566424-1:~# vzctl set 9999 --save --devices b:8:176:rw
	root@db-566424-1:~# ls -lah /var/lib/vz/private/9999/root.hdd/
	total 562M
	drwx------ 2 root root 4.0K Jan 26 16:26 .
	drwxr-xr-x 3 root root 4.0K Jan 26 15:59 ..
	rw-rr- 1 root root 790 Jan 26 15:59 DiskDescriptor.xml
	rw------ 1 root root 0 Jan 26 15:59 DiskDescriptor.xml.lck
	rw------ 1 root root 434M Jan 26 16:26 root.hdd
	root@db-566424-1:~#
	# Ploop Snapshots
	root@db-566424-1:~# vzctl snapshot 9999 --name snapshot_1 --description "First snapshot"
	root@db-566424-1:~# vzctl snapshot-list 9999
	root@db-566424-1:~# vzctl snapshot-switch 9999 --id 79affde2-eb31-4f93-9511-168b12ea62fa
	root@db-566424-1:~# vzctl snapshot-mount 9999 --id 79affde2-eb31-4f93-9511-168b12ea62fa --target /mnt/
	root@db-566424-1:~# vzctl snapshot-umount 9999 --id 79affde2-eb31-4f93-9511-168b12ea62fa
	root@db-566424-1:~# vzctl snapshot-delete 9999 --id 79affde2-eb31-4f93-9511-168b12ea62fa

view raw gistfile1.sh hosted with ❤ by GitHub

Resources: [1]. http://openvz.org/