Link-LXC-USERCR

From Linux Checkpoint / Restart Wiki
Revision as of 21:12, 14 April 2010 by Ntl (Talk | contribs)

Jump to: navigation, search

Contents

Overview

The following instructions describe the process to build/install components necessary to checkpoint/restart (C/R) LXC containers using the C/R implementation being pushed into mainline.

The three main components to be built/installed are:

  • C/R enabled Linux kernel
  • USERCR - the user-space component of checkpoint/restart
  • LXC

Note: This page is still under construction. Feel free to fix obvious mistakes or email sukadev@linux.vnet.ibm.com.

Basic debugging of C/R

There is a strong dependency between each of these components (USERCR and Linux kernel are tightly coupled as are USERCR and LXC) so it is important to ensure the versions match up correctly. Otherwise attempts to checkpoint/restart typically fail with terse -EINVAL or -EBUSY errors.

If this happens, usually more debug information is found in dmesg output. If lxc-checkpoint fails some additional error information can be found by running

  • $ /bin/ckptinfo -ev <checkpoint-statefile>

Terminology

Statefile / checkpoint-image - this refers to the file in which the application state is saved after a successful checkpoint.

LXC - This refers to implementation of linux containers. Specifically this refers to the git tree git://lxc.git.sourceforge.net/gitroot/lxc/lxc.

USERCR - The user-space commands/library components that utilize the kernel system calls to checkpoint/restart applications. Specifically this refers to the git tree: git://git.ncl.cs.columbia.edu/pub/git/user-cr.git

1. Build C/R-enabled Linux kernel

Get the C/R-enabled kernel sources

   $ cd /root
   $ git clone  git://www.linux-cr.org/pub/git/linux-cr.git linux-cr
   $ cd linux-cr
   $ git checkout -b test-1 0fdca57255b8b5bc9a4f107bee7f1e47d2630cb3
   $ git log --pretty=oneline HEAD^..
   0fdca57255b8b5bc9a4f107bee7f1e47d2630cb3 Disable softirqs when taking the socket

Ensure following tokens are set in .config

  CONFIG_CHECKPOINT_SUPPORT=y
  CONFIG_SYSVIPC_CHECKPOINT=y
  CONFIG_CHECKPOINT=y
  CONFIG_CHECKPOINT_NETNS=y
  CONFIG_CHECKPOINT_DEBUG=y
  CONFIG_CGROUPS=y
  CONFIG_CGROUP_FREEZER=y
  CONFIG_NAMESPACES=y
  CONFIG_CGROUP_NS=y
  CONFIG_UTS_NS=y
  CONFIG_IPC_NS=y
  CONFIG_USER_NS=y
  CONFIG_PID_NS=y
  CONFIG_NET_NS=y
  CONFIG_FREEZER=y

To work around a build issue with unexported symbols in this kernel version, ensure that CONFIG_IPV6 is either y or n, not m.

Build, install, reboot on new kernel

After every reboot, ensure '-o newinstance' mount option to /dev/pts works (see Documentation/filesystems/devpts.txt for details). In short, run following commands on each reboot:

  $ rm /dev/ptmx
  $ ln -s pts/ptmx /dev/ptmx
  $ chmod 666 /dev/pts/ptmx

2. Build/install USERCR

Get latest USERCR sources

   cd /root
   $ git clone git://git.ncl.cs.columbia.edu/pub/git/user-cr.git user-cr
   $ cd user-cr
   $ git checkout -b test-1 fa382a1758ef395858b1bea530ccdb4e1360b76f
   $ git log --pretty=oneline HEAD^..
   fa382a1758ef395858b1bea530ccdb4e1360b76f Add keep_frozen field to struct app_restart_args

Apply any USERCR patches

If any patches are needed for the USERCR tree, they will be available at: http://lxc.sourceforge.net/patches/lxc+usercr/usercr-patches/.

Get those patches, and use, say, git am -3 to apply each of the patches in the directory to the USERCR tree.

Build and install USERCR binaries

Build USERCR by pointing to corresponding kernel-source tree we used above and install binaries. This should create restart.o and checkpoint.o needed by LXC, which, for now, are left in the current directory.

Note: You may need to compile checkpoint.o and restart.o with -fPIC compiler option.

   $ KERNELSRC=/root/linux-cr make 
   $ ls restart.o checkpoint.o
   restart.o   checkpoint.o
   $ make install

3. Build/install LXC

Get latest LXC sources

   $ cd /root
   $ git clone git://lxc.git.sourceforge.net/gitroot/lxc/lxc lxc.git
   $ cd lxc.git
   $ git checkout -b test-1 f78a1f32f41f6acbbf0b78e6498736dbd22e2301
   $ git log --pretty=oneline HEAD^
   f78a1f32f41f6acbbf0b78e6498736dbd22e2301 fix when console is not specified

Apply LXC patches

Get the patch set from http://lxc.sourceforge.net/patches/lxc+usercr/lxc-patches/ Apply each of the patches in the directory to the LXC git tree

   $ git am -3 lxc-patches/0001-patch
   $ git am -3 lxc-patches/0002-patch
   etc

Configure, build, install LXC binaries

Configure/build/install LXC binaries using following commands. The configure command will fail if /root/user-cr specified below, does not contain checkpoint.o, restart.o and app-checkpoint.h files

   $ autogen.sh
   $ ./configure --with-libcr=/root/user-cr
   $ make
   $ make install

4. Checkpoint/restart a simple LXC container

   $ lxc-execute --name foo --rcfile lxc-no-netns.conf -- /bin/sleep 1000
   $ lxc-checkpoint --name foo --statefile /root/lxc-foo.ckpt
   $ lxc-stop --name foo
   $ lxc-restart --name foo --statefile /root/lxc-foo.ckpt
   $ lxc-stop --name foo

5. Checkpoint/restart an LXC container running a VNC server

VI editing session in a VNC session

Run a "vi" editing session inside a VNC server using "twm" window manager

   $ cat /root/.vnc/xstartup
   #!/bin/sh
   xsetroot -solid grey
   vncconfig -iconic &
   xterm -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &
   twm &
   $ lxc-execute --name foo --rcfile lxc-no-netns.conf -- /usr/bin/vncserver :1
   $ vncviewer :1
   # Open a vi session in vnc viewer
   $ lxc-checkpoint --name foo  --statefile /root/vnc.ckpt
   $ lxc-stop --name foo
   $ lxc-restart --pause --name foo --statefile /root/vnc.ckpt
   # Leaves the server frozen due to --pause
   $ lxc-unfreeze --name foo
   $ vncviewer :1
   # Should bring up the old VNC session with vi window
Personal tools