We see that you are viewing the US version of our website from another country. If you would like to view a local version of our website, please click the link below...

Sales: 1-800-961-2888
Support: 1-800-961-4454
Login
  • MyRackspace Portal
  • Cloud Control Panel
  • Rackspace Webmail Login
Share this on:

Using a Custom Kernel with pv-grub

Article Id: 638 | Cloud Servers | Article


Contents

Obligatory Disclaimers

  • pv-grub is an advanced feature. It should only be used by experienced Linux administrators with a demonstrated need to run their own kernels.
  • Not all kernels are supported. Your kernel must have pv-ops support for running in a Xen domU. Also, kernels compressed with bzip or LZMA will not work.
  • Follow the instructions as listed, in the order listed. Do not blindly copy/paste these commands, as some of them will require editing.
  • Do not attempt these instructions on a production system. If they are applied incorrectly, your Cloud Server may become unbootable.

 

Grub configuration

Every instance booted with pv-grub requires a working grub configuration file in /boot/grub/menu.lst:

default=0
timeout=5
 
title=DISTRO-NAME "KERNEL VERSION STRING"
    root (hd0)
    kernel /boot/KERNEL-VMLINUZ-FILENAME ro console=hvc0 root=/dev/xvda1
    initrd /boot/KERNEL-INITRD-FILENAME

 

Distribution-specific adjustments

Arch

pacman -Sy pacman
pacman -Sy base-devel
pacman -Syu
sed -i 's/sda/xvda/' /etc/fstab
pacman -Sy kernel26
mkdir -p /boot/grub
cp /usr/src/KERNEL-VERSION/vmlinux /boot/vmlinux26

Note: The stock arch kernel is bzipped and won't work with pv-grub. Use the uncompressed vmlinux instead.

Now add (or uncomment) the following line in /etc/inittab for console:

h0:2345:respawn:/sbin/agetty -8 38400 hvc0 linux

CentOS 5.x

yum -y upgrade
mkdir -p /boot/grub
echo "alias eth0 xennet" > /etc/modprobe.conf
echo "alias eth1 xennet" >> /etc/modprobe.conf
echo "alias scsi_hostadapter xenblk" >> /etc/modprobe.conf
echo "xvc0" >> /etc/securetty
sed -i 's/sda/xvda/' /etc/fstab
yum -y install kernel-xen

Now add (or uncomment) the following line in /etc/inittab for console:

8:2345:respawn:/sbin/mingetty xvc0

Debian 5.x

apt-get update; apt-get -y upgrade
mkdir /boot/grub
sed -i 's/sda/xvda/' /etc/fstab
apt-get -y install linux-image-xen-amd64

Now add (or uncomment) the following line in /etc/inittab for console:

8:2345:respawn:/sbin/getty 38400 hvc0

Fedora 13

mkdir -p /boot/grub
echo "alias eth0 xennet" > /etc/modules
echo "alias eth1 xennet" >> /etc/modules
echo "alias scsi_hostadapter xenblk" >> /etc/modules
sed -i 's/sda/xvda/' /etc/fstab
echo "hvc0" >> /etc/securetty
yum -y install kernel

Gentoo

emerge --sync
emerge --update portage
emerge gentoo-sources genkernel #Update /usr/src/linux symlink if necessary
sudo sed -i 's/sda/xvda/' /etc/fstab
genkernel --menuconfig all

Enable the following in the kernel configuration:

Processor Type and Features -> Paravirtualized guest support -> "Y"
Hit enter, go to "Xen guest support" -> "Y"

Then exit the configuration, save, and build.

Now add (or uncomment) the following line in /etc/inittab for console:

h0:12345:respawn:/sbin/agetty 38400 hvc0 linux

RHEL 5.x

yum -y upgrade
mkdir -p /boot/grub
echo "alias eth0 xennet" > /etc/modprobe.conf
echo "alias eth1 xennet" >> /etc/modprobe.conf
echo "alias scsi_hostadapter xenblk" >> /etc/modprobe.conf
echo "xvc0" >> /etc/securetty
sed -i 's/sda/xvda/' /etc/fstab
yum -y install kernel-xen

Now add (or uncomment) the following line in /etc/inittab for console:

8:2345:respawn:/sbin/mingetty xvc0

Ubuntu 9.10/10.04

mkdir /boot/grub
cp /etc/init/tty1.conf /etc/init/hvc0.conf
sed -i 's/tty1/hvc0/' /etc/init/hvc0.conf
sed -i 's/sda/xvda/' /etc/fstab
apt-get -y install linux-virtual

Ubuntu 10.10

mkdir /boot/grub
cp /etc/init/tty1.conf /etc/init/hvc0.conf
sed -i 's/tty1/hvc0/' /etc/init/hvc0.conf
apt-get -y install linux-virtual

Kernel upgrade

Contact Rackspace Cloud support and request the pv-grub feature, informing them that you have completed these steps and would like to use your own kernel. Your Cloud Server will be rebooted, and -- if all goes well -- will come back up with your new kernel. You may verify this with 'uname -a'.

© 2011-2012 Rackspace US, Inc.

Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License


See license specifics and DISCLAIMER

Comments

Comment: 
Do you mean pygrub here? pv-grub is totally different than pygrub.

Comment: 
Nope, the article does mean pv-grub. We've started using pv-grub as the bootloader for newer instances (Fedora 15 and I think any we release after that). You can also ask support to switch an older instance to pv-grub - that situation is mainly what this article covers, talking about changes that should be made to the system when switching from using a kernel external to the instance to using one inside the instance using pv-grub.

Comment: 
In Archlinux, it's important to add "xen-blkfront xen-fbfront xenfs xen-netfront xen-kbdfront" to the /etc/mkinitcpio.conf MODULES section (or some combination therein, I just added every xen module), and then regenerate the initrd, or else it won't boot correctly :-)

Comment: 
Thanks Bryce. I haven't tried compiling my own kernel on Arch, I admit, but if you use the stock kernel package you can get away with skipping the mkinit stuff. You just have to make sure you use the uncompressed kernel instead of the bzipped one they install by default, since (last time I tried it) our Xen appears to have trouble with anything other than uncompressed or gzipped.

Comment: 

For latest stock kernel in Arch, I had to set barrier=0 in /etc/fstab e.g.:

/dev/xvda1 / ext3 defaults,barrier=0 0 1
/dev/xvda2 swap swap defaults,barrier=0 0 0

Otherwise, root fs will get mounted read-only very often with error messages like:

[ 8.389389] end_request: I/O error, dev xvda1, sector 1053144
[ 8.389406] end_request: I/O error, dev xvda1, sector 1053144
[ 8.389415] Buffer I/O error on device xvda1, logical block 131643
[ 8.389444] Aborting journal on device xvda1.
[ 8.390160] EXT3-fs (xvda1): error: ext3_journal_start_sb: Detected aborted journal
[ 8.390178] EXT3-fs (xvda1): error: remounting filesystem read-only

Rackspace Comment: 

Curious. Thanks, Santo. We'll look into that and get the article updated appropriately.

Comment: 

This is also needed in gentoo 3.1+ kernels.

Comment: 

Hi Jered,

I second Santo's comment. I just got linux 3.1 to install on my node today, and only by setting barrier=0 was I able to make any writes to disk. There may be a better way to make the disk writable, however I am unaware of it.

Also, a note: The uncompressed image does _not_ work with linux 3.1, while the vmlinuz-linux bzImage (installed by the Arch package "linux") _does_ work (my node is currently running on vmlinuz-linux; however, it kernel panicked with the uncompressed image).

Thanks!

Matt

Comment: 

Additional note: I found the Arch wiki's Xen page to be extremely helpful, in addition to this page.

Comment: 

I would like to see a complete step-by-step for Ubuntu, with an explanation of each step. The procedure offered in the article is very abbreviated and contrast rather sharply with the one at http://failverse.com/setting-up-pv-grub-cloud-servers/

Why not just have pv-Grub enabled for instances by default? Some type of failover is needed to boot the original virtual image if the customer/new kernel does not boot.

Rackspace Comment: 

Our newer images actually do use pv-grub by default now, so as new versions are released for various distributions you should see them all with pv-grub support right off the bat.

We have to port this article over to Rackspace Cloud still, but you can follow the instructions in this Slicehost article since the procedure is the same:

http://articles.slicehost.com/2010/10/27/using-pv-grub-to-run-custom-ker...

Access to the command console and rescue mode would be done through the RS control panel, of course, but otherwise the instructions should still apply.

Add new comment

©2012 Rackspace, US Inc. About Rackspace | Fanatical Support® | Hosting Solutions | Investors | Careers | Privacy Statement | Website Terms | Sitemap