I've been playing around with the EC2 service at Amazon and figured I would document a little about how you create your own FC6 AMI. The Amazon documentation goes over everything you need to know about creating your own FC4 AMI and if you don't want to roll your own you can use one of the public AMIs. Amazon just started letting people publish their own AMIs on their site so you should expect to see more as time goes by.
The first step of course is to have an EC2 enabled account. If you haven't already signed up for one there are more beta openings available (as of 01/10/07) so you may still be able to get one. You will also need to be signed up for S3. Once you do that it is helpful to read the getting started guide and try out a few of the public AMIs. Doing so will get you to get your keys set up for S3, EC2, and SSH. In the following I assume you have read and followed the getting started guide and have set up all the keys you will need for S3, EC2, and SSH.
Creating your FC6 image
Here are the steps you need to create your FC6 image. Two notes before getting started: 1) I am using an FC6 box to run the following commands on so your luck may vary with older system and 2) Some of these can be done as a non-root user but you might as well be root for all of them.
If you are in a hurry you may download all of the following steps in a single script that will generate the custom bootable AMI.
1) Create the image file and initialize the filesystem on it (note that I'm only making giving myself 1G of space for this install, if you think you will need more room you should create a larger file by changing the seek value):
dd if=/dev/zero of=fc6-i386.img bs=1M count=1 seek=1024 /sbin/mke2fs -F -j fc6-i386.img
2) Mount the file with a loopback device:
mount -o loop fc6-i386.img /mnt
3) Create base directories and device files:
mkdir /mnt/dev mkdir /mnt/proc mkdir /mnt/etc for i in console null zero ; do /sbin/MAKEDEV -d /mnt/dev -x $i ; done
4) Create the initial fstab file:
cat <<EOL > /mnt/etc/fstab /dev/sda1 / ext3 defaults 1 1 none /dev/pts devpts gid=5,mode=620 0 0 none /dev/shm tmpfs defaults 0 0 none /proc proc defaults 0 0 none /sys sysfs defaults 0 0 /dev/sda2 /mnt ext3 defaults 1 2 /dev/sda3 swap swap defaults 0 0 EOL
5) Mount the proc under the new root filesystem so yum will work correctly:
mount -t proc none /mnt/proc
6) Create your a yum configuration file:
cat <<EOL > /tmp/yumec2.conf [main] cachedir=/var/cache/yum debuglevel=2 logfile=/var/log/yum.log exclude=*-debuginfo gpgcheck=0 obsoletes=1 reposdir=/dev/null [base] name=Fedora Core 6 - i386 - Base mirrorlist=http://fedora.redhat.com/download/mirrors/fedora-core-6 enabled=1 [updates-released] name=Fedora Core 6 - i386 - Released Updates mirrorlist=http://fedora.redhat.com/download/mirrors/updates-released-fc6 enabled=1 EOL
7) Run yum to install the base group of packages to your root filesystem (this may take some time but you should see it progress, I have had all kinds of trouble with yum in the past so if it hangs you may want to kill it and try again):
yum -c /tmp/yumec2.conf --installroot=/mnt -y groupinstall Base
8) Clean the yum cache:
yum -c /tmp/yumec2.conf --installroot=/mnt -y clean packages
9) Move the TLS directory out of the way:
mv /mnt/lib/tls /mnt/lib/tls-disabled
10) Modify the boot script to download your SSH key and stick it in root's directory:
cat <<EOL >> /mnt/etc/rc.local
if [ ! -d /root/.ssh ] ; then
mkdir -p /root/.ssh
chmod 700 /root/.ssh
fi
# Fetch public key using HTTP
curl http://169.254.169.254/1.0//meta-data/public-keys/0/openssl > /tmp/my-key
if [ $? -eq 0 ] ; then
cat /tmp/my-key >> /root/.ssh/authorized_keys
chmod 600 /root/.ssh/authorized_keys
rm /tmp/my-key
fi
# or fetch public key using the file in the ephemeral store:
if [ -e /mnt/openssh_id.pub ] ; then
cat /mnt/openssh_id.pub >> /root/.ssh/authorized_keys
chmod 600 /root/.ssh/authorized_keys
fi
EOL
11) Set sshd to allow remote root connections and now hang on DNS problems:
cat <<EOL >> /mnt/etc/ssh/sshd_config UseDNS no PermitRootLogin without-password EOL
12) Create the networking scripts:
cat <<EOL > /mnt/etc/sysconfig/network NETWORKING=yes HOSTNAME=localhost.localdomain EOL cat <<EOL > /mnt/etc/sysconfig/network-scripts/ifcfg-eth0 ONBOOT=yes DEVICE=eth0 BOOTPROTO=dhcp EOL
13) Sync and umount your root filesystem:
sync umount /mnt/proc umount /mnt
You have now created your very own bootable AMI. If you want to fiddle with it from this point you may continue to use the yum command as in the above examples or you can also remount the filesystem and chroot to it using a command like this:
chroot /mnt /bin/sh
One thing to remember if you use chroot like this is that everything is local now. You will want to mount the proc filesystem and probably add entries to /etc/resolve.conf so any hostnames you try to resolve will work.
The next step is to get the AMI to S3 so that it can be booted.
Bundling and Uploading your AMI
Everything you need to know about bundling and uploading your custom AMI is in the developer documentation under "Working With AMIs" then "Bundling an AMI".
One key to remember here is that you need to start your instance with the -k option to allow the key to be copied into place. If you don't do that or specify the incorrect key name you will end up with an instance you can't log into.
Thanks! I was just flailing at this. Just to make sure I wasn't screwing anything up, I used the script; but, I got similar results. Some directories don't get created, like tls and most etc directories (only fstab and rc.local are in etc).
I'm sure it's some newbie mistake and I'm not sure you have time to cater to me. Either way, thanks for doing the work to document and script all these steps.
Awesome guide! I'll be putting it to use for one my projects very soon. :)
Pingback: Debian EC2 AMI @ IONCANNON
There are apparently some configuration steps that are included with the groupinstall base. I've tried to selectively add just a few packages (namely httpd and openssh-server), but I can't boot the AMI in Xen or EC2.
In Xen I am getting a "can't mount /dev/sda1" error, although the device was created using MAKEDEV.
Maybe something is screwy with my fstab.
You don't need to move the TLS directories out of the way, that error message about nosegneg is an error. FC6 by default includes nosegneg-friendly libraries and that error message within the Xen Hypervisor will be fixed with subsequent versions.
I get the following error, anyone know the solution?
Failed to add groups file for repository: base
Error: No Groups on which to run command
Pingback: How to create a Fedora 7 Instance for EC2 @ IONCANNON
One thing that seems to have bitten quite a few people in both FC6 and FC7, is kudzu move the ifcfg-eth0 file to ifcfg-eth0.bak if an instance is rebooted. This causes networking not to start up on reboots after the initial launch. kudzu needs to be disabled to avoid this breakage.
Some other niceties would be:
a) disabling IPv6 since EC2 won't route it anycase,
b) disabling zeroconf networking since you'll always be getting an IP from EC2.
c) disabling services such PC Card, Bluetooth, gpm, CUPS, etc that don't really make sense in an EC2 environment.
Hi,
I am creating an instance using 'loop back' file, I created my image file – but I'm getting the following error while bundling the Image.
[root@ec2 home]# ec2-bundle-image -i my-image.fs -k /home/.ec2/pk-xxx.pem -c /home/.ec2/cert-yyy.pem -u 1111111 -r i386 -p my-image -d /home/bundle
Splitting /home/bundle/my-image.tar.gz.enc…
Created my-image.part.00
Created my-image.part.01
Created my-image.part.02
Created my-image.part.03
Created my-image.part.04
Created my-image.part.05
Created my-image.part.06
Created my-image.part.07
Created my-image.part.08
Created my-image.part.09
Created my-image.part.10
Created my-image.part.11
Created my-image.part.12
Created my-image.part.13
Created my-image.part.14
Created my-image.part.15
Created my-image.part.16
Created my-image.part.17
Created my-image.part.18
Created my-image.part.19
Created my-image.part.20
Created my-image.part.21
Created my-image.part.22
Created my-image.part.23
Created my-image.part.24
Created my-image.part.25
Created my-image.part.26
Created my-image.part.27
Created my-image.part.28
Created my-image.part.29
Created my-image.part.30
Created my-image.part.31
Created my-image.part.32
Created my-image.part.33
Created my-image.part.34
Created my-image.part.35
Created my-image.part.36
Created my-image.part.37
Created my-image.part.38
Generating digests for each part…
Digests generated.
Creating bundle manifest…
Error: private method `gsub' called for 39:Fixnum
ec2-bundle-image failed.
Any Help – —
Ram
@richard shade if you use the fedora core 4 provided by amazon and try to do a yum groupinstall Base, you will get this error because yum 2.4.1 doesn't support this option. You should install a newer version of yum before trying to install the "Base".
Let's try that first
wget http://linux.duke.edu/projects/yum/download/3.0/yum-3.0.5.tar.gz
tar -xvzf yum-3.0.5.tar.gz
cd yum-3.0.5
make DESTDIR=/ install
then you are set to install the base
How do you debug this distro.
How do you add a stack like the Ruby on Rails stack (Apache/NGINX, Mongrel, Ruby, Rails, mySQL) ?
I have a vmware image that I'm trying to upload to EC2. Would I just cp -R / /mnt/ all the files in it to a disk image mounted like above? I guess I'll go try and update everybody else…
I've been trying to create an image for CentOS for amamon EC following the steps provided above as well as the script provided by RightScale (http://blog.rightscale.com/2007/04/). In both the cases when I try and do a groupinstall of the Base I get the following below. I'm running the above steps from a machine already running CentOS 5 and version of yum is 3.0.5 as well. Any pointers will be appreciated.
Setting up repositories
Reading repository metadata in from local files
Traceback (most recent call last):
File "/usr/bin/yum", line 29, in ?
yummain.main(sys.argv[1:])
File "yummain.py", line 94, in main
result, resultmsgs = base.doCommands()
File "cli.py", line 381, in doCommands
return self.yum_cli_commands[self.basecmd].doCommand(self, self.basecmd, self.extcmds)
File "yumcommands.py", line 228, in doCommand
return base.installGroups(extcmds)
File "cli.py", line 1071, in installGroups
self.doRepoSetup()
File "cli.py", line 109, in doRepoSetup
self.doSackSetup(thisrepo=thisrepo)
File "__init__.py", line 341, in doSackSetup
File "packageSack.py", line 331, in excludeArchs
File "sqlitesack.py", line 589, in excludeArchs
File "sqlitesack.py", line 431, in db2class
File "/usr/lib/python2.4/site-packages/sqlite/main.py", line 97, in __getattr__
raise AttributeError, key
AttributeError: CHECKSUM_VALUE