You are here

Archive for No name

Installing Linux VMs under KVM with cobbler and koan

I spent my weekend working on getting cobbler and koan working on a new server. This server will serve several virtual machines under KVM and I want a way to automatically reprovision VMs while I am working on tuning and testing.

cobbler "is a Linux installation server that allows for rapid setup of network installation environments." It allows for easy management of kickstart scripts, DHCP, and other services needed for provisioning new machines. koan, which stands for "kickstart-over-a-network," is a client for cobbler and is used "for reinstallation and virtualization support."

Directions on how to install cobbler can be found on the site for cobbler. In a nutshell, Fedora users can install it directly from the repository servers and RHEL and CentOS users can install it from the EPEL and EPEL testing repositories. (For this, I am using CentOS 5.4 and the version of cobbler and koan from the EPEL testing repository. cobbler was originally installed from the EPEL repository so this may create some oddities in behavior.) To support some functionality, you will want to install the qspice-libs and yum-utils packages.

For cobbler to work correctly under SELinux, cobbler check suggests the following changes:

/usr/sbin/setsebool -P httpd_can_network_connect true
/usr/sbin/semanage fcontext -a -t public_content_t "/var/lib/tftpboot/.*"
/usr/sbin/semanage fcontext -a -t public_content_t "/var/www/cobbler/images/.*"

This allows the Apache httpd server to connect to the network and assigns the public_content_t file context to files in /var/lib/tftpboot/ and /var/www/cobbler/images/. If the standard firewall is installed, traffic will need to be allowed to TCP ports 80 and 26161. cobbler check will also recommend enabling TFTP and allowing traffic to UDP port 69 but these are not required for provisioning virtual machines with koan. You should also review the settings in /etc/cobbler/settings. At a minimum, the server and next_server settings need to be changed. (If you're using KVM, you probably want to change the default_virt_bridge and default_virt_type settings as well.)

To start working with cobbler, a distribution needs to be created. The man page supports importing a repository. However, this may originally take a while since the base repository tends to be large. (The x86_64 repository for CentOS 5.4 is about 4.4 GB.) An example is:

cobbler import --path=rsync://ftp.linux.ncsu.edu/CentOS/5/os/x86_64/ --name=centos5 --arch=x86_64

This will create a distribution named centos5-x86_64. (You will want to use a mirror local to you.)

cobbler allows identifying other repositories that will be mirrored locally and used for the build process via the repo kickstart option. If you want to install with the newest packages available, you will want to add the updates repository, like so:

cobbler repo add --arch=x86_64 --name=centos5-updates-x86_64 \
    --mirror=rsync://ftp.linux.ncsu.edu/CentOS/5/updates/x86_64/

All this will do is tell cobbler to mirror the repository locally. In order to actually set up the mirror, you will need to run cobbler reposync.

Next, identify an installation profile. This is usually tied to a specific server role. Here, we'll define a profile for a VM for a DNS server:

cobbler profile add --name=centos5-vm_dns --distro=centos5-x86_64 --virt-ram=512 \
    --virt-type=qemu --virt-cpus=1 --repos="centos5-updates-x86_64" \
    --kickstart=/var/lib/cobbler/kickstarts/vm-dns.ks --kopts="serial console=ttyS0,115200" \
    --kopts-post="console=ttyS0,115200"

This specifies that the profile should use the CentOS 5 distribution and the CentOS 5 updates repository created above, configure it to run as a VM under KVM, use 512 MB of RAM, and one virtual CPU. The kickstart file template, which has already been created, resides under /var/lib/cobbler/kickstarts/vm-dns.ks. (Information on setting up the kickstart template can be found on the cobbler website.) The kopts setting specifies that kickstart should be started with the serial and console settings. The kopts-post setting specifies that the installed machine should have the console kernel option defined. (These are needed so that the VM makes a console available that can be used via virsh console.)

Usually each profile will have multiple systems under it. A system corresponds to a machine, physical or virtual. (There is still some benefit in using cobbler when there is only one system per profile like I'm doing on my testing server. However, it will really shine when you have to install many mostly identical machines.) A system is defined in the following manner:

cobbler system add --name example-dns --profile=centos5-vm_dns \
    --ip=192.168.122.3 --gateway=192.168.122.1 --subnet=255.255.255.0 \
    --hostname=dns.example.com --static=1

This creates a system named dns.example.com with the static IP 192.168.122.3.

To install a virtual machine, you use koan like so:

koan --server=192.168.122.1 --virt --system=example-dns --virt-path=/dev/mapper/examplevg-dns

This will install a VM using the example-dns system defined above to /dev/mapper/examplevg-dns, a logical volume created via LVM. This should work properly if using a file or a normal partition.

If you've used virt-install, you're probably used to seeing the console opened immediately once the VM is started. koan does not do this. To see the VM, you will need to open the console manually via virsh console. (For example, to see the console for the VM being created by the koan statement above, use virsh console example-dns.)

A few issues I encountered were:

  • Error message: libvir: QEMU error : internal error cannot parse QEMU version number in ''
    As mentioned here, the error is corrected by installing the qspice-libs package.
  • Error message:
    libvir: QEMU error : internal error unable to start guest: qemu: could not open disk image /dev/mapper/examplevg-dns

    Also, errors like these appear in the audit log:

    type=AVC msg=audit(1266799848.453:623): avc:  denied  { getattr } for  pid=32208 
    comm="qemu-kvm" path="/dev/mapper/examplevg-dns" dev=tmpfs ino=78122 
    scontext=system_u:system_r:qemu_t:s0-s0:c0.c1023 
    tcontext=system_u:object_r:fixed_disk_device_t:s0 tclass=blk_file
     
    type=AVC msg=audit(1266799848.453:624): avc:  denied  { read } for  pid=32208 comm="qemu-kvm" 
    name="examplevg-dns" dev=tmpfs ino=78122 scontext=system_u:system_r:qemu_t:s0-s0:c0.c1023 
    tcontext=system_u:object_r:fixed_disk_device_t:s0 tclass=blk_file

    The solution to this is to add the virt_image_t context to the block file with:

    chcon -t virt_image_t /dev/mapper/examplevg-dns

    I tried to set this with semanage but was unsuccessful. This probably means that I need to look at the documentation better.

    This is apparently not an issue in Fedora 11 and later and in the upcoming RHEL 6 due to sVirt. In a mailing list post, Daniel Walsh states that they hope to get this ported into RHEL 5.6.

  • Despite the server and next_server settings in /etc/cobbler/settings, koan still tries to install using URLs referencing 127.0.0.1.

    I don't really know the cause of this. It seemed odd but I couldn't trace it. For now, I used a workaround by setting the server for the profile manually with --server=192.168.122.1.

    It could either be that there is something strange with my setup or that this is intended.

  • Error when running koan:
    exceptions.TypeError
    cannot concatenate 'str' and 'dict' objects
      File "/usr/lib/python2.4/site-packages/koan/app.py", line 215, in main
        k.run()
       File "/usr/lib/python2.4/site-packages/koan/app.py", line 329, in run
        self.virt()
       File "/usr/lib/python2.4/site-packages/koan/app.py", line 652, in virt
        return self.net_install(after_download)
       File "/usr/lib/python2.4/site-packages/koan/app.py", line 571, in net_install
        after_download(self, profile_data)
       File "/usr/lib/python2.4/site-packages/koan/app.py", line 650, in after_download
        self.virt_net_install(profile_data)
       File "/usr/lib/python2.4/site-packages/koan/app.py", line 1103, in virt_net_install
        kextra                        = self.calc_kernel_args(pd)
       File "/usr/lib/python2.4/site-packages/koan/app.py", line 1050, in calc_kernel_args
        kextra = kextra + " " + options

    I'm not certain why I saw this particular error. My solution was to comment out line 1050 and add this line in its place:

    kextra = kextra + " " + utils.hash_to_string(options)

    If I remember, I'll post to the cobbler mailing list.

    Update: I opened ticket #576 for this issue. My email to the cobbler-devel mailing list includes a patch.

If this looks interesting, I definitely suggest looking at the cobbler website. If you install many machines, you may also be interested in looking at the web interface which may make this task even easier.

So what's it good for?

I've discussed why I would find an iPad useful. However, what are some other scenarios?

Sam Ruby presents the idea of using an iPad as a wireless display for a development machine. (This obviously depends on the quality of the VNC or other remote connect client.) It doesn't allow for moving toward independent development, which I think Tim Bray was getting at in his post "Nothing Creative" but it does allow for a reasonable facsimile. It remains to be seen whether or not the iPad would be useful as a remote terminal for a development machine.

One of my friends is working on her Master's thesis. To this end, she has had to go to several herbariums to measure specimens. Her body of research is kept on a large, heavy laptop on which she keeps the rest of her life. Traveling with the laptop is always a concern because of how important it is and how difficult transport is. Since all of her data lives in a spreadsheet currently, she could copy that to the iPad and use the Numbers app to make the changes to her spreadsheet to support added or changed data points. This would let her leave her laptop in the hotel room, or even home, keeping it safe and out of harm's way.

Joseph Kim presents ten ways that the iPad could help doctors improve patient care. I'll personally have to use one to see if I, as a patient, prefer filling out forms on an iPad rather than using paper forms on a clipboard but I can definitely see the use here. And as others see the use, it will evolve over time.

I had a realization a few days ago. If we measure the iPad purely on the scale of the clock speed of its processor (this is silly, I know), we have a handheld device with a 1 GHz CPU. PCs of this speed weren't common until 2001. (This is based on personal experience and may be slightly wrong.) According to Wikipedia, version 6.0 of Adobe Premiere Pro was released in January 2001. So the only thing preventing us from making movie content on the iPad is the lack of an application. Such an application would probably only handle editing and the encoding of raw video would be offloaded to a proper computer. It's possible we won't see anything like this on the first iteration of the iPad but we may see it in a future product generation.

I wish I could remember the exact line but in one of the episodes of the acclaimed Connections mini-series, James Burke says something like "This is one of those times where the possibilities are virtually limitless." (I wish I could remember the exact topic he's speaking of at that time too so I could refer to it and someone would correct me.) I think, I feel, that the iPad will revolutionize the way we do business, the way we use the internet, but it will take us several years to realize fully all of the possibilities because they could really be virtually limitless.

But it is not, does not...!

I've heard a lot of complaints about the iPad. Some people can't even understand why I'd want one. So I figured I'd address that.

At RoughlyDrafted Magazine, Daniel Eran Dilger is doing a series of posts debunking myths about the iPad. I'm going to try not to repeat what he's written too much since he's doing a better job than I could. However, here's a few I'll address. (He does cover two issues I commonly hear that I've omitted here: "It's just a big iPod Touch." and "It's just another Kindle.")

The bevel's too big.

Well, first, how big is the bevel? The technical specifications for the iPad state that the iPad is 9.56 inches high, 7.47 inches wide, and has a 9.7 inch diagonal screen. Since the aspect ratio of the screen is 4:3, the screen should be 7.76 inches high and 5.82 inches wide. This leaves 1.8 inches on the height and 1.65 on the width for the bevel. Divide by two and this suggests an average bevel of 0.9 inches for the top and bottom and 0.825 inches for the left and right sides.

The iPad is designed as a handheld device. Unlike a laptop, there is nothing to support it but the user's hands. Unlike a phone or other small device, it cannot fit within the palm of the user's hand. So it must be held by the user much like one would hold a clipboard or a book. To hold it, the thumb has to have a place on top of the device.

The thumb rests in the bevel. Under the assumption that my hand is average-sized, I measured the width of my thumb. My thumb is about 0.875 inches across. This correlates roughly with the size of the bevel. If the bevel were smaller, the thumb would obscure the screen.

The 4:3 aspect ratio is wrong.

This depends a lot on what you're going to use the device for. As pointed out at The Unofficial Apple Weblog, the 4:3 aspect ratio is the standard ratio for just about everything but video. Trade paperbacks, such as System Performance Tuning, are 9 inches high by 7.5 inches wide, 4:3. The PDFs generated by The Pragmatic Bookshelf for their books, e.g. the one for Metaprogramming Ruby, specify these dimensions.

Using a 16:9 aspect ratio for video would also increase the size of the device. In order for the screen to be the same width (5.82 inches), it would need to be about 10.35 inches long. Adding in the bevel and it'd be about 12.15 inches high, making it a slightly odd size. It would be tailored well for showing video but not for much else.

You can't make phone calls with it.

This seems to come from a belief that the iPad is supposed to replace the iPhone. It's not. In the keynote speech, Steve Jobs says that the iPad is intended for a role between the smartphone (i.e. the iPhone) and the laptop.

So since the iPad is not a smartphone, why should it be set up for making phone calls? The consumer presumably already has some sort of phone already that they will retain.

It doesn't support Flash.

No. And why does it need to? The iPad will come with a native YouTube app. While I agree that it would be nice to see some other flash animations, I don't see it as a necessity.

Daniel Eran Dilger addresses this some. He makes one point that bears reiterating: "Flash is the primary reason Safari crashes, and even accounts for the vast majority of Apple’s Mac OS X crash reports." If your primary goal is to provide a stable, usable device, then providing a technology that has been shown to cause instability is a non-optimal choice.

One point that he does not make is: Adobe Flash Player is itself insecure. In 2009, Adobe released five security advisories for the Flash Player, each one identified by Adobe as critical and each one possibly allowing an attacker to take control of the system. A significant amount of malware, such as Gumblar, has been distributed via these vulnerabilities in the Flash Player. Given this track record, would you want Flash on anything that should be secure?

It doesn't support multitasking.

There is some merit to this. Under the current version of the iPhone OS, when you switch away from an application, it saves its state and stops. This means that network connections are closed. If you have an SSH or VNC client running and switch applications, you then have to reconnect to the server. I can see this being annoying.

However, for most other use cases, I'm not certain this is a huge deal. Between push notifications and the available applications, this no longer seems to be an issue for instant messaging. This may be handled for most other tasks for which you would want to have a background network application running.

There's a lot of complaining about a lack of active multitasking, i.e. having multiple applications shown on the screen simultaneously. For a screen of the size of the iPad, this seems like a bad idea. While the resolution of the screen is 1024x768, it's only 7.76 inches high by 5.82 inches wide. This means there are about 132 pixels per inch. On a 15 inch diagonal screen, this was only 85 pixels per inch. So while the resolution is the same, the density is much higher and it becomes harder to have multiple windows displayed comfortably. Thinking back on that 15 inch monitor with windows overlapping windows and having to switch to check on their status (something that even plagues me on newer monitors), I am reminded of this passage from Frederick Brooks' essay "No Silver Bullet":

The so-called "desktop metaphor" of today's workstation is instead an "airplane-seat" metaphor. Anyone who has shuffled a lap full of papers while seated between two portly passengers will recognize the difference--one can see only a very few things at once.

If the desktop metaphor cannot be achieved on a physically larger monitor (or even on today's much larger monitors with much larger resolutions), how can it possibly be achieved on a smaller screen? Showing only one application, while understandably limiting, prevents the sort of madness multiple windows would create.

It's not perfect

No, it's not. But is anything?

Why I want an iPad

Unless you've been living under a rock, you know that Steve Jobs and Apple have announced the iPad, the long rumored Apple tablet. Name jokes aside, I think we're seeing the beginning of a revolution.

The iPad is the first piece of technology in my lifetime that I am honestly looking forward to. I think it is going to be the beginning of a revolution. I don't know where that revolution will lead or what it will mean to me as someone used and reasonably well-adapted to the current era of computers. It will certainly be interesting.

I have a lot of what I want to say about the iPad which I'll spread out over a handful of entries. Today, I'll start with why I want one, how I think I would use it.

My commute to work each day is just over an hour each way. Even though I have a laptop (an aging iBook G4), I find it difficult to do significant work during the commute. Mass transit methods are not really set up for using a laptop. When you fly on a plane, when you take a train somewhere, you have a tray table on which you can put a laptop. You don't have that on a bus, in the subway, carpooling, etc.

So I spend a lot of my time reading. (Or sleeping.) It's easy to bring a book and read it. However, if what I need to read is only available in electronic form, it becomes necessary to print it out. One of my new projects is to replace my aging webserver (which happens to host this site and others) so I have been reading server security documents. I spent $130 to have FedEx Office (formerly Kinkos) print out some of these documents.

And since I've done this once, I will likely do it again. If nothing changes, some day, another $130 will be spent to have more documents printed. If I do this three more times, the cost comes to $520, more than the cost of a base iPad. This certainly makes it cost effective, not including the definite benefits that the iPad provides over several binders of documents.

I also know in the future that I will be expected to start carrying a laptop on my commute. Between the size of most laptops and the limitations of my laptop bag, this will limit what I can carry with me. After my notebook, I can probably only carry one book. Or I can carry an iPad. Based on Apple's announced dimensions, the iPad is slightly thinner than Web Design For Developers (which is about 0.6875 inches thick). With a case, it's probably thicker but probably not more than an inch thick. Even though I like books, there is a definite sense of economy here. (This assumes that the iPad will be able to use PDFs. However, since the iPhone can, the iPad should be able to as well. And since there is so much content available as PDFs, I think that it would be a mistake for PDF support to be omitted. However, what I think the iPad will/should do is really something for another discussion.)

A 3G iPad would provide me with something I don't currently have: An internet connection usable during the commute (or, theoretically, even while traveling). (I absolutely despise doing web browsing on the Blackberry I have.) There is definite appeal in having a reliable SSH client to fix problems that arise while I'm out. Being able to check blog entries or other resources to help me research whatever project I'm working on would help. Also, to add to the sense of economy above, I know that the iPhone can use the mobile interface for Safari Books Online so the iPad should be able to as well, something my current phone cannot do.

And I find that as I think about the iPad more, I can find more uses for it for me. I don't know that I would call it "magical" but it certainly has the potential to be revolutionary. And just like the iPhone changed things, the iPad certainly will.

Changes to the site

Saturday night, I finally updated the site to Drupal 6. While I was doing this, I made a few other changes. If you read this purely through an RSS feed, you should not notice any changes. If you come to the site, you should see some. I am hoping that these changes are for the better.

Tagline

First, I changed the tagline for the site. I decided that the old one, "Curiosity, Creativity, Code," was probably a little too presumptuous and didn't convey everything I wanted. Also, it specifically mentions code while one of my hobbies (and my day job) is system administration. Since this is figuring highly in my current project which I will likely post about, I figured I should make sure it's in scope.

I'm not particularly happy with the new tagline, "Thoughts From a Sysadmin And Occasional Coder." It is more accurate but it's not catchy. I may change it a couple more times before I'm satisfied.

Visual changes

  • The content portion of the site is now shown in a slightly larger font so it should be more readable on larger monitors with larger resolutions.
  • Headers now have a dotted line below them to separate them from the rest of the content.
  • The search field now has a form button.

Module changes

  • The ajax module has been added to improve experience with the forms.
  • Those who create accounts can receive emails when posts are updated (e.g. when a comment is added) through the subscriptions module.
  • Spam protection is now handled through Mollom.

An open question on email

I have been comparing email statistics between two years ago and this week. The percentages of emails rejected as viruses seems to have changed dramatically. Two years ago, the number of virus emails flagged by ClamAV made up at least 10% (and as high as 30%) of the email volume. This week, I'm seeing less than 1%.

I have no reason to doubt that ClamAV is performing normally. However, such a change is still unexpected. Has anyone else noticed a decrease (at least percentage-wise) of virus emails?

On discipline

I threw out a couple days worth of code a couple weeks ago.

I read somewhere that knowing that you can throw out code and start over should be uplifting and exhilarating. It's not. When I threw it out the topic branch in my local git repository, I was annoyed, I was disgusted, I was unhappy. Perhaps that has to do with why I threw it out.

I have this project I'm working on in my spare time. As I've mentioned before, I'm trying out BDD with RSpec and cucumber. And I came to a point that I realized that most of what I was doing was not behavior-driven. It wasn't even adhering to the YAGNI principle. I had a possible case, one that may not arise, and I was writing to try to meet it without a test case. I had realized I was doing this early on and instead of stopping then, I just said that I would run it through rcov and add tests later.

Sometimes I am very, very stupid.

The problem with BDD or TDD or any new way of doing things is that it requires discipline to properly follow them until you get in the habit of doing them. Once you're in the habit, it's not so hard but you have to get there first. Until then, you have to face your urges to use the old method, to take the quick way you've already learned, and deny them. It requires discipline.

And discipline, at least for me, is hard.

I will be trying again on this particular feature tonight or tomorrow night. Hopefully I will have the discipline this time.

A sign in git tag's documentation

I was reading the git tag and found this bit:

But if you have pushed things out (or others could just read your repository directly), then others will have already seen the old tag. In that case you can do one of two things:

  1. The sane thing. Just admit you screwed up, and use a different name. Others have already seen one tag-name, and if you keep the same name, you may be in the situation that two people both have "version X", but they actually have different "X"'s. So just call it "X.1" and be done with it.
  2. The insane thing. You really want to call the new version "X" too, even though others have already seen the old one. So just use git-tag -f again, as if you hadn't already published the old one.

I find it surprising that this has to be said in the documentation. The "sane" method is, to me, clearly the most obvious one. However, as Joel Spolsky once wrote of a "No Dogs Allowed" sign at a restaurant: "The real reason that sign is there is historical: it is a historical marker that indicates that people used to try to bring their dogs into the restaurant." So I suppose enough people have used or wanted to use the "insane" method for it to be mentioned.

When rake spec doesn't work

I mentioned over a week ago that rake rspec wasn't working for me. After looking around for a bit today, I found a solution.

Something in the RSpec chain (maybe RSpec itself) requires version 1.2.3 of the test-unit gem. However, gem install test-unit will install version 2.0.3 by default. You can have both versions of the gem installed. However, having both versions causes the rake task to not work properly. Removing the 2.0.3 version of the gem corrects the issue and the rake task will work normally.

Array#compact vs. checking for nil manually

In the prototype code for a project, I wrote something like:

  1. items << item if item

As is, it's not an ugly line. However, it's inside a loop that creates an object based on each line in a text file. So, to provide context:

  1. items = []
  2.  
  3. File.open( filename, 'r' ) do |file|
  4.   file.each_lines do |line|
  5.     item = build_item( line.strip )
  6.     items << item if item
  7.   end
  8. end

So if the line contains one million lines, it runs that statement one million times. (As to why I'm testing, build_item may return nil if the line matches a given set of criteria.)

Array#compact returns a copy of the array with all nil elements removed from the array. (Array#compact! removes all nil elements from the current array.) Removing the if condition from the loop and adding items.compact! after the File.open block should do the same job as what I originally wrote.

To test, I wrote this program:

  1. require 'benchmark'
  2.  
  3. test_array1 = Array.new
  4.  
  5. 10_000_000.times do
  6.   test_array1 << "Foo"
  7.   test_array1 << nil
  8. end
  9.  
  10. test_array2 = Array.new( test_array1 )
  11. test_array3 = Array.new( test_array1 )
  12.  
  13. new_array = Array.new
  14.  
  15. Benchmark.bm do |x|
  16.   x.report( '#compact' ) { test_array1.compact }
  17.   x.report( '#compact!' ) { test_array2.compact! }
  18.   x.report( '#delete' ) { test_array3.delete( nil ) }
  19.   x.report( '#each' ) do
  20.     test_array1.each do |item|
  21.       new_array << item if item
  22.     end
  23.   end
  24. end

Running this under Ruby 1.9.1 (p243) returned these results:

      user     system      total        real
#compact  0.240000   0.100000   0.340000 (  0.336496)
#compact!  1.100000   0.120000   1.220000 (  1.218401)
#delete  3.140000   0.070000   3.210000 (  3.213278)
#each  3.130000   0.050000   3.180000 (  4.720402)

(For the curious, twenty runs of the same script returned similar numbers.)

The numbers indicate that Array#compact is significantly faster than the original code. Whether or not this optimization is needed is another matter entirely. Even for an array of twenty million elements, the difference in time is about four seconds. I suppose the difference between the two should come down to which is easier to read and understand.

I also decided to test Array#delete since items.delete( nil ) should be equivalent to item.compact!. Array#compact!, somewhat unsurprisingly, is faster although only by about two seconds. I'm not sure why Array#compact! should be almost a second slower than Array#compact.

Pages