Virtual servers and SANs

Tempora mutantur nos et mutamur in illis

We're undergoing a major project to consolidate most of our server infrastructure as virtual machines under Windows 2008 Hyper-V. Suffice to say this is going to be a pretty intense 6 weeks, but at the same time it's a rare opportunity. We're also installing Active Directory (having previously been under NetWare's eDirectory, but never heavily used), so we also get to build a new domain from the ground up.

The primary motivator for this was that one of the core business applications here now requires an AD environment going forward. We then looked at how we could best take advantage of the new capabilities we'd have with AD, and that some of the hardware was past its life expectancy. Then we looked at other smaller projects planned for the ywar, and realised what infrastructure was needed for those. We had initially planned on a traditional server-based architecture for this refresh but suddenly realised that this was a prime candidate for virtualisation, and even more than that, it only made sense to do all this with a SAN. Once we got it all priced out and saw that for a relatively small increase in price we could make our environment extremely flexible in terms of service provisioning, well one thing led to another, and here we are.

There are a few challenges we have to deal with:

• Our current internal DNS server is BIND9 running under Ubuntu Linux, and for the sake of making sure that "weird stuff doesn't happen", this needs to be moved to Windows.
• We're moving from an externally hosted MDaemon mail server to an internally hosted Exchange server, so there is an obviously large transition there.
• All of the virtual machines are hosted in one physical server. Our disaster recovery plan calls for a second virtual host to be located offsite and be ready to take over primary operation within one business day should our entire head office go up in flames. Data replication is the tricky part here.

If we do everything right, our users won't notice much of a difference - certainly very few service interruptions - and they'll have a lot of new features at their disposal along with some performance improvements. Not to mention that we expect our power usage to be cut in half.

This will indeed be a tremendous learning experience.

Ubuntu and the HP Mini 5101 — follow-up

I'm very much enjoying using my Mini 5101. It performs much better than I expected, and now that I have my corporate VPNs and SSH tunnels set up in Ubuntu I have a fully portable support platform, for just such an emergency. I even have a 3G USB modem working (more on that some other time as there is a new model coming out I want to try as well).

Just to clean up some loose ends, I think the ACPI issue has to do with a strange BIOS setting. Intel Atom processors are all single-core as far as I am aware, and yet the dual-core option int he BIOS was enabled by default. Once I disabled this I haven't had any other boot problems since.

The webcam was easy enough to set up simply by making sure the camera was enabled in the BIOS and installing the luvcview and uvccapture packages.

Why would I bother with Windows?

Ubuntu and the HP Mini 5101

I just got a new HP Mini 5101 netbook and the first thing I had to do was rip out Windows and install Ubuntu. I'm just more productive that way.

Unfortunately, not all of the hardware works "out of the box". After two days of Google research, I've got two important things working.

Bluetooth and wired networking work fine with the default installation of Ubuntu 9.04. The trouble spots come from ACPI, WiFi and sound.

Getting the WiFi interface up and working was easy. Just install the linux-restricted-modules package and reboot. Everything comes up on its own then.

Sound was a little more tricky. You need to do two things:

1. Install the linux-backports-modules package for your kernel version (sudo apt-get install linux-backports-modules-`uname -r`)
2. Add the line options snd-hda-intel model=laptop to the file /etc/modprobe.d/alsa-base.conf then reboot.

ACPI is much more complex as it seems to cause a lot of problems booting — enough so that you can't get past kernel errors when you start up. You can get around this by adding acpi=off to the kernel parameters in your grub startup, however this doesn't give you battery information, disables the second CPU core and can cause system lockups. My current workaround — as strange as it is — is to boot only while on battery power, but the battery has to be mostly charged. This has worked every time so far, although I'm not sure why.

I wrote this entry on my new HP Mini and I have to say the best thing about this netbook is the keyboard. It's almost full sized so I don't feel cramped or like I need a special keyboard wand.

More as I find other things that don't quite work as expected. I haven't tried the webcam yet, but I'm not sure that I even care about it at this point.

VMware Server and Multi-Core CPUs

It's always been confusing to me as to what to do with VMware Server guests running on a host that has a multi-core processor.

First, just to clarify, from the information I have gathered VMware will assign a single physical processor to each virtual processor on a guest machine. If you have a multi-core processor, and your host operating system recognises each core as a separate CPU, then each virtual CPU on a guest will be assigned to a single core on your host.

You can get really fancy with assigning specific cores by tinkering with the VMX configuration file.

I've noticed that once I change my guest machine to use multiple CPUs, the CPU usage on my host shoots up to almost 100% and seems to stay there. Curiously though, this doesn't seem to affect any of the other processes running on the host. The answer, as stated in the VMware Knowledge Base, is that the host is executing the "system idle" process on the guest.

On Windows, when your processor isn't doing anything, Windows will execute a sort of 'do nothing' command that shows up in Task Manager as the "System Idle" process. It literally does nothing, and any unused CPU cycles go to this process. (Don't ask me why, I'm just the messenger here.) Unfortunately, VMware seems to translate this as an actual process and executes it on the host, but there it also does nothing. However, the host doesn't recognise this translation as an idle process and shows the VMware process itself as using those CPU cycles. So, even though it looks like your host is doing a lot while your guest is doing nothing, neither are really doing anything.

That still doesn't make me feel comfortable when I first see high CPU usage on my systems, but as long as it isn't causing any problems then I'll just leave things be.

64 bits: Check and Double-check

"I am not accustomed to saying anything with certainty after only one or two observations." -- Andreas Vesalius

In my last post I described how to determine if a processor is 64-bit in Linux using the information in 'cpuinfo'. Of course, if you really want to know for sure then you have to get the same answer from multiple sources looking at the same information.

Enter cpuid, a cross-platform utility that is a little more transparent than the information displayed from your OS. The bad news is that it appears the Linux version of this utility hasn't been updated since 2002, but the good news is that it shows the raw output from the queries to your processor and with a little research you can pretty much decipher it yourself, if the information you're looking for isn't already displayed in human underneath.

To focus on checking whether or not your process is 64-bit, you need to look at the raw output from the processor. Again, from my handy-dandy Pentium 4 desktop:

 eax in    eax      ebx      ecx      edx
00000000 00000005 756e6547 6c65746e 49656e69
00000001 00000f49 00020800 0000641d bfebfbff
00000002 605b5001 00000000 00000000 007c7040
00000003 00000000 00000000 00000000 00000000
00000004 00004121 01c0003f 0000001f 00000000
00000005 00000040 00000040 00000000 00000000
80000000 80000008 00000000 00000000 00000000
80000001 00000000 00000000 00000001 20000000
80000002 20202020 20202020 20202020 6e492020
80000003 286c6574 50202952 69746e65 52286d75
80000004 20342029 20555043 30302e33 007a4847
80000005 00000000 00000000 00000000 00000000
80000006 00000000 00000000 04006040 00000000
80000007 00000000 00000000 00000000 00000000
80000008 00003024 00000000 00000000 00000000

Doesn't make much sense to me either. But the important line is the one highlighted. Don't ask me anything about assembly language. I couldn't even figure out how to do this on my Commodore 64.

According to the CPU ID specification, an EAX input of 0x80000001 will set bit 29 of the EDX output if long mode is available -- in other words, if your processor is 64-bit capable. The EDX output is 0x20000000, which in binary is [ 0010 0000 0000 0000 0000 0000 0000 0000 ]. Counting from zero at the right towards the left, yup, that's bit 29 all right.

Am I 64-bit ready?

"You will never come up against a greater adversary than your own potential." -- Star Trek: The Next Generation 'Evolution'

I recently had to determine if a server running SUSE Linux had 64-bit processors or not. I did the usual stuff -- check the machine specs, which just said Xeon processor without explicitly stating 64-bit, then checked the kernel version, which listed itself as running on i686 rather than x86_64.

There had to be some way of either getting the specific processor model or some information about the hardware that would tell me for sure. After much Googling and gnashing of teeth, I found it.

At your command line enter cat /proc/cpuinfo and this will dump information for each core of each physical processor. The line to look at is "flags". As an example, the flags for my desktop Intel Pentium 4 list these:

fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pebs bts pni dtes64 monitor ds_cpl cid cx16 xtpr lahf_lm

The "lm" flag I've highlighted stands for "long mode" as basically means that the processor is 64-bit. The complete set of flags is included in the cpufeature.h file in the Linux kernel headers.