Virtualization.com

News and insights from the vibrant world of virtualization and cloud computing

Guest Posts

Reading Tip: Andrew Kutz On the Future Of The Virtualization Ecosystem

by Leave a Comment

Here’s a must-read blog post from Andrew Kutz over on O’Reilly’s On Lamp: Ubiquitous underpinnings and idyllic interfaces – The virtualization ecosystem of the (very near) future.

An excerpt:

The virtualization ecosystem is changing; a division is occurring that is separating the ecosystem into two areas: specialized and commoditized software. Hypervisors are becoming commoditized such that they no longer need to be purchased from a specific vendor. A few years ago when virtualization was starting to heat up, playgrounds across the nation were quickly subject to the VMware-kids and Xen-kids cliques. Either you were down with ESX or you were hip to Xen; there was no middle-ground. However, today we find ourselves in an entirely different predicament. In addition to VMware and Citrix (bye, bye XenSource, we hardly knew ya), we have the soon-to-be released Hyper-V, VirtualIron, KVM, Sun xVM, and even Oracle is getting into the game. The number of available hypervisors on the market has grown drastically in a relatively short period of time. Compare that to when you had only two choices; now there is a plethora of virtualization solutions to answer the demand for reducing physical servers in the data center and consolidating resources. Now that hypervisors are a dime-a-dozen it only makes sense that a virtual machine (VM) should be able to run on any of them.

Read the rest!

The Current State of Open Source Virtualization

by 6 Comments

We’ve started by looking back at a decade of Open Source virtualization, and in this second part of the series we’ll tackle today’s landscape (last updated in March 2008).

The least you can say about the current state of Open Source virtualization is that the field is extremely diverse: different approaches in the virtualization area are all represented, with paravirtualization, OS virtualization and hardware-assisted virtualization in various colors and flavours.

Let’s start with paravirtualization:

Xenmaster Ian Pratt released the 1.0 version of Xen somewhere in September 2003, it wasn’t till the Xen 2.0 release (around December 2005) that Xen adoption really started to accelerate. Ian announced the 2.0 release in November 2004 with support for both Linux 2.4, 2.6, FreeBSD and Live Migration support.

Xen pioneered Paravirtualization, giving it both a giant performance boost but also an argument for the naysayers who claimed it was impossible to run Windows on the platform. The fact that the Cambridge lab had access to the Windows source code and even had it running on Xen wasn’t really an argument since they were unable to redistribute it.

Different Linux distributions adopted quickly making Xen the de facto Linux virtualization solution. Also: the Open Solaris project was working on Xen support, first only as a guest but later also as a host operating system.

Then came the VT capabilities, and once again Xen was leading the pack, bringing out a Xen version that supported hardware-assisted virtualization. So the Open Source Xen version was beating the competition on different levels – speed, flexibility, etc. – but had one key element missing: the management layer, a GUI, the part that people actually spend money on …

Meanwhile, the company XenSource Inc. had been founded by the original developers of Xen and they started to work on a set of management tools and bang, next thing we know is Citrix announcing the acquisition of XenSource for $ 500 million USD in the summer of 2007.

While the discussion between Xen and VMware was still going on to see what infrastructure was needed in the kernel to support virtualization, KVM (Kernel Based Virtual Machine) had come out of nowhere: a lightweight kernel module that enabled the VT Capabilities of the new generation of CPUs and that ended up in the mainstream kernel in no time. KVM was ultimately included in the 2.6.20 release of the Linux kernel after merely a couple of months of development.

KVM enabled Qemu to benefit from the VT features and a new team was born. KVM is the lean and mean, small virtual machine, and the fact that it was so small only made it easier to adopt in the main tree. KVM is maintained by Avi Kivity who is working at Qumranet, with Moshe Bar amongst its founders about to launch a product called Solid ICE, aiming for the desktop virtualization market. KVM however is not doing all the work, a modified Qemu version acts as the user space part that enables the full power of KVM.

Today different distributions support both KVM and Xen and are working towards a single tool set to manage them both.

Qemu started to pop up everywhere in the virtualization arena in 2007, e.g. within the VirtualBox project from innotek, a German software company located in Stuttgart.

VirtualBox is one of the most important open source solutions if you want to run other operating systems on your desktop. It’s free, it’s open and it has all the features you would expect from its commercial counterparts! Sometimes these commercial counterparts, facilitate ‘match making’ events, which outcomes are not intended. For example at VMworld in New York in September ’07, Achim Hasenmueller, co-founder and kernel wizard at innotek was introduced to the Sun Microsystems management and less than four months later they announced their ‘marriage’ (Sun acquired innotek for an undisclosed amount in February 2007). As VirtualBox was already running on a multitude of Operating Systems such as Windows, Linux and OS/X, they evidently also added Sun’s Solaris to this impressive list. VirtualBox also supports a large number of guest platforms, including common Windows flavors (NT 4.0, 2000, XP, Server 2003, Vista).

We’ve been talking mostly about paravirtualisation and hardware-assisted virtualization with KVM, Xen and VirtualBox, but of course there is much more out there. Let’s have a look at the players on the Operating System Level virtualisation are, an identical copy of one kernel providing a secured container where user space programs can run. Today, there are 2 main players in this area (VServer and OpenVZ). VServer was started by Jacques Gelinas and is currently lead by Herbert Pötzl from Austria. The Linux-VServer started July 2001 as BSD Jail reimplementation for Linux. In 2004, it was rewritten from scratch for the 2.6 kernel.

Not much of a surprise, people tend to think that Linux-VServer and OpenVZ have a lot in common, and some people even think OpenVZ was once based on a fork of Linux-VServer. According to Herbert Pötzl that isn’t true today: the projects do not share any code, although they provide roughly similar functionality in often quite different ways … In 2003 however , Linux-VServer was forked into FreeVPS by Alex Lyashkov and soon after that, it was integrated into the H-Sphere product, maintained by Positive Software.

SWsoft was founded back in 1999 and released their commercial Virtuozzo product in 2001, as a proprietary virtualization solution for Linux and later also supporting Windows. When SWsoft acquired Plesk in 2003, a proprietary framework to manage hosted solution, evidently virtualization fitted nicely in this picture since the OS level virtualization OpenVZ uses is a perfect match for web hosting.

SWsoft then went on to buy Parallels and managed to keep it a secret for almost 3 years. In late 2007, they finally decided that their Parallels brand was better known than their Virtuozzo or Plesk brands and decided to change the company name into Parallels alltogether. Having a single kernel for each virtual machine that runs in your environment is both the advantage and the disadvantage of OpenVZ and Linux-VServer. Its advantage of being a lightweight solution that can scale easily to hundreds of machines with no significant penalty is also its biggest disadvantage – what if something goes wrong with that kernel? Other approaches such as Xen and KVM allow you to run different kernels , or even different operating systems, which of course requires much more memory for each instance.

If you are into Hot Motorcycles you’ll remember the 1999 Virtual Iron company, a company that manufactered a CD that helped people create a customized bike. Fast forward to 2004, where a domain-squatter was using the site, and in February 2005 a company that looks like the Virtual Iron company we know now, started using the domain. Virtual Iron had a product called Virtual Iron VFE in store, which they presented at Linux World and later also more in depth at OLS. They claimed to have developed a Virtual Machine Monitor that was also Clustered. The Virtual Iron VFe product transparently created a shared memory multi-processor out of number of servers.

Yes, this sounds familiar, it sounds like an SSI implementation, it sounds like openMosix or OpenSSI, and that’s exactly what some people thought it was. Rumors on the net claimed that Virtual Iron was indeed violating the GPL while reusing and modifying openMosix code while not redistributing it’s changes, true or false, we’ll probably never know. In August 2005 Virtual Iron started shifting as they announced they were working on having their software manage other platforms too. Today, their product is based on an open-source Hypervisor, which name you can most likely already guess (yes, indeed, they use Xen). What happened with the SSI alike technology is unclear.

The final player in this area we need to point to is Paul Rusty Russel’s Lguest, formerly known as Lhype, almost known as Rustyvisor or Wonkavisor. It is an experimental Hypervisor developed by Rusty intended as proof of concept for the paravirt ops. Redhat has been working on it also, but who knows what the future will bring?

Which brings us to the final part: where to put your money? That kinda depends on your needs:

  • If I’m talking to a hoster who needs to run lots and lots of similar machines with easy management, I’ll be pointing him to Linux-Vserver
  • If someone is looking at bare metal hardware virtualisation for his Linux machines, it’s Xen all the way
  • If he needs a platform to test different distributions and operating systems on his desktop I’ll probably be pointing to VirtualBox
  • If someone really wants to head into placing his desktops virtualized in the data center, KVM would be my bet

What if someone wants to do nothing else but use Linux as a base framework to run Windows virtual machines?

In that case the commercial Xen offerings such as the one from XenSource, Suse and Redhat would be best as they can provide you also with adapted drivers for the guest operating system. But ask me again in 6 months and I’ll probably tell you otherwise.

Watch out for the third part of this article series, with more on Xen!

Looking Back At A Decade of Open Source Virtualization

by 3 Comments

Will 2008 become the “Virtual Year”?

That’s what some people would have us believe now that the virtualization hype is reaching never before seen heights, and large acquisitions & mergers are starting to become quite common (Citrix bought Xensource, Novell picked up PlateSpin, Sun acquired innotek, Quest Software snapped up Vizioncore while VMware treated itself to Thinstall, and so on).

But few people realize or fail to acknowledge that the large majority of virtualization techniques and developments were started as, or remain Open Source projects.

Where are we coming from ?

Even without looking back, we know that IBM was one of the pioneers in the virtualization area; they were talking about Virtual Machines before I was even born. But who remembers one of the first Open Source virtualization takeovers? Back in 1999, Mandrake Software bought Bochs . Yes, that’s nineteen ninety nine, even before the y2k hype. Kevin Lawton had been working on the Bochs project together with different other developers since 1994. In 1999, he also had started working on Plex86, also known as FreeMWare.

Kevin back then compared Plex86 to other tools such as VMWare, Wine, DOSEMU and Win4Lin. Plex86 in the meanwhile has been totally reinvented. While at first it was capable of running almost all operating systems, it is now a very light virtual machine designed only to run Linux.

Wine was also a frequently covered topic at different Linux Kongress venues. As its initiators claim themselves, Wine is not an emulator, but it most certainly used to be a key player in the virtualization area. Its attempts to run non-native applications in a different operating system, in this case mostly Windows applications on a Linux platform, didn’t exactly pass by unnoticed.

However, installing VMWare or Qemu became such an easier alternative than trying to run an application with Wine. And Win4Lin, its commercial brother, had similar adoption issues. Corporate adoption for neither Wine nor Win4Lin was successful, and Win4Lin recently reinvented itself as a Virtual Desktop Server product, where it is bound to face a lot of stiff competition.

People who claim desktop virtualization was ‘born in 2007’ obviously missed part of history. Although most Unix gurus claim desktop virtualization has been around for several decades via the X11 system, the Open Source alternatives to actually do the same on different platforms (or cross-platform) have also been around for a while.

Who has never heard of VNC, the most famous product that came out the Olivetti & Oracle Research Laboratory (ORL) in Cambridge, England? VNC was one of the first tools people began to use to remotely access Windows machines. System administrators who didn’t feel like running Windows applications on their Unix desktop just hid an old Windows desktop under their desk and connected to it using VNC. It was also quickly adopted by most desktop users as a tool to take over the desktop of a remote colleague. After the Olivetti & Oracle Research Laboratory closed different spin-offs of VNC such as RealVNC , TightVNC and UltraVNC popped up.. and it’s still a pretty actively used tool.

But VNC wasn’t the only contender in the field. Back in 2003, I ran into NX for the very first time , written by the Italian folks from NoMachine , with a FreeNX release co-existing alongside a commercial offering. It was first claimed to be yet another X reinvention, however NX slightly modified the concept and eliminated the annoying X roundtrips. The fact that NX used proxies on each side of the connection guaranteed that it could function even on extremely slow connections.

In the early days of this century, there was some confusion between UML and UMLinux. While Jeff Dike called his User-mode Linux the port of Linux to Linux, it was in essence a full blown Linux kernel running as a process on another Linux machine.

Apart from UML, there was UMLinux, also a User Mode Linux project, featuring a UML linux machine which booted using Lilo and from which an out-of-the-box Linux distribution could be installed. Two projects, one on each side of the Atlantic, with both a really similar goal and similar naming was simply asking for confusion. In 2003, the UMLinux folks decided to rebrand to FAUmachine. hence ending the confusion once and for all.

Research on virtualization wasn’t conducted exclusively in Germany; the Department of Computer Science and Engineering of the University of Washington was working on the lesser known Denali project. The focus of the Denali project is on lightweight protection domains; they are aiming at running 100s and 1000s VM’s concurrently on one single physical host.

And apparently, one project with a confusing name wasn’t enough. The Open Source community seemed desparate for more of that. Hence, the Linux-VServer project and Linux Virtual Server came around around the same time. The Linux Virtual Server actually hasn’t got that much to do with virtualization, at all. In essence, Linux Virtual Server is a load balancer that will balance TCP/IP connections to a bunch of other servers hence acting to the end user as one big High Performant and Highly Available Virtual Server. (The IPVS patch for Linux has been around since early 1999).

Linux VServer (released for the first time in late 2001) on the other hand provides us with different Virtual Private Servers that are running in different security contexts. Linux VServer will create different user space segments , so that each Virtual Private server looks like a real server and can only ‘see’ its own processes.

By then, Plex86 had a big competitor coming from France, where Fabrice Bellard was working Qemu. At first, Qemu was really a Machine Emulator. Much like Bochs (anyone still running AmigaOS?), you could create different virtual machines from totally different architectures. Evidently froml X86, but also from ARM, Sparc, PowerPC, Mips, m68k and even development versions for Alpha and alternative 64bit architectures. Qemu however was perceived by a lot of people as slow compared to other alternatives. There was an Accelerator module available providing an enormous performance boost, however that didn’t have such an open license as the rest of Qemu, which held back its adoption significantly. It was only about a year ago (early 2007) that the Accelerator module also became completely open source.

The importance of Qemu however should not be underestimated, as most of the current hot virtualization projects are borrowing Qemu knowledge or technology left and right. KVM (Kernel-based Virtual Machine) is the most prominent user of Qemu, but even VirtualBox, Xen (in HVM mode) and the earlier mentioned Win4Lin are using parts of Qemu.

As this is an overview of the recent Open Source Virtualisation history the focus has been on running virtual machines on Linux, or connecting to a remote platform from a Linux or Unix desktop, where most of the early developments have taken place. We shouldn’t fail to mention CoLinux in this regard, however. CoLinux allows you to run Linux as a Windows process, giving people on locked down desktops an alternative for VMWare to run Linux on their desktop.

Xen is with no doubt the most famous open source virtualization solution around, certainly after its acquisition by Citrix. Xen was conceived at the XenoServer project from the University of Cambridge, an initiative aiming to build an infrastructure for distributed computing and to create a place where one can safely execute potentially dangerous code in a distributed environment. Xen was first described in a paper presented at SOSP in 2003 but work on it began somewhere in 2001.

Next week, we’ll talk more about virtualization and open source with a detailed look at today’s landscape.

Preinstalled Hypervisors And The Future of Operating Systems

by Leave a Comment

Jay Lyman from The 451 Group (also check out the interview we did with John Abbott, Chief Analyst & Research Director at The 451 Group) wonders about the future of Linux distributions in the virtualization arena.

Now that VMWare announced that it will embed its ESX 3i hypervisor in different server platforms from HP, Dell, Fujitsu-Siemens and IBM, the question pops up how Operating System Vendors will deal with this change of platform.

VMWare certainly isn’t the only one with those plans, since Ian Pratt from XenSource mentionned exactly the same during his Fosdem talk.

How do the OS vendors react to this new feature ? According to Lyman’s blog post, Red Hat claims

it is hardware vendors such as AMD and Intel that will create that standard virtualization layer and capability.

and

Novell indicates VMware may be taking somewhat of a risk, though, since OEMs like HP will look to upsell to their own software to create and manage VMs, which ESX 3i can’t do.

A hypervisor still needs management tools, so that the guest OS’s can be initiated, stopped and migrated. Applications aren’t running on hypervisors (yet); they need an operating system for IO, Memory Management and Network stacks at least for the foreseeable future.

On a longer term, we’ll have applications running natively on the hypervisor for sure. But today Operating System vendors are hoping for a uniform and better way to support different available and upcoming hypervisors and off course those lightweight systems will also benefit from these improvements.

If I were in the Operating System market I wouldn’t worry yet at this pointis , just as with all other features that hardware vendors are selling it is still ‘only’ a feature. When ordering a Dell you can choose between different CPU’s, different hard disks, different Operating Systems and most likely in the near future, different hypervisors as well.

How to build a Virtual Private Server (VPS) With Debian 3.1 (Sarge) And OpenVZ

by Leave a Comment

Till Brehm has released an essential how-to at HowToForge.Com for installing OpenVZ, the open source version of SWsoft Virtuozzo, on a Linux Debian 3.1 32bit operating system.

It starts from the very beginning step, patching kernel, up to the final goal, building a so-called Virtual Private Server (VPS). With OpenVZ you can create multiple Virtual Private Servers (VPS) on the same hardware, similar to Xen and the Linux Vserver project. OpenVZ is the open-source branch of Virtuozzo, a commercial virtualization solution used by many providers that offer virtual servers. The OpenVZ kernal patch is licensed under the GPL license, and the user-level tools are under the QPL license. He clearly warns that this is not the only way of setting up such a system and issues no guarantee that this will work for you 🙂

Read how it worked for him.