Ian Pratt

KVM vs Xen, Who Will Win The Fight?

May 9, 2008 by Kris Buytaert Leave a Comment

Ian Pratt and Benny Schnaider are using strong words against each other.

As KVM is gaining more and more popularity by being adopted in several Linux distributions, including Ubuntu, the battle between different virtualization technologies continues to be interesting.

While KVM is being adopted by a variety of software and distribution vendors, Xen is being adopted by hardware vendors to be shipped directly with the iron.

We asked Ian at FOSDEM if he felth the Xen community was changing and if he thought the contributions from the community were slowing down.

“We certainly haven’t seen that. If you think about the life of the Xen project, there have been a number of significant changes. When we left the University to set up XenSource, people were worried we might go off and take Xen in closed source or something, but we didn’t. One of the things that we did do was just to provide greater transparency by setting up the Xen advisory board and the Xen.org website. The advisory board has members from companies like Intel, AMD, HP, IBM, … big companies that are now contributing to Xen and have oversight from the advisory board, so I think the community is pretty happy and it’s going from strength to strength.”

According to ZDNet, Ian also claims that “KVM is not a true hypervisor. It tries to add virtualization capabilities to the Linux kernel but it’s not a true hypervisor approach. The Xen community is alive and well. Xen is a true hypervisor architecture that’s better for scalability, security and availability.

One of the biggest arguments against Xen is that KVM is already in the kernel. Theodore Ts’o thinks “it’s inevitable that Red Hat and Novell will standardize on KVM because of its inclusion in the kernel.” Xen never finished their efforts and KVM was quickly adopted into that same Linux Kernel.

Strong words also from the KVM front:

“If Xen will die or not die, I don’t know. But KVM will take over and be the virtualization selection of choice,” said Benny Schnaider, CEO and co-founder of Qumranet.

KVM or Xen? Time will tell, today both have different features and it will take some time until their feature set is similar, so the choice is about what YOU need, not about what the vendors claim you need.

[Source: ZDNet]

Remus: High Availability via Asynchronous Virtual Machine Replication

April 16, 2008 by Kris Buytaert 3 Comments

The 5th Usenix Symposium on Networked Systems Design and Implementation (NSDI’08) starts today in San Francisco, CA. The NSDI symposium focuses on the design principles of large-scale networks and distributed systems.

XenMaster Ian Pratt (Senior Lecturer, University of Cambridge Computer Laboratory/XenSource (Citrix)) will be presenting the Keynote.
Apart from this Keynote a very interesting paper about Virtualization and High Availability will be presented by the Department of Computer Science at the University of British Columbia.

In a paper titled Remus: High Availability via Asynchronous Virtual Machine Replication Brendan Cully, Geoffrey Lefebvre, Dutch Meyer, Mike Feeley, Norm Hutchinson and Andrew Warfield describe how with minor modifications to Xen 3.1.2
they achieve fast (frequencies as high as forty times per second) replication of Xen virtual machines.

Their tool Remus allows systems to transparently move to another physical machine in the event of a failure on the primary machine , with only seconds of downtime, while preserving the original host state such as active network connections , memory and disk state.

Today most Virtualization vendors claim they achieve High Availability while actually they only implement an easier way to recover from failure or a method to migrate away from a machine that is showing early problems. Most of the current Virtualization based solutions still require you to restart a new virtual machine, or implement High Availability on application level.
‘
Mind , this is a research paper don’t expect these kind of features to show up at your local IT shop within the next couple of weeks.

Thanks to Usenix opening up their conference proceedings for everybod,y you can get the full paper here.

The Present And Future of Xen

April 7, 2008 by Kris Buytaert Leave a Comment

Over the past few months, a number of people have been vouching the idea that Xen’s development and adoption is slowing down because of the Citrix’ acquisition.

Paula over at ZDNet appears to have misunderstood what XenSource/Citrix explained her.

Citrix might have jumped some marketing hoops by first claiming to be an application delivery company, then trying to position itself next to VMWare as a virtualization company, and subsequently buying XenSource in order to reclaim its application delivery role while trying to benefit from the Xen brand.

But rest assured: they are still heavily backing Xen.

Paula proclaims:

“Open source backers will likely take another look at Xen and re-consider other open source firms embracing virtualization”

Well … it’s true different open source integrators were confused when XenSource decided to take their XenEnterprise 4.0 management GUI Windows-only, leaving the Linux Desktop users in the cold. Yes, some customers were lost, but the open source guys were never interested in a GUI that limited the functionality of an open and free product they were used to work with anyway, and neither were the kernel code contributors. So the question is if they were looking at XenSource in the first place.

At FOSDEM, we asked Ian Pratt how he feels about the evolution of the Xen open source community after the Citrix acquisition and if he thought the contributions from the community were slowing down. An excerpt from our interview:

“We certainly haven’t seen that , if you think about the life of the Xen project, there have been a number of significant changes. When we left the University to set up XenSource people were worried we might go of and take Xen in closed source or something, but we didn’t. It’s the same group of guys, basically myself, Keir Frasier, Steve Hand working on the project, and now many more of course. The Citrix acquisition of XenSource was obviously something we had to explain to people. I think the community has seen that nothing has changed . One of the things that we did do was just to provide greater transparency, set up Xen.org , the Xen advisory board and the Xen.org website. The advisory board has members from companies like Intel, AMD, HP, IBM, … big companies that are now contributing to Xen and have oversight from the advisory board, so I think the community is pretty happy and it’s going from strength to strength.”

In a video interview with Tarry Singh at VMworld Europe 2008, Simon Crosby stated:

“You have to understand that Xen is the foundation of the faith, we (Citrix) are first and foremost committed to this community and to that method of development for the server tools and Hypervisor. So one of the community questions was that Citrix was not known for open source and what was gonna happen with Xen.

It turned out, otherwise we would never have agreed to go, that Citrix has thrown a huge amount of money towards the open source community. We’ve setup Xen.org – we’ve always wanted to do that, as a start-up we could never get there.

Xen.org is run by a charter committee of the major contributors and it has its own program management and it’s independent of us (Citrix) and that’s exactly the way we wanted it. Ian still leads the project, we still probably contribute about 60% of the code, but also all of these major partners deliver to us.

So the community is going from strength to strength, which is terrific.

We had 2 or 3 developer summits each year, at the one we had at the end of last year in Santa Clara we had more than 200 people attend.”

When comparing Xen.org to Eclipse.org, Simon replied:

“The difference is that Eclipse is an independent legal entity. With the Xen community, we discussed whether or not to do that and the cost of it and we decided that we would not do that, so we set up a steering committee which oversees several key components of Xen. First of all, the road map that advises the project on where to go, it sets the policy by which the Xen trademark is used and then that is all administered for the benefit of the community with the explicit admission by the advisory board by Citrix

It leaves us without the cost and the legal infrastructures of setting independent .org but with all principle and all the guiding.”

The Last Xen Summit was a 200-person conference, mainly Xen developers, with people from Sun, HP, Novell, RedHat, Virtual Iron, Oracle, Intel, AMD, Samsung, Solarflare, Google and of course Xensource/Citrix It’s too bad Ohloh doesn’t support Mercurial (the Xen.org source managemt system) or we could have had real statistics on the Xen contributions but it’s fairly obvious most of the big players are contributing.

Which linux distro didn’t adopt Xen as a virtualization technology ? Xen Adoption in which distro ? True they are supporting other open source technologies apart from Xen and they are working on creating a uniform way to manage different virtualization techniques, but no matter how you look at it.. they all adopted Xen.

So let’s have a look at the companies that are adopting Xen in their products, starting with Citrix.

When XenEnterprise initially launched, Peter Levine told the world that their target audience was to provide easy-to-install (in less than 10 minutes as he could do himself) bare-metal virtualization for the Windows market. They were not planning a RedHat / MySQL style Xen distribution in free and commercial versions with support and updates, they went straight for a target audience that was used to buy proprietary software from a vendor, the Windows users.

Apart from Citrix, which is planning to launch their XenEnterprise 4.1 release in the next couple of months, amongst the first adopters were RedHat and Novell.

When Novell claimed first adoption , RedHat was saying Xen wasn’t stable enough yet. But today, Xen is a core part of both major Linux Distributors offerings. (and with them lots of other Linux distributions) The race however continues when RedHat and Novell started fighting over which version was about to offer better Window guest support 🙂

Amazon was also a really fast adopter. When Jeff Barr announced the Elastic Compute Cloud Beta, he told the world that they had built EC2 using Xen. In essance, Elastic Compute Cloud, or EC2, allows users to deploy server instances on-demand. Amazon isn’t selling Xen as a product, they are using Xen to provide the world with one of their most used services. Different startups and SMBs are using EC2 as their home on the web.

And let’s have a look at the people selling Xen implementations.

Virtual Iron came from a dubious non-Xen background and is now positioning its platform as an Enterprise-level platform for server virtualization and virtual infrastructure management. Their product consists of a Java-based Virtualization Manager that is used to manage the virtualization services that are deployed on bare-metal servers.

Back in 2007, Simon Crosby wrote:

“The Virtual Iron Hypervisor is not the Xen Hypervisor – it’s a proprietary product (some of which is open-sourced because they use bits & pieces of Xen code). Virtual Iron has not yet made any significant contributions back to the Xen community. Presumably they believe this gives them an edge in the market. Maybe it does. But if that’s the case, I don’t understand why they don’t just stand up and say so, rather than trying to jump on the Xen brand-wagon.”

Simon invited them multiple times to join the Xen community and it seems they accepted the invitation as the Virtual Iron people were even presenting on the last Xen summit. According to Wikipedia, today – as so many others – their platform is based on the Xen Hypervisor. The exact answer is probably somewhere in the source code.

When in November 2007, Oracle first launched their OracleVM there was a lot of fuzz because Oracle claimed both features other people didn’t have as well as better performance. Back then I blogged:

“First, seemingly Larry is claiming that his Xen package is better than others since he supports Live Migration and all the others don’t. I don’t know where he gets that idea.. I have to admit I don’t remember which year it was, but it was somewhere in December when I first started with Live Migration of Xen machines and it was also on a CentOS platform. No fancy GUI, no hardcover manuals that had it all documented. But fast and seamlessly working live migration, ready for everybody to use. Second, he is claiming that Xen was re-engineered by Oracle to be faster than the competition. The way you read it there is that Oracle took Xen, modified it then started redistributing it. Is that really what happened ? Are they redistributing the source, or are they violating the GPL ? Because if they are redistributing the source, everybody just got a faster Xen.”

Earlier, Charles Philips from Oracle had been telling GCN that cite “We’re big proponents of Linux and standard technologies, so we’re going to put the time toward Xen,” Phillips said. “Our strategy will be around Xen.” which didn’t surprise anybody as Oracle had been pushing before to get a single virtualization supportive interface into the Linux kernel.

The openSolaris people started out with building a DomU for Xen, at first, but what they really wanted was to run openSolaris for both their Dom0 and DomU. Today Sun is supporting the sun seemed to be working on Xen for Sparc but it seems they abandoned that effort.

Obviously Sun has is Solaris Zones technology but as running different isolated environments on one kernel is totally different from running different kernels Zones and xVM are obviously complementary technologies.

Toon Vanagt asked Ian Pratt :

“When I look at the Xen GPL License, I found it interesting that Xen is being renamed as xVM by Sun, OracleVM by Oracle. Oracle first announced OracleVM and then had to admit it was actually a tweaked Xen version. But they didn’t publish the tweaked code, did they?”

Ian’s reply:

“Oh no, they have. I mean, the fact is that there are lots of different vendors shipping Xen products that also they ship the Xen engine, pick up the Hypervisor the core engine and incorporate it into their own products. Obviously, the Linux vendors Novell and Redhat, there is Sun, there is obviously Xensource / Citrix , Virtual Iron, etc. Lots of different companies are doing that. Actually the GPL license means that any changes they make will go back into the main project. In reality, pretty much all those companies just pick it up as is. Take the latest stable release, which is maintained, they might add the odd little patch to it, but it really is all very clear, there is all uniformity in the Xen versions that are out there.”

“Most of those companies are very close to mainline Xen. They post a couple of patches in some cases, but not always. What they’ll be doing is taking Xen and it’s really on top of Xen , in the rest of their Virtualization stack that runs in user space, that’s where they’ll be probably doing their own of things, they’ll have their own management tools, they’ll have their own way of wanting to present virtualization to the user, so if you think about what the operating system vendors are typically doing is they want to expose virtualization using the same tools and user interfaces etc. they use for exposing other facilities in that operating system. Which if you think about it, is quite different from what companies like say Xensource is trying to do , which is to try and effectively build a virtual machine hosting appliance. You know, you just put the CD in the server, install it and just manage it from let’s say a windows GUI or a webinterface . So every company is bringing Xen to market in a different way for a different kind of user . And that is where the differentiation happens, but the core engine is the same throughout.”

So which Xen should you choose? One of the main decision points when choosing a Xen vendor is probably whether you want to virtualize Linux, Windows, or a mix. The different vendors have different relationships with Microsoft and will therefore be able to provide different levels of support and integration with their products. But one thing is certain: you have a lot more choice when going for a Xen alternative, than the other way around.

Building and maintaining a community is and will remain a difficult thing. Sun is learning that and Citrix / Xen.org will also have to learn that. Six months from now, the story might be totally different. But today Xen.org is growing stronger every day with corporate contributions from all over the planet.

The Current State of Open Source Virtualization

March 26, 2008 by Kris Buytaert 6 Comments

We’ve started by looking back at a decade of Open Source virtualization, and in this second part of the series we’ll tackle today’s landscape (last updated in March 2008).

The least you can say about the current state of Open Source virtualization is that the field is extremely diverse: different approaches in the virtualization area are all represented, with paravirtualization, OS virtualization and hardware-assisted virtualization in various colors and flavours.

Let’s start with paravirtualization:

Xenmaster Ian Pratt released the 1.0 version of Xen somewhere in September 2003, it wasn’t till the Xen 2.0 release (around December 2005) that Xen adoption really started to accelerate. Ian announced the 2.0 release in November 2004 with support for both Linux 2.4, 2.6, FreeBSD and Live Migration support.

Xen pioneered Paravirtualization, giving it both a giant performance boost but also an argument for the naysayers who claimed it was impossible to run Windows on the platform. The fact that the Cambridge lab had access to the Windows source code and even had it running on Xen wasn’t really an argument since they were unable to redistribute it.

Different Linux distributions adopted quickly making Xen the de facto Linux virtualization solution. Also: the Open Solaris project was working on Xen support, first only as a guest but later also as a host operating system.

Then came the VT capabilities, and once again Xen was leading the pack, bringing out a Xen version that supported hardware-assisted virtualization. So the Open Source Xen version was beating the competition on different levels – speed, flexibility, etc. – but had one key element missing: the management layer, a GUI, the part that people actually spend money on …

Meanwhile, the company XenSource Inc. had been founded by the original developers of Xen and they started to work on a set of management tools and bang, next thing we know is Citrix announcing the acquisition of XenSource for $ 500 million USD in the summer of 2007.

While the discussion between Xen and VMware was still going on to see what infrastructure was needed in the kernel to support virtualization, KVM (Kernel Based Virtual Machine) had come out of nowhere: a lightweight kernel module that enabled the VT Capabilities of the new generation of CPUs and that ended up in the mainstream kernel in no time. KVM was ultimately included in the 2.6.20 release of the Linux kernel after merely a couple of months of development.

KVM enabled Qemu to benefit from the VT features and a new team was born. KVM is the lean and mean, small virtual machine, and the fact that it was so small only made it easier to adopt in the main tree. KVM is maintained by Avi Kivity who is working at Qumranet, with Moshe Bar amongst its founders about to launch a product called Solid ICE, aiming for the desktop virtualization market. KVM however is not doing all the work, a modified Qemu version acts as the user space part that enables the full power of KVM.

Today different distributions support both KVM and Xen and are working towards a single tool set to manage them both.

Qemu started to pop up everywhere in the virtualization arena in 2007, e.g. within the VirtualBox project from innotek, a German software company located in Stuttgart.

VirtualBox is one of the most important open source solutions if you want to run other operating systems on your desktop. It’s free, it’s open and it has all the features you would expect from its commercial counterparts! Sometimes these commercial counterparts, facilitate ‘match making’ events, which outcomes are not intended. For example at VMworld in New York in September ’07, Achim Hasenmueller, co-founder and kernel wizard at innotek was introduced to the Sun Microsystems management and less than four months later they announced their ‘marriage’ (Sun acquired innotek for an undisclosed amount in February 2007). As VirtualBox was already running on a multitude of Operating Systems such as Windows, Linux and OS/X, they evidently also added Sun’s Solaris to this impressive list. VirtualBox also supports a large number of guest platforms, including common Windows flavors (NT 4.0, 2000, XP, Server 2003, Vista).

We’ve been talking mostly about paravirtualisation and hardware-assisted virtualization with KVM, Xen and VirtualBox, but of course there is much more out there. Let’s have a look at the players on the Operating System Level virtualisation are, an identical copy of one kernel providing a secured container where user space programs can run. Today, there are 2 main players in this area (VServer and OpenVZ). VServer was started by Jacques Gelinas and is currently lead by Herbert Pötzl from Austria. The Linux-VServer started July 2001 as BSD Jail reimplementation for Linux. In 2004, it was rewritten from scratch for the 2.6 kernel.

Not much of a surprise, people tend to think that Linux-VServer and OpenVZ have a lot in common, and some people even think OpenVZ was once based on a fork of Linux-VServer. According to Herbert Pötzl that isn’t true today: the projects do not share any code, although they provide roughly similar functionality in often quite different ways … In 2003 however , Linux-VServer was forked into FreeVPS by Alex Lyashkov and soon after that, it was integrated into the H-Sphere product, maintained by Positive Software.

SWsoft was founded back in 1999 and released their commercial Virtuozzo product in 2001, as a proprietary virtualization solution for Linux and later also supporting Windows. When SWsoft acquired Plesk in 2003, a proprietary framework to manage hosted solution, evidently virtualization fitted nicely in this picture since the OS level virtualization OpenVZ uses is a perfect match for web hosting.

SWsoft then went on to buy Parallels and managed to keep it a secret for almost 3 years. In late 2007, they finally decided that their Parallels brand was better known than their Virtuozzo or Plesk brands and decided to change the company name into Parallels alltogether. Having a single kernel for each virtual machine that runs in your environment is both the advantage and the disadvantage of OpenVZ and Linux-VServer. Its advantage of being a lightweight solution that can scale easily to hundreds of machines with no significant penalty is also its biggest disadvantage – what if something goes wrong with that kernel? Other approaches such as Xen and KVM allow you to run different kernels , or even different operating systems, which of course requires much more memory for each instance.

If you are into Hot Motorcycles you’ll remember the 1999 Virtual Iron company, a company that manufactered a CD that helped people create a customized bike. Fast forward to 2004, where a domain-squatter was using the site, and in February 2005 a company that looks like the Virtual Iron company we know now, started using the domain. Virtual Iron had a product called Virtual Iron VFE in store, which they presented at Linux World and later also more in depth at OLS. They claimed to have developed a Virtual Machine Monitor that was also Clustered. The Virtual Iron VFe product transparently created a shared memory multi-processor out of number of servers.

Yes, this sounds familiar, it sounds like an SSI implementation, it sounds like openMosix or OpenSSI, and that’s exactly what some people thought it was. Rumors on the net claimed that Virtual Iron was indeed violating the GPL while reusing and modifying openMosix code while not redistributing it’s changes, true or false, we’ll probably never know. In August 2005 Virtual Iron started shifting as they announced they were working on having their software manage other platforms too. Today, their product is based on an open-source Hypervisor, which name you can most likely already guess (yes, indeed, they use Xen). What happened with the SSI alike technology is unclear.

The final player in this area we need to point to is Paul Rusty Russel’s Lguest, formerly known as Lhype, almost known as Rustyvisor or Wonkavisor. It is an experimental Hypervisor developed by Rusty intended as proof of concept for the paravirt ops. Redhat has been working on it also, but who knows what the future will bring?

Which brings us to the final part: where to put your money? That kinda depends on your needs:

If I’m talking to a hoster who needs to run lots and lots of similar machines with easy management, I’ll be pointing him to Linux-Vserver
If someone is looking at bare metal hardware virtualisation for his Linux machines, it’s Xen all the way
If he needs a platform to test different distributions and operating systems on his desktop I’ll probably be pointing to VirtualBox
If someone really wants to head into placing his desktops virtualized in the data center, KVM would be my bet

What if someone wants to do nothing else but use Linux as a base framework to run Windows virtual machines?

In that case the commercial Xen offerings such as the one from XenSource, Suse and Redhat would be best as they can provide you also with adapted drivers for the guest operating system. But ask me again in 6 months and I’ll probably tell you otherwise.

Watch out for the third part of this article series, with more on Xen!

Preinstalled Hypervisors And The Future of Operating Systems

March 5, 2008 by Kris Buytaert Leave a Comment

Jay Lyman from The 451 Group (also check out the interview we did with John Abbott, Chief Analyst & Research Director at The 451 Group) wonders about the future of Linux distributions in the virtualization arena.

Now that VMWare announced that it will embed its ESX 3i hypervisor in different server platforms from HP, Dell, Fujitsu-Siemens and IBM, the question pops up how Operating System Vendors will deal with this change of platform.

VMWare certainly isn’t the only one with those plans, since Ian Pratt from XenSource mentionned exactly the same during his Fosdem talk.

How do the OS vendors react to this new feature ? According to Lyman’s blog post, Red Hat claims

it is hardware vendors such as AMD and Intel that will create that standard virtualization layer and capability.

and

Novell indicates VMware may be taking somewhat of a risk, though, since OEMs like HP will look to upsell to their own software to create and manage VMs, which ESX 3i can’t do.

A hypervisor still needs management tools, so that the guest OS’s can be initiated, stopped and migrated. Applications aren’t running on hypervisors (yet); they need an operating system for IO, Memory Management and Network stacks at least for the foreseeable future.

On a longer term, we’ll have applications running natively on the hypervisor for sure. But today Operating System vendors are hoping for a uniform and better way to support different available and upcoming hypervisors and off course those lightweight systems will also benefit from these improvements.

If I were in the Operating System market I wouldn’t worry yet at this pointis , just as with all other features that hardware vendors are selling it is still ‘only’ a feature. When ordering a Dell you can choose between different CPU’s, different hard disks, different Operating Systems and most likely in the near future, different hypervisors as well.