Kevin Lawton

Guest Post: “Fault tolerance a new key feature for virtualization”

August 6, 2009 by Robin Wauters Leave a Comment

Below is a an article originally published on the guest author’s blog. Who’s the author, you ask?

Kevin Lawton! Bio: pioneer in x86 virtualization, serial entrepreneur, business and technology visionary, prolific idea creator, news and business book junkie. Founding team member in a microprocessor startup, the author and lead for two Open Source projects, a public speaker, and at the forefront of what is now a multi-billion dollar x86 virtualization industry. I have a degree in computer science and started my career at MIT Lincoln Laboratory.

–

Fault tolerance a new key feature for virtualization

VM migration has been a key feature and enabling technology which has differentiated VMware from Microsoft’s Hyper-V. Though as you may know, Windows Server 2008 R2 is slated for broad availability on or before October 22, 2009 (also the Windows 7 GA date), and Hyper-V will then support VM migration. So you may be wondering, what key new high-tech features will constitute the next battleground for differentiation amongst the virtualization players?

Five-Nines (99.999%) Meets Commodity Hardware

One such key feature is very likely to be fault tolerance (FT) — the ability for a running VM to suffer hardware failure on one machine, and to be restarted on another machine without losing any state. This is not just HA (High Availability), it’s CA (Continuous Availability)! And I believe it’ll be part of the cover-charge that virtualization vendors (VMware, Citrix/XenSource, Microsoft, et al) and providers such as Amazon will have to offer to stay competitive. When I talk about fault tolerance, I don’t mean using special/exotic hardware solutions — I’m talking about software-only solutions which handle fault tolerance in the hypervisor and/or other parts of the software stack.

Here’s a quick summary of where the various key vendors are w.r.t. fault tolerance. Keep watch of this space, because the VM migration battle is nearly over now.

VMware’s product line now offers Fault Tolerance, which they conceptually introduced at VMworld 2008. This was perhaps the biggest wow-factor feature VMware talked about at that VMworld. FT is not supported in VMware Essentials, Essentials Plus or vSphere Standard editions. It’s supported in more advanced(/expensive) versions.

In the Xen camp, there are two distinct FT efforts, Kemari and Remus. Integration/porting to Xen 4.0 are on theroadmap. If/when that occurs, the Xen ecosystem will benefit. After battle-testing, it’s easy to conceive of Amazon offering FT as a premium service. It does after all chew through more network capacity, and will necessitate extra high level logic on their part. There’s also a commercial FT solution for XenServer from Marathon, called everRun VM.

Microsoft appears to be leveraging a partnership with Marathon for their initial virtualization FT solution. This is probably smart given it allows Microsoft a way to quickly compete on fault tolerance, with a partner that’s been doing FT for a living. One would imagine this option will come at a premium though, perhaps a revenue opportunity for Microsoft for big-money customers, with an associated disadvantage vis-à-vis similar features based on free Xen technology and massive scale virtualization (clouds). That may make Marathon a strategic M&A target.

Licensing Issues, Part II

Just when you thought software-in-a-VM issues were mostly resolved, the same questions may be raised again for FT, given there is effectively a shadow copy of any given FT-protected VM. It’s not hard to imagine Microsoft aggressively taking advantage of this situation, given they live at both virtualization/OS and application layers of the stack.

Networking is Key

Fault tolerance of VMs is yet another consumer and driver of high bandwidth, low latency networking. The value in the data center is trending from the compute hardware to the networking. FT is another way-point in the evolution of that trend, allowing continuous availability on commodity hardware. You probably won’t run it on all your workloads (they will run with a performance penalty), but you might start out with the most critical stateful workloads. If you want to do this on any scale, or with flexibility, architect with lots of networking capabilities. For zero-sum IT budgets, this would mean cheaper hardware and better networking, something that might be a little bitter-sweet for Cisco, given its entrance into the server market.

Virtualization 3.0 According To Pioneer Kevin Lawton

March 20, 2009 by Robin Wauters Leave a Comment

A must-read article from Kevin Lawton, self-proclaimed “pioneer in x86 virtualization, serial entrepreneur, business and technology visionary, prolific idea creator, news and business book junkie”.

The full article is here, but we give you an excerpt:

To get to where virtualization needs to go, we need to be able to look at virtualization as a fabric, stretching or overlaying numerous physical sites. And Cloud Computing will absolutely exacerbate this need. Many things that we’ve contemplated on a small scale (e.g. load balancing, power management, down-time maintenance), need to be brought to a larger context of a virtualization fabric stretching across physical sites. Virtualization needs to stretch to the cloud. To be sure, there are a number of issues to solve to make this happen, including networking and storage continuity. But I’d like to present a part of this next evolutionary step, virtualization 3.0, which is critical to its success yet unanswered elsewhere to my knowledge.

Memory density in servers continues to go up following its own exponential path. And as virtualization is used for increasingly higher-end workloads, the size of per-VM memory will continue to rise. Just imagine if you piled up all the RAM from all of your data centers, in one spot! Yet, to enable a fluid and dynamic virtualization 3.0 fabric, we need to rapidly allow all kinds of intra and inter-site VM migrations to occur, often driven automatically. That requires a whole new approach to how we manage VM memory; huge volumes of it effectively need to be transported rapidly. On the storage front, there are a number of technologies afoot, which are enablers of virtualization 3.0. But, I’ve been working for some time on concepts for making VM memory a 1st class citizen of the virtualization 3.0 vision.

Full post here.

Looking Back At A Decade of Open Source Virtualization

March 10, 2008 by Kris Buytaert 3 Comments

Will 2008 become the “Virtual Year”?

That’s what some people would have us believe now that the virtualization hype is reaching never before seen heights, and large acquisitions & mergers are starting to become quite common (Citrix bought Xensource, Novell picked up PlateSpin, Sun acquired innotek, Quest Software snapped up Vizioncore while VMware treated itself to Thinstall, and so on).

But few people realize or fail to acknowledge that the large majority of virtualization techniques and developments were started as, or remain Open Source projects.

Where are we coming from ?

Even without looking back, we know that IBM was one of the pioneers in the virtualization area; they were talking about Virtual Machines before I was even born. But who remembers one of the first Open Source virtualization takeovers? Back in 1999, Mandrake Software bought Bochs . Yes, that’s nineteen ninety nine, even before the y2k hype. Kevin Lawton had been working on the Bochs project together with different other developers since 1994. In 1999, he also had started working on Plex86, also known as FreeMWare.

Kevin back then compared Plex86 to other tools such as VMWare, Wine, DOSEMU and Win4Lin. Plex86 in the meanwhile has been totally reinvented. While at first it was capable of running almost all operating systems, it is now a very light virtual machine designed only to run Linux.

Wine was also a frequently covered topic at different Linux Kongress venues. As its initiators claim themselves, Wine is not an emulator, but it most certainly used to be a key player in the virtualization area. Its attempts to run non-native applications in a different operating system, in this case mostly Windows applications on a Linux platform, didn’t exactly pass by unnoticed.

However, installing VMWare or Qemu became such an easier alternative than trying to run an application with Wine. And Win4Lin, its commercial brother, had similar adoption issues. Corporate adoption for neither Wine nor Win4Lin was successful, and Win4Lin recently reinvented itself as a Virtual Desktop Server product, where it is bound to face a lot of stiff competition.

People who claim desktop virtualization was ‘born in 2007’ obviously missed part of history. Although most Unix gurus claim desktop virtualization has been around for several decades via the X11 system, the Open Source alternatives to actually do the same on different platforms (or cross-platform) have also been around for a while.

Who has never heard of VNC, the most famous product that came out the Olivetti & Oracle Research Laboratory (ORL) in Cambridge, England? VNC was one of the first tools people began to use to remotely access Windows machines. System administrators who didn’t feel like running Windows applications on their Unix desktop just hid an old Windows desktop under their desk and connected to it using VNC. It was also quickly adopted by most desktop users as a tool to take over the desktop of a remote colleague. After the Olivetti & Oracle Research Laboratory closed different spin-offs of VNC such as RealVNC , TightVNC and UltraVNC popped up.. and it’s still a pretty actively used tool.

But VNC wasn’t the only contender in the field. Back in 2003, I ran into NX for the very first time , written by the Italian folks from NoMachine , with a FreeNX release co-existing alongside a commercial offering. It was first claimed to be yet another X reinvention, however NX slightly modified the concept and eliminated the annoying X roundtrips. The fact that NX used proxies on each side of the connection guaranteed that it could function even on extremely slow connections.

In the early days of this century, there was some confusion between UML and UMLinux. While Jeff Dike called his User-mode Linux the port of Linux to Linux, it was in essence a full blown Linux kernel running as a process on another Linux machine.

Apart from UML, there was UMLinux, also a User Mode Linux project, featuring a UML linux machine which booted using Lilo and from which an out-of-the-box Linux distribution could be installed. Two projects, one on each side of the Atlantic, with both a really similar goal and similar naming was simply asking for confusion. In 2003, the UMLinux folks decided to rebrand to FAUmachine. hence ending the confusion once and for all.

Research on virtualization wasn’t conducted exclusively in Germany; the Department of Computer Science and Engineering of the University of Washington was working on the lesser known Denali project. The focus of the Denali project is on lightweight protection domains; they are aiming at running 100s and 1000s VM’s concurrently on one single physical host.

And apparently, one project with a confusing name wasn’t enough. The Open Source community seemed desparate for more of that. Hence, the Linux-VServer project and Linux Virtual Server came around around the same time. The Linux Virtual Server actually hasn’t got that much to do with virtualization, at all. In essence, Linux Virtual Server is a load balancer that will balance TCP/IP connections to a bunch of other servers hence acting to the end user as one big High Performant and Highly Available Virtual Server. (The IPVS patch for Linux has been around since early 1999).

Linux VServer (released for the first time in late 2001) on the other hand provides us with different Virtual Private Servers that are running in different security contexts. Linux VServer will create different user space segments , so that each Virtual Private server looks like a real server and can only ‘see’ its own processes.

By then, Plex86 had a big competitor coming from France, where Fabrice Bellard was working Qemu. At first, Qemu was really a Machine Emulator. Much like Bochs (anyone still running AmigaOS?), you could create different virtual machines from totally different architectures. Evidently froml X86, but also from ARM, Sparc, PowerPC, Mips, m68k and even development versions for Alpha and alternative 64bit architectures. Qemu however was perceived by a lot of people as slow compared to other alternatives. There was an Accelerator module available providing an enormous performance boost, however that didn’t have such an open license as the rest of Qemu, which held back its adoption significantly. It was only about a year ago (early 2007) that the Accelerator module also became completely open source.

The importance of Qemu however should not be underestimated, as most of the current hot virtualization projects are borrowing Qemu knowledge or technology left and right. KVM (Kernel-based Virtual Machine) is the most prominent user of Qemu, but even VirtualBox, Xen (in HVM mode) and the earlier mentioned Win4Lin are using parts of Qemu.

As this is an overview of the recent Open Source Virtualisation history the focus has been on running virtual machines on Linux, or connecting to a remote platform from a Linux or Unix desktop, where most of the early developments have taken place. We shouldn’t fail to mention CoLinux in this regard, however. CoLinux allows you to run Linux as a Windows process, giving people on locked down desktops an alternative for VMWare to run Linux on their desktop.

Xen is with no doubt the most famous open source virtualization solution around, certainly after its acquisition by Citrix. Xen was conceived at the XenoServer project from the University of Cambridge, an initiative aiming to build an infrastructure for distributed computing and to create a place where one can safely execute potentially dangerous code in a distributed environment. Xen was first described in a paper presented at SOSP in 2003 but work on it began somewhere in 2001.

Next week, we’ll talk more about virtualization and open source with a detailed look at today’s landscape.