video

During the Fosdem 2008 conference, we had a chance to sit down (on a bench) with Xen Guru Ian Pratt. Below is the second part (watch the first part here) of our exclusive interview, where Ian shines his light on the Xen GPL license, OracleVM, xVM (Sun), the future of virtualization and XenServer.

We cut the interview into 4 digestable pieces, which we publish one at a time. As said, this is the second part (you can also find a written transcript below for your convenience):

This video is also available on Vimeo and Streamocracy.

(0:02) Could Virtualization also help the infrastructure to become more self-healing or self-provisioning?

“Sure. It is already the case that you can have a pool of physical hardware. Something that Xen calls a resource pool and than a pool of VMs running on top. You can configure things as the referrer to give “down notices”, so you can fail over those virtual machines. There are plenty of people who do that today on Xen.”

(0:31) When I look at the Xen GPL-license. I find it interesting that Xen is being renamed as xVM by Sun, OracleVM by Oracle. When Oracle first announced OracleVM it quickly had to admit it was actually a tweaked Xen version. I heard they initially did not publish the tweaked code.

“Oh no, they have. The fact is that there are lots of different vendors, shipping Xen products as they pick up the Xen hypervisor core engine and incorporate it into their own products. The Linux vendors: like Novell and Redhat, there is Sun, there is obviously XenSource / Citrix and Virtual Iron. Lots of different companies are doing that. Actually the GPL license means that any changes they make will go back into the main project. In reality, pretty much all those company just pick it up as is. They take the latest stable release, which is maintained, they might add the odd little patch to it, but it really is all very clear, there is a lot of uniformity in the Xen versions out there.”

(1:40) As many of these projects like Sun’s xVM and Oracle’s VM are using the Xen project. At what level are they tweaking their own software to be more integrated with Xen or to be more stable or faster?

“Most of those companies are very close to mainline Xen. They post a couple of patches. In some cases they maybe not. What they will be doing, is taking Xen and it’s really on top of Xen , in the rest of their Virtualization stack (that runs in user space) that’s where they’ll be probably doing their own things. They might have their own management tool. They will have their own way of wanting to present virtualization to the user. So if you think about what the operating system vendors are typically doing, is they want to expose virtualization using the same tools and user interfaces as they use for exposing other facilities inside that operating system. Which is quite different from what a company like XenSource was trying to do, which tried and effectively build a virtual machine hosting appliance. You just put the CD in the server, install it and just manage it from a windows GUI or web interface. Every company is bringing Xen to market in a different way for a different kind of user. And that is where the differentiation happens. The core engine is the same throughout.”

(03:31) You think that is what the future will bring us? You buy a piece of hardware and just initiate it, to install an operating system.

“I think it will go way further than that. We have always envisioned getting Xen embedded in the firmware as we think that the hypervisor is a core part of the platform. We think it should come with servers when they role of the production line and they should all have Xen installed on them. And the really cool thing is that this is happening. Dell is already announcing that in their new servers shipping later this year, Xen will be a factory installed option in flash memory. Other hardware vendors are to follow soon. People will have ubiquitous virtualization, every server will have Xen installed on it. You will be able to install multiple operating systems and virtual appliances, etc. on top of the hardware.”

(04:33) So now the x86-type of servers are becoming very similar to what mainframes are used to for decades?

“Exactly, it will be a similar model. The difference will be that you can start using these x86-servers, connecting them into resource pools and than running pools of VMs on top of these pools of servers. That is when things start becoming really interesting.”

(04:58 ) You think people will need to rethink their whole infrastructure even more drastically than they do today?

“Yes, today virtualization is typically used for sort of server consolidation. Often used for taking legacy applications or old versions of operating systems and consolidating them onto a single machine…
I think that the way that things are going to be tomorrow and start happening today (and for which Xen is brilliantly prepared) is actually for running production workloads, where all of your machines and partners are running hypervisors and that enables you to run any virtual machine image on any physical machine to take advantage of being able to move workloads around by using live relocation. Also balancing of VMs to servers and even features like fault tolerance and the stuff we talked about, which you want for production workloads.”

(05:59) If you ask people to name a virtualization vendor, VMware will probably come up first. They definitely have a track record to have built this market. But if you look at really big IT-datacenter applications like Amazon, Google or MySpace, they actually deployed Xen as their core engine. It appears all of the Fortune top 100 companies in the United States are VMware clients. So why do banks go for VMware and these major datacenters for Xen?

“I think you will see plenty of banks switching to Xen and plenty of them already have, as it is obviously a lot cheaper to deploy Xen. The reason that companies such as Amazon go with Xen, is that when you do these large virtualization deployments, you want to be using something to secure great performance and some of the high-end second-generation virtualization features. Xen certainly has all that and also has the advantage that it is open source. So there is not going to be vendor lock-in, with a number of different Xen-vendors to procure from and the price is right. For a sophisticated company like Amazon, they will just download the open source version and they will have 20 engineers deploying it across machines. There are plenty of other companies that will rather tank one of the pre-packaged versions from one of the Xen-vendors. I think that many of the large virtualization deployments -such as Amazon- are on Xen because it works better.”

Go back to part 1 of this interview

Some time ago, we had a chance to sit down (on a bench) together with Xen Guru Ian Pratt, well known for co-founding and ultimately selling XenSource – the company behind the open-source Xen project – to Citrix in October 2007.

This exclusive interview was taken as part of our video coverage at the Fosdem 2008 conference held at the ULB (Brussels Free University, hence the “inspiring” Solbosch campus background). Toon Vanagt, owner and publisher of Virtualization.com, interviewed the rather jet-lagged Ian Pratt on a sunny Sunday morning about Xen, XenServer and the virtualization landscape as a whole.

We cut the interview into digestable pieces which we will publish one at a time. Here’s the first part, the second part can be found here (you can also find a written transcript below for your convenience):

This video is also available on Vimeo and Streamocracy.

Hello Ian Pratt, you are one of the founders of XenSource, which was recently renamed to XenServer after it was acquired by Citrix. Could you give an introduction to para-virtualization, hypervisors or OS-enlightment as Microsoft likes to market it?

“The work on Xen really started in the University of Cambridge back in 2001 as we were interested in figuring out the best way to build virtualization systems. We realized there were two techniques which -when used together- were going to enable you to do a great job at virtualization.

One is getting facilities into the hardware to make the job of virtualizing the platform easier. This means getting stuff into the CPU, chipset and in particular into the I/O-devices, like the NICs and hostbus adapters. But second also working with the operating system vendors to try to get stuff into the operating system to enable the OS to call down into the hypervisor to work better in a virtualized scenario.

We pushed hard on both of those fronts, working to design network interface adapters that had this special hardware support and also working to add these extensions into operating systems like Linux and then other free operating systems and now even an OS like Microsoft Windows. That is how we get to this current generation of virtualization software that really is able to achieve great performance and have great security to provide all the great benefits of virtualization.”

(1:48) Ian, it is quite remarkable that the Xen project is one of the rare open source software projects that actually managed to get its feature requests into the large hardware vendor production. How did you achieve this?

“Well, there is a long lead time on getting anything build into hardware. As Xen had been running for a quite while as a university project, we were talking to all the different hardware vendors. You have to remember in the early days Xen was sponsored by some of those vendors and also working with the operating system vendors. Also we did things like build network interfaces that had these facilities in.
We prototyped them and wrote papers about them. And then companies really began to see that virtualization was important. Let’s be fair: VMware had a great part to play in showing the world that virtualization was important and then I think Xen has done a great job at showing people how to it should really be done.”

(2:47) It is interesting you mention VMware, because Xen is an open source project and VMware remains a closed source product to date. One of the major challenges for people looking at which vendor to select, is the specific Virtual Machine format and how to avoid vendor lock-in. So what is your opinion on the Open Virtual Format (OVF) and how do you see the evolution in this field.

“OVF actually came about as a collaboration between us and VMWare. We had been working on a format we called the Open Virtual Appliance (OVA) and had been putting quite a bit of work in to that. We were obviously really concerned about the interoperability issues. We had a discussion with VMware as they had been working on their next-generation format for their hypervisor and we actually collaborated together and came up with the OVF specifications. And now both sides are implementing that. We will have to see how it works out in practice. You still have to do a certain amount of preparation on the virtual machine to make it able to work on both platforms and it is really down to the people who produce virtual appliances to follow the best practices and make sure their Virtual Machines are portable. But at least now there is a common file format and metadata format for transferring things between different virtualization solutions. Or at least there will be in the future when it is implemented and ratified by the DTMF and al that boring stuff is out of the way.”

(4:20) So you think that once these meta-data have been defined for Virtual Machines and have adopted by Xen, VMware and Microsoft, we will actually be able to do Vmotion or Virtual Machine Relocation between those different vendors?

“Doing live virtual machine relocation is kind of like changing the engine on a plane in flight!

That is certainly further down the road. OVF is really about having a format in which you can package a given virtual appliance, which might actually consist of the multiple virtual machines and install it onto a given hypervisor and have it run there. And hopefully you also will also be able to use it for moving an installed virtual machine between different hypervisors, but there is a way to go, before we can do this live relocation. It is a worthy end goal, but there is a lot of stuff that would need to happen to make that work.”

(5:16) It is one of those things, when you see it happen for real; which now creates a strong WOW-effect in Virtualization.

“It certainly is and it would be nice to be able to live relocate a virtual machine from Xen to Hyper-V or to VMware, but there is a lot of work to do.”

(5:34) Will VM-mirroring ever be possible?

“Absolutely will not only be possible, it has existed for some time. There is some great work that has been done and a couple of things to point out here. One, there is a commercial product available on top of XenServer, which does this today by a company called Marathon Technologies, where they have 2 virtual machines running on different physical hardware on top of Xen and they are synchronizing the state between the two in real-time to the extent that you can just walk up to one of these machines and yank the power cord straight out from their back. None of the users of these applications or services provided on that server will even notice anything has happened, because it instantaneously (or within milliseconds) failed over to the other VM.

So that was the commercial product. There is also a lot of great work going on in open source. For example a project at the University of Michigan, using a technique called deterministic replay. That is very cool. Also work done by the University of British Colombia on a project called Remus, which I think is really cool, because it works for Virtual Machines that are multi-processor, so you can have an SNMP guest and you could be synchronizing that VM-image to another machine. It is looking like they do not even necessarily need to be in the same building. You might be able to synchronize over a suitably fat pipe across the wide area network. You can use it for disaster recovery. We want to get this cool stuff into mainline Xen.”

(7:16) When looking at VM-relocation, the typical reasons people use this for is either to avoid downtime, disaster recovery and high availability, to relocate workloads or to enforce security policies: either with fire walls inside the VM or to lock the OS at the root-level. Can you tell us something more on those security policies you can enforce in Xen?

“One of the nice things you can do with Virtualization is that you can actually stand outside the OS and look into it. And implement some of these facilities which you would normally do using software installed within the VM. You can now actually do it outside and you do not have to worry whether the administrator has actually configured the fire wall, virus scanner or back-up correctly within the VM. Because we can actually do all of these tasks from outside now. I think that is going to be a far more common thing in the future, where you will try to take care of all of those things within the virtualization layer, so that your administrator of the VM does not have to worry about or risks to mess it up. You can kind of protect administrators from themselves.

You will see virus scanners running as part of the virtualization stack or platform and these will scan the contents of all of the VMs running on it. It is like taking the firewall that you might have on the edge of the network, where it connects to the outside world and kind of pulling that in, to put it closer to the VMs that are actually running applications and actually implementing that firewall in a distributed fashion across all of your virtualized platforms.”

Watch the second part of the interview here.