Today I got into a heated discussion with a “Virtualization Expert” at Gartner today about the risks associated with virtualizing your DMZ, primarily into the same environment as your non-DMZ servers.
As you can see in my diagram below, this is my plan; Create a completely isolated vSwitch with dedicated NICs for my DMZ portgroup, which is separated from the vSwitch that contains my Service Console, VMKernel and other Virtual Machine portgroup.
The discussion I got into this afternoon was that this was still a security risk, and presented a vulnerability to potentially all my other Virtual Machines on the host. Frankly, I just don’t see it. The only risk I could potentially ever see is if one of the guests in the DMZ portgroup was compromised, it could potentially affect other systems within that vSwitch — there should be no way it could cross talk to the other portgroups on other vSwitches in this system. Right? The Gartner engineer kept bringing up how someone could attack that vSwitch and gain access to the system to compromise it…then he kept giving examples about how this happened in Hyper-V (which has nothing to do with ESX).
So, I’m leaving this open…what is your opinion? Regardless of Intrusion Protection Systems and Intrusion Detection Systems and Firewalls, etc. – How strong is a vSwitch? Can someone that hacks a guest within an isolated vSwitch (one with no Service Console on it) gain access to your host?
Are these so-called Experts at firms like Gartner and Forrester really Experts?
You tell me….
:: December 5, 2008 by Rick Scherer
Posted under Networking, Security, VMware, this blog has 5,120 views and 23 responses.
(2 votes, average: 4.5 out of 5)
Loading ...
1:41 pm on April 8th, 2009
Hey Jerry, I think 3rd party vNetwork switches will resolve the PCI compliance issue. With vSphere you can now offload the network control back to the networking group.
http://www.vmware.com/technology/virtual-datacenter-os/infrastructure.html
The discussion on this post really discusses the FUD around placing DMZ and Non-DMZ virtual machines on the same host — technically there is no security risk just as long as the networks are on separate vSwitch.
4:48 pm on April 7th, 2009
The real issue here isn’t the security of the vSwitch or whether we trust the administrator. As more and more applications are deployed in the virtual environment, it becomes necessary to segregate the virtual network, and to the extent possible, do this without messing up all those great benefits we get from virtualization in the first place - resource pooling, DRS, vMotion, etc. Moreover, most compliance related requirements (ex. PCI) require that the person separating the network be a different person than the system admin - in this case the VMWare admin, exactly to the points above.. What’s needed is a virtual firewall that sits in between virtual switches and controls traffic according to a corporate policy. If it has the ability to detect and/or prevent attacks that originate from VM’s within a virtual subnet (that do not even make it to the IDS/IPS gizmo sitting on the perimeter) so much the better.
Provably resolving this issue will help us all gain the trust of application owners, auditors and analysts, allowing us to grow and expand our vEmpires.
Check out the product that won best of show for security at VMWorld last year. VMSafe coming makes it even better.
7:32 am on April 7th, 2009
I completely agree with you Tim, if the attacker is in the vSwitch that means they’ve already owned your box.
Typical FUD that we will deal with most likely forever. The product is mature (5+ years for anything IT is mature), the problem is “most” firms like the ones mentioned above have security experts discussing risk on a product they really know nothing about.
6:55 am on April 7th, 2009
I wrote a security class for VMWare and discussed this topic greatly in the course. The thing that no one has mentioned is that if the attacker has access to the vSwitch code then he also has access to the kernel. Why would he waste his time hacking into a vSwitch when he already owns the box. Keep in mind that vSwitches only process the frame data, nothing more. So the code they interpret would be very difficult to circumvent. This was purpously done by VMWare. Less complexity = more security.
This could very well change with the addition of the vSafe API to debut this year. While its intent is to strengthen security, but it will also add complexity. Refer to statement above..
I think this fear is there simply because of the unknown. Virtualization is a newer technology. Those technologies always get the blame for things that may or could go wrong. Remember when you implemented AD? Did it not also get the blame for a lot of things that went wrong during its implementation. I think it will just take time for the products to mature and develop a comfort level with the sys-admins.
8:55 pm on December 20th, 2008
Rick,
Thanks for the excellent clarification and the FUD debunking. I think what had me confused was when you said “the Gartner engineer kept bringing up how someone could attack that vSwitch and gain access to the system to compromise it.” Attacking the vSwitch sounded pretty ominous, but now I see what you are talking about. Thanks for your time!
2:55 pm on December 20th, 2008
It is not the vSwitch technology that is vulnerable, it is the fact of having different systems on the same lan segment (think of this the same as having multiple systems on the same physical switch with no firewall/ips type of protection).
If you are worried about the possibility of having your internal network comprimised then I would see no problem separating the Serv Con and VMKernel into their own vSwitch. You can have a maximum of 248 vSwitches so don’t worry about having one extra :) Remember though, even though your separating your internal network from the Serv Con. and VMKernel, if they are on the same subnet it still wouldn’t matter because an attack can still be attempted.
So lets sum up. The vulnerability isn’t in the vSwitch, its in putting different subnets on the same vSwitch (just like you wouldn’t do this on a physical switch). If you separate Serv Con./VMKernel and your VM network and they are on the same subnet, you are not achieving any stronger security goals.
2:38 pm on December 20th, 2008
I think I was unclear in my original question…
Is it more secure to have your internal VMs (Network0 in your diagram) on their own vSwitch? What I’m getting at is whether it would be a good idea to be just as paranoid about your internal VMs as you are with your DMZ VMs. If an internal VM gets compromised in your diagram, wouldn’t your ESX host be vulnerable?
Is their some inherent vulnerability in the vSwitch technology or this simply a basic networking problem were there’s no firewall between the hosts on a vSwitch? The nature of the vulnerability is unclear to me at this point?
2:16 pm on December 20th, 2008
Hey Cameron, as a best practice you would want to separate the physical NICs of your Service Console and VMKernel, but I really do not see a security risk running them on the same vSwitch. The only risk is if you would run a DMZ network on the same vSwitch as your internal network - which is the same risk as doing this in a physical world.
If your limited on NICs you can always use two NICs for your Serv Con. and your VMKernel, just set your active and standby settings in your failover order for each portgroup. (ie: Serv Con would be active on NIC 1 and standby on NIC 2, VMKernel would be active on NIC 2 and standby on NIC 1).
Good luck with your deployment and feel free to submit any questions you may have to me.
1:19 pm on December 20th, 2008
I’m starting to plan out an ESX rollout for this summer, so this is a great discussion.
Generally speaking, would it be even more secure to isolate your VMKernel and Serv Cons onto a separate physical NIC and vSwitch?
1:42 pm on December 9th, 2008
Rick,
Before I fully collasped our DMZ and hosted it as you have outlined above within our VMware enviroment I spoke with the two authors of the VMware “DMZ Virtualization with VMware Infrastructure:” paper and they confirmed (and many of the posts already have) that the only risk to the VMware environment is the same risk with a physical DMZ. Admin’s not knowing what they are doing. If you can mitigate that it’s a great cost and management savings. I doubt the Gartner “expert” really knows what he is talking about.
4:43 pm on December 8th, 2008
Look at the diagram above, the Service Console/VKernel and internal network are all part of vSwitch0 — vSwitch1 is dedicated for DMZ guests.
This ensures that your ESX host is still managed by an internal VC server, but you still can have DMZ guests on the same box for better utilization of resources.
4:17 pm on December 8th, 2008
Does your internal VC talk to the DMZ VM’s? How do you communicate with the DMZ VM’s?
4:14 pm on December 8th, 2008
Well since there is no link between the DMZ and our Internal network there is no way to put a virtual FW between them. There is already a physical firewall and IPS between our DMZ and Internal Network (two actually).
3:43 pm on December 8th, 2008
Just for added security I would place a FW appliance between your DMZ and your internal network…just a thought
2:55 pm on December 8th, 2008
Wasn’t Matt, but I think this is a common practice among all uninformed “virtualization experts” — just spout out FUD because you don’t know anything about what your talking about.
FUD + Unknowing Management = Pure Profit
2:11 pm on December 8th, 2008
Was this Gartner “expert’s” name Matt Cain? I have had issues with him spouting nothing but year-old MS FUD with one of my accounts, too. Very, very poorly informed.
1:03 pm on December 8th, 2008
vSwitch0 would still be within our private network, for internal Virtual Machines and service console access. vSwitch1 would be dedicated for DMZ based virtual machines.
12:47 pm on December 8th, 2008
Following the Vmware “Best Practices” document is a good start and depending on your level of paranoia will dictate your infrastructure design. My general rule is use best practices when possible dictated by the environment, be conservative, somewhat paranoid and if you are concerned about anything being hacked, virtual or physical - disconnect from the network!!! Not an option today…but remember layer2 is not exempt from attacks…My 20 years of experience says go forward with a vSwitch(s)for a DMZ.
Questions: Is your vSwtich0 connected to your internal network? What ports and protocols are being used to communicate with your DMZ hosts? How is VC communicating with your DMZ?
9:33 pm on December 6th, 2008
It always amuses we when people say that human error can cause an issue with the “collapsed DMZ”. How any people do you know that run the same IP subnet in their DMZ as in the normal network? I like most people run say 10.x.x.x internally but run 192.x.x.x in our dmz. If an admin accidentally selects the wrong vlan, the error will be picked up pretty quick when the guest vm fails to talk on the network due the wrong vlan been selected.
8:32 am on December 6th, 2008
I really appreciate all the input on this subject so far. For those of you that haven’t read my bio, I’ve been considered an “expert” in virtualization for over 5 years now, and the reason I designed my proposal the way I did was because I knew it would provide the greatest protection because of its isolation. Like I said in the post, this “expert” really got under my skin….but because he was from Gartner my upper management will most likely follow his suggestion of completely separating it on a hardware level as well.
Keep the comments coming, I will be sure to show all of this to my management for review (And proof that I am not alone in my beliefs!)
11:42 pm on December 5th, 2008
This “expert” obbiously don’t have a clue.
Just saying “it could” doesn’t prove the reality and if a competitor has the problem also proves nothing about VMware infrastructure.
This is an ongoing FUD attack without ANY basis in reality but unfortunately IT Managers don’t have the knowledge to debunk these type of “experts” from Gartner and will believe the lie and even promulgate it.
Gartner should check they’re “experts” before they loose all validity from the technical community.
PS. IDC has already gone that route.
Louw
9:08 pm on December 5th, 2008
Are you able to ignore them? Have you checked out http://www.vmware.com/files/pdf/dmz_virtualization_vmware_infra_wp.pdf ?
There is no risk of traffic between the vSwitches in your model. The risk is from human error and misconfiguration (which is really the same in the physical work IMHO).
Rodos
7:30 pm on December 5th, 2008
One of the risks is simply human error if somebody connects the wrong VM to the wrong switch down the road.
See this white paper for more detail and decision criteria for different DMZ topologies for virtualization
http://www.vmware.com/resources/techresources/1052