Changing ESXi Hostname on an EVO:RAIL Appliance

The other day I did a VSPEX Blue (EMC’s flavor of EVO:RAIL Appliance) install for a customer that couldn’t conform to the hostname standards set forth by the EVO:RAIL Installation Wizard. After a little bit of digging and work I was able to change the hostnames and get everything working like it came from the factory. You simply cannot just change the ESXi hostname, you also need to modify EVO:RAIL Manager so things like health reporting actually function.

So, how do we do this?

First a quick warning… no guarantees :)  this worked for me and my customer and has been reported in a couple KB articles but due to the nature of EVO:RAIL you may want to open a ticket with your OEM Vendor for help on doing this.

It’s important to note two things… 1) if you’re building out a new EVO:RAIL Appliance and know you need to use custom hostnames, avoid putting in a DNS Server entry during installation – this will force the unit to use a local dnsmasq service for name resolution and will make it easier for us to modify the hostnames.  and 2) you should only place a single ESXi host into maintenance mode/change hostname at a time, this is important in allowing workloads to stay online.

And now on to the steps…

1.) On the vCenter Server Appliance modify the dnsmasq add-on file, by using vi /var/lib/vmware-marvin/dnsmasq/hosts – modify this file to reflect the new hostname(s)

2.) Place the first ESXi host into maintenance mode, then remove it from the vCenter inventory

3.) Add new hostname(s) to the local file on the vCenter Server Appliance, by using vi /etc/hosts – This is a temporary change to avoid any inconsistencies, we will remove these later

4.) Perform ESXi hostname change, I found the easiest way to do this was via esxcli by SSH’ing directly to the host

5.) Add new hostname to vCenter Server through vSphere (Web) Client, and exit Maintenance Mode

6.) Go to Step #2 for any additional ESXi hosts you’d like to change

7.) Restart the dnsmasq service on the vCenter Server Appliance; /etc/init.d/vmware-dnsmasq restart

8.) Clean up the /etc/hosts file by removing the changes you made in Step #3

That should be it for the changing of the ESXi hostname.  However, you may notice now that the Health tab in EVO:RAIL Manager isn’t looking too happy.
This is because of two things, obviously 1) the hostname change but also 2) the morefId changed which breaks the connection between EVO:RAIL Manager and vCenter.

So, let’s go fix EVO:RAIL Manager.

1.) On the vCenter Server Appliance we’ll need to first determine the new morefId of the ESXi host(s).  SSH into the vCenter Server Appliance and run the following command; sudo /opt/vmware/vpostgres/9.0/bin/psql -d VCDB vc -c”select id, dns_name from VPX_HOST;”

As you can see, our newname-esx01.vlab.local host has a new morefId of 26.

2.) Change the morefId and hostname by running vi /var/lib/vmware-marvin/hosts.json – change the morefId and hostname for the host(s) you’d like to modify

3.) Restart the EVO:RAIL Manager process with /etc/init.d/vmware-marvin restart

4.) Check EVO:RAIL Manager Health and all should be good!

It’s as easy as that. So, if your organization has a specific hostname requirement that cannot be met by the EVO:Rail Installation Wizard, have no fear… you can always modify after the build if needed!

Say you did an install without specifying a corporate DNS server, and you’d like to go back and add one after the fact all you need to do is add an entry into the /etc/dnsmasq.conf file. You can even add multiples.

server=IP.ADDRESS.OF.DNS.SERVER
server=IP.ADDRESS.OF.2ND.DNS.SERVER 

Then you need to restart the dnsmasq service,  /etc/init.d/vmware-dnsmasq restart

There you go, much better :)


Created on January 13, 2016 by Rick Scherer

Posted under EVO:RAIL.

This blog has 31,566 views and no responses.

Tags: , , , , , , , , ,

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

vSphere 5 Video Series – Install vCenter 5.0 in Around 5 Minutes

In this video we’re going to cover the installation, configuration and usage of the VMware vCenter 5.0 Server Appliance (vCSA). The vCSA is a brand new production ready Virtual Appliance that allows you to stand up vCenter Server in literally a few minutes. Once you watch the video you’re going to be like, “Hey Rick, that was more than 5 minutes!”.  For that I do apologize, but when you do watch it you will realize we’re doing a lot more than just installing vCenter 5.0.

First a little disclaimer. vCSA is not for everyone, but in my opinion it should definitely be looked at and should be leveraged wherever it can. vCSA is obviously the direction of where the vCenter Server product is going and hopefully relatively soon it should be at par with its Windows based big brother.

So, why isn’t it for everyone?

As of right now it does not have support for integration with VMware Update Manager (VUM), VMware vCenter Linked-Mode, VMware vSphere Storage Appliance (vSA), VMware vCenter Heartbeat and VMware View Composer. Another concern you may have is that it’s embedded database option (based on DB2) is limited to 5 Hosts and 50 Virtual Machines. Think of the embedded option to be similar to the SQL Express Option in vCenter Server for Windows, great for POC, Demo, Test and extremely small SMB situations, but not practical for production. The final nail in the coffin might be that it only supports Oracle to offer external DB functionality.

Some of those constraints are not going to be avoidable, for example if you require more than 1,000 hosts or 10,000 powered on virtual machines you’re obviously going to need Linked-Mode and the Windows based vCenter Server. If you’re looking to deploy a VDI solution based on VMware View, you’re going to need the Windows based vCenter Server as well. But, if you’re like the majority of VMware vSphere customers, have less than 1,000 hosts, are confident in VMware DRS and HA to protect your vCSA and are OK with the fact that you need Oracle for the external database (which you can virtualize as well)….the vCSA might be for you!

One last thing I wanted to comment on was VMware Update Manager, in my opinion the lack of VUM support for the vCSA might not be that big of an issue, and here’s why; With the introduction of vSphere 5.0, VMware also introduced a few new features, Auto Deploy and Image Builder. These features tied together with Host Profiles truly enable the concept of stateless ESXi. My thought is, if you need to update your ESXi host, simply update the Auto Deploy rule and reboot the machine. Upon the next boot it will automatically be updated and configured properly.  Obviously VUM does a lot more than just ESXi patching, but again, for the majority of vSphere customers they’d be just fine with Auto Deploy.

So have a view of this video to see just how easy it is. I have sped up some portions of the video, specifically the loading of the vSphere Client as well as the deployment of the vCSA OVF template. Also, I suggest watching the video in full-screen mode by clicking the icon on the bottom right of the video. If for whatever reason the video isn’t displaying, you can also use the following link to view; http://youtu.be/o2f1b1vYSis


Created on October 13, 2011 by Rick Scherer

Posted under vCenter, vSphere.

This blog has 2,201 views and one response.

Tags: , , , , , , , , , , , , , , , , , , , , , ,

1 Star2 Stars3 Stars4 Stars5 Stars (4 votes, average: 5.00 out of 5)
Loading...

vSphere 5 Video Series – Installing ESXi 5.0 in Under 5 Minutes

This is going to be the first of many in my vSphere 5 Video Series where I’ll cover the basics to getting vSphere 5.0 installed, configured and operating. In this video see just how easy it is to do a bare-metal installation of ESXi 5.0. Out of all of the install and upgrade options available for vSphere 5.0 this by far is the easiest and cleanest method in my opinion.

Upgrading can be extremely easy as well, by leveraging VMware Update Manager (VUM) it allows existing configurations to be migrated and will even allow you to migrate from ESX to ESXi. One thing to keep in mind when doing a migration with VUM from vSphere 4 to vSphere 5 is that if you are using the ESX edition of vSphere 4 and have custom scripts, agents or modules loaded into ESX those will not be migrated into ESXi 5.0.

Whatever your situation might be, my recommendation has always been to do a fresh installation of ESXi then leverage Host Profiles to push the configuration to the host. Even if you’re not an Enterprise Plus customer you can still get all of the benefits, like Host Profiles, free for 60 days by simply not licensing the product. Remember, you must license before the 60 days are up to avoid any service disruption.

Another great feature of vSphere 5 is Image Builder and Auto Deploy, I’ll cover Auto Deploy into more detail later, but with Image Builder you can build custom ESXi builds that include third party drivers and other custom data. Don’t worry though, you can still do custom installations with kickstart if you’d like, but after you see Auto Deploy you’re not going to want to.

So have a view of this video to see just how easy it is. I have sped up some portions of the video, specifically the blade server booting, the hardware discovery process and actually installation portion. Also, I suggest watching the video in full-screen mode by clicking the icon on the bottom right of the video. If for whatever reason the video isn’t displaying, you can use the following link to view; http://youtu.be/aN9mc9YNiC0


Created on October 10, 2011 by Rick Scherer

Posted under vSphere.

This blog has 2,327 views and no responses.

Tags: , , , , , ,

1 Star2 Stars3 Stars4 Stars5 Stars (2 votes, average: 5.00 out of 5)
Loading...

Patch for vmware vSphere 4.0

vmware has recently released a patch for their vSphere 4.0 product line, which affects both ESX and ESXi.

Details from vmware;

We are pleased to inform you that a new VMware ESX 4.0 Patch is available as of April 28, 2011.

Improvements included in this patch:

  • An update for the Certificate Revocation List (CRL) to revoke an RSA key that HP uses for code signing certain software components
  • Remediation of a denial of service possibility. By sending malicious network traffic an attacker could exhaust the available sockets which would prevent further connections to the host
  • Refinements in handling of shared folders

Detailed information regarding resolved and known issues and enhancements can be found at ESX 4.0 Patch Release Notes

VMware ESX 4.0 Patch is available for download at:
Download VMware ESX 4.0 Patch http://www.vmware.com/patch/download/.

Thanks,

VMware vSphere Product Management Team

One of the patches included (ESX400-201104401-SG for ESX and ESXi400-201104401-SG for ESXi) resolves a couple different issues, one updates the Certification Revocation List (CRL) to revoke a key that HP uses for code-signing certain software components. HP server contains a new key pair and has re-signed the affected software components with the new key. What this means is that if you apply this patch on a HP server and you are using specific HP management agents (like the HP Management Agent for VMware ESX 4.x) you will need to download the software with the updated key and re-install it.

The other fix within the above mentioned patch resolves a potential denial of service attack against the vmkernel over it’s management interface. When an attacker exhausts all available sockets the ESX(i) host will become inaccessible via vCenter or the vSphere client. Virtual Machines will continue to run and have network connectivity, but the ESX(i) host may need to be rebooted in order to be able to connect to the machine again. The ESX(i) system might intermittently lose connectivity caused by applications that do not correctly close sockets. If this occurs, an error message similar to the following might be written to the vpxa log file:
socket() returns -1 (Cannot allocate memory)
An error message similar to the following might be written to the VMkernel log file:
socreate(type=2, proto=17) failed with error 55
The Common Vulnerabilities and Exposures Project (cve.mitre.org) has assigned the name CVE-2011-1785 to this issue. More information on this patch can be found in KB 1037258 (ESX) and KB 1037259 (ESXi).

Another patch, specific to ESXi (ESXi400-201104402-BG), has also been released. The only information on this patch can be found in KB 1037553 which states “This patch improves the way shared folders are handled.”.


Created on April 30, 2011 by Rick Scherer

Posted under vSphere.

This blog has 1,647 views and 3 responses.

Tags: , , , ,

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

VMware vSphere 4.1 U1 Released

Yesterday, February 10th, VMware made available for general consumption U1 to their vSphere 4.1 product line. This update includes all prior patches as well as a number of new enhancements to the vSphere product suite.

This release provides the following improvements, I’ve included some notes along with the high-level updates:

VMware ESX/ESXi

  • Support for up to 160 logical processors
    • Prepared for the release of the Westmere-EX processor
  • Inclusion of additional drivers
    • 3ware and Neterion drives are now included
  • Enablement of Intel Trusted Execution Technology (ESXi only)
    • More information on this can be found in KB 1033811 
  • Additional guest operating system support
    • Provides support for RHEL 6, RHEL 5.6, SLES 11 SP1 for VMware, Ubuntu 10.10, and Solaris 10 Update 9 guest operating systems
  • Bug and security fixes

VMware vCenter

  • Additional guest operating system customization support
    • Windows 7 SP1, Windows Server 2008 R2 SP1, RHEL 6, RHEL5.5
  • Additional vCenter Server database support
    • SQL 2008 R2, SQL 2005 SP3, Oracle 11g R2, DB2 9.7.2
  • Bug and security fixes

VMware vCenter Update Manager

  • The VMware vCenter Update Manager Utility to help users reconfigure the setup of Update Manager.
  • Bug and security fixes.

VMware vCenter Orchestrator

  • Bug Fixes

For additional details regarding the new fixes and improvements, please refer to the following release notes:
VMware ESX
VMware ESXi
VMware vCenter

VMware vSphere 4.1 Update 1 is available for download from the VMware website as well as through VMware Update Manager (for your ESX and ESXi hosts). Keep in mind, the only way you can switch from ESX to ESXi is to do a fresh install. I’d highly recommend using Host Profiles to make this migration quicker and easier.

Jason Boche has already noted one potential issue with using VMware Update Manager (VUM) to upgrade from 4.1 to 4.1 U1, check that out here.


Created on February 11, 2011 by Rick Scherer

Posted under vSphere.

This blog has 4,699 views and one response.

Tags: , , , , , , ,

1 Star2 Stars3 Stars4 Stars5 Stars (2 votes, average: 4.50 out of 5)
Loading...

VMware vSphere 4.1 is Available!

Just about an hour ago VMware lifted the NDA on VMware vSphere 4.1 and made available all of the information on this latest release as well as the bits for download available to the public.

I will be covering a more in-depth review on this latest release really soon but I did want my readers to know that the bits are available for download from the VMware website.

http://downloads.vmware.com/d/info/datacenter_downloads/vmware_vsphere_4/4

The upgrade to ESX(i) 4.1 should be relatively easy by using traditional update methods such as VMware Update Manager (VUM).  However, the upgrade to vCenter Server 4.1 is more of a migration since it will only support a full 64-bit environment. Still don’t fret, a vCenter server migration is pretty simple just make sure you have a FULL backup of your vCenter Server database.

Also a little FYI….rumor has it that this will be the final build containing a full ESX install (Service Console). Today might be a good day to start planning that migration to ESXi.


Created on July 13, 2010 by Rick Scherer

Posted under vCenter, vSphere.

This blog has 2,432 views and one response.

Tags: , , , , ,

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

VMware ESX(i) 4.0 Update 2

You’ve guessed it, VMware has released Update 2 of their flagship bare-metal virtualization product ESX and ESXi. This update addresses a number of issues found since the release of Update 1 as well as a number of enhancements, such as:

Enablement of Fault Tolerance Functionality for Intel Xeon 56xx Series processors— vSphere 4.0 Update 1 supports the Intel Xeon 56xx Series processors without Fault Tolerance. vSphere 4.0 Update 2 enables Fault Tolerance functionality for the Intel Xeon 56xx Series processors.

Enablement of Fault Tolerance Functionality for Intel i3/i5 Clarkdale Series and Intel Xeon 34xx Clarkdale Series processors— vSphere 4.0 Update 1 supports the Intel i3/i5 Clarkdale Series and Intel Xeon 34xx Clarkdale Series processors without Fault Tolerance. vSphere 4.0 Update 2 enables Fault Tolerance functionality for the Intel i3/i5 Clarkdale Series and Intel Xeon 34xx Clarkdale Series processors.

Enablement of IOMMU Functionality for AMD Opteron 61xx and 41xx Series processors— vSphere 4.0 Update 1 supports the AMD Opteron 61xx and 41xx Series processors without input/output memory management unit (IOMMU). vSphere 4.0 Update 2 enables IOMMU functionality for the AMD Opteron 61xx and 41xx Series processors.

Enhancement of the esxtop/resxtop utility vSphere 4.0 Update 2 includes an enhancement of the performance monitoring utilities, esxtop and resxtop. The esxtop/resxtop utilities now provide visibility into the performance of NFS datastores in that they display the following statistics for NFS datastores: Reads/s, writes/s, MBreads/s, MBwrtn/s, cmds/s, GAVG/s(guest latency).

Additional Guest Operating System Support— ESX/ESXi 4.0 Update 2 adds support for Ubuntu 10.04. For a complete list of supported guest operating systems with this release, see the VMware Compatibility Guide.

Read More…


Created on June 10, 2010 by Rick Scherer

Posted under vSphere.

This blog has 6,156 views and one response.

Tags: , , , ,

1 Star2 Stars3 Stars4 Stars5 Stars (2 votes, average: 4.00 out of 5)
Loading...

VMware ESX(i) 4.0 Patch Bundle 06

As handful of patches have just been released by VMware for their flagship bare-metal virtualization products ESX and ESXi.

With no surprise to me the majority of the patches are for ESX and relate to security flaws and vulnerabilities found within the Service Console.  Keep in mind these vulnerabilities in no-way mean the virtual machines being hosted are at risk. These patches are typically for underlying services that the Service Console rely on, such as openssl, java, gzip and ntp. Sometimes these patches also resolve issues on how the Service Console communicates with the vmkernel layer as well as system devices.

Two of the patch bundles for ESXi share some common fixes with it’s ESX brother which cover a NTP vulnerability, a shared interrupt issue between the vmkernel and console as well as a patch that properly enables quiescing utilizing the Microsoft Windows VSS components found in Windows 2008 R2 and Windows 7.

More information on these patches can be found by reviewing the individual bundles;

ESX 4.0 – ESX400-201005001
Includes 9 updates, including fixes for NTP, gzip, bind, vmkernel, krb5, webCenter, Expat, sudo and gcc.

ESXi 4.0 – ESXi400-201005001
Includes two updates, ESXi400-201005401-SG for the ESXi firmware and ESXi400-201005402-BG for VMware Tools.

For updating your ESX(i) hosts, simply use Update Manager or download the patches from the VMware website and use the Host Update Utility to perform these updates.


Created on May 28, 2010 by Rick Scherer

Posted under vSphere.

This blog has 1,306 views and 2 responses.

Tags: , , , ,

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

NetXen HP NC522SFP Network Flooding

I had a very fun weekend. It started at 4am Saturday with a migration of ~125 virtual machines from an old AMD based environment to a new Intel Nehalem based environment. Who could’ve known that within a few hours all hell would’ve broken loose.

Enter in problem of network flooding from the NetXen based HP branded NC522SFP.  Because all of the 10GbE ports from the (9) new ESXi servers were creating thousands of pause frames on the Cisco Nexus 5020 switches, I thought originally that it was an issue on the switch.  Talks with Cisco revealed nothing.  We attempted to disconnect one of the connected ports (each ESXi host is dual connected into a pair of N5Ks using vPC) to remove a potential spanning tree loop….no dice.

A reboot of the host resolved the problem, things appeared to be running normally and we decided to let it be and wait until Monday.

10 hours goes by, it is now Sunday morning and the problem returns.  First host loses storage (we’re doing NFS over 10GbE here), then two more…until all 9 in this cluster are pretty much toast.  I decide to open a ticket with VMware.  Wouldn’t you know, there is a potential known bug and resolution.

Bug 496013

Description: Some NetXen based 10GbE cards using the unm_nic and nx_nic drivers sometime flood the network with pause frames causing the port to become disabled.

Resolution: NetXen believes upgrading the firmware to version 4.0.516 will resolve the problem.

I’ve gone ahead and patched 4 of the hosts with this new firmware, so far it has been stable (knock on wood).   I’ll let you know if something happens.

Checking which version of the firmware you’re running is simple. From a command-line (ESX or ESXi hidden CLI), type ethtool -i <vmnic#> (replace vmnic# with the alias to the vmnic you’d like to check).  You should see output similar to:

driver: nx_nic

version: 4.0.301

firmware-version: 4.0.406

bus-info: 0000:07:00.0

Update – Utility CD with firmware patch now included…

As you can see above, the firmware is out of date. To update the firmware you will need to boot from a Linux utility CD that has the appropriate driver, you then run a firmware update utility provided by HP.  To make this process easy I have created a bootable SLAX utility CD with the drivers pre-loaded. You can download the ISO from here (file temporarily removed). Once booted run the installer located in the root filesystem (ie: ./CP011471.scexe).

Let me know if you have any questions.


Created on January 11, 2010 by Rick Scherer

Posted under ESX 3.5 Tips, ESXi 3.5 Tips, Networking, Storage, vSphere.

This blog has 21,612 views and 40 responses.

Tags: , , , , , , , , ,

1 Star2 Stars3 Stars4 Stars5 Stars (2 votes, average: 4.00 out of 5)
Loading...

Strange vCenter 4.0 U1 and ESXi 4.0 U1 SSL Issue

Last week I came across a problem that really stumped me, it even stumped the Tier-1 and Tier-2 support at VMware.  I’m posting the symptoms on here in a hope that someone else has experienced this issue and can share some light.

How about a little background on the environment, vCenter Server 4.0 U1 and multiple ESX(i) hosts (3.5, 4.0, 4.0 U1).   The vCenter Server as well as a number of ESXi 4.0 hosts were upgraded to U1 a couple days after it was released,  this problem however happened ~8 days after the upgrade.

Symptom 1: All ESX(i) hosts disconnect from vCenter Server, however, they are still online and no VMs went down.  Within 15 minutes all hosts appear to be reconnected.

Symptom 2: After the hosts reconnect, the ESX hosts appear to be functioning normally. However, the ESXi hosts display an error on the Overview tab as well as in the Events tab; “Unable to Synchronize with host that is unavailable.”

Symptom 3: Random VMotions start, for no apparent reason (DRS engaged, yet no constraints causing DRS to be invoked).  However, these VMotions fail at 10% due to the fact that the source and destination host is not available.

Symptom 4: /var/log/messages file displays errors with keywords: [VpxdVmomi] Error getting vpxa info: SSL Exception: Unexpected EOF From hosts, blacklisting showing up.   — I apologize for paraphrasing.

So, all this starts happening and I start investigating….pulling logs, restarting vCenter, and just sit there stumped.  I did notice that the rui.crt on the vCenter server expired, but back in 2008.  I went ahead and renewed the certificate and even restarted the entire vCenter server.  No luck.  I engaged VMware Support and their Tier-1 and Tier-2 support were stumped,  nothing even showed up in their internal database on this issue.

Then it all disappeared.  Roughly 90 minutes after it started, the problem just went away and everything was good.

Have you seen this issue?  What were your troubleshooting steps?  Did you resolve it or figure out the resolution?


Created on December 9, 2009 by Rick Scherer

Posted under vSphere.

This blog has 4,047 views and 5 responses.

Tags: , , , , ,

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...