Chapter 3 - A New Beginning…

The first decade of a new century of a new millennium has recently ended.

The past ten years have been the most challenging, painful and yet most rewarding in my entire life. I’ve lost many friends, but I’ve gained many new ones. I’ve lost love, but have found new love that is deeper than ever before. I’ve found a greater respect for family and being together. I’ve also seen the world reach an all time high, then crash lower than it ever has before.

All of the situations, feelings and things that someone would normally endure over a lifetime, I’ve experienced in 10 short years.

Recently I’ve had to make a number of life changing decisions. Decisions that not only affect me, but will also affect my family, friends and a number of other people. Decisions that people typically handle one at a time.

For me, this was all thrown in my face at once.

One of the most stressful things to do in ones life, purchase a home, almost got the best of me. But, with a big sigh of relief we have recently closed escrow on a beautiful new home (two weeks later than we should have, adding additional unwanted stress to the situation). Now we get to choose carpet, paint, appliances and all of the other great things that home-ownership brings.

So, enter the second piece of the puzzle. For the past 8 years I’ve enjoyed a great position as a Solutions Architect in local government. In these past years, I’ve grown as a person as well as a professional. The things I’ve learned and the friends I’ve made will last a lifetime, but, it is time for me to continue to grow as a professional and take the next leap in my career.

Starting on February 1st, I will become an employee of EMC Corporation. I will be joining fellow bloggers such as Scott Lowe and Ed Saipetch, working as a VCE Architect in Southern California.  As Scott mentioned in his blog, this change in employment will not affect the blog in any way. In fact, it’ll let me blog even more as I’ll be reporting on exciting things coming out of the lab and in the industry in general.

It will be extremely hard leaving all that I have with my current employer, but like I said, this will be an enormous leap for my career and family. I want to thank everyone that has coached me along the way and the friendships that were made. I hope we can continue to work together in the future and continue to keep in touch.

As you can see, quite a bit of change all happening at once. Now that it is behind me I feel that a huge weight has been lifted from my shoulders, and I am excited to see what this new decade will bring.

Hello 2010…
Hello New Home…
Hello New Opportunity!


:: January 15, 2010 by Rick Scherer

Posted under Good Reading, this blog has 1,140 views and 11 responses.

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading ... Loading ...

NetXen HP NC522SFP Network Flooding

I had a very fun weekend. It started at 4am Saturday with a migration of ~125 virtual machines from an old AMD based environment to a new Intel Nehalem based environment. Who could’ve known that within a few hours all hell would’ve broken loose.

Enter in problem of network flooding from the NetXen based HP branded NC522SFP.  Because all of the 10GbE ports from the (9) new ESXi servers were creating thousands of pause frames on the Cisco Nexus 5020 switches, I thought originally that it was an issue on the switch.  Talks with Cisco revealed nothing.  We attempted to disconnect one of the connected ports (each ESXi host is dual connected into a pair of N5Ks using vPC) to remove a potential spanning tree loop….no dice.

A reboot of the host resolved the problem, things appeared to be running normally and we decided to let it be and wait until Monday.

10 hours goes by, it is now Sunday morning and the problem returns.  First host loses storage (we’re doing NFS over 10GbE here), then two more…until all 9 in this cluster are pretty much toast.  I decide to open a ticket with VMware.  Wouldn’t you know, there is a potential known bug and resolution.

Bug 496013

Description: Some NetXen based 10GbE cards using the unm_nic and nx_nic drivers sometime flood the network with pause frames causing the port to become disabled.

Resolution: NetXen believes upgrading the firmware to version 4.0.516 will resolve the problem.

I’ve gone ahead and patched 4 of the hosts with this new firmware, so far it has been stable (knock on wood).   I’ll let you know if something happens.

Checking which version of the firmware you’re running is simple. From a command-line (ESX or ESXi hidden CLI), type ethtool -i <vmnic#> (replace vmnic# with the alias to the vmnic you’d like to check).  You should see output similar to:

driver: nx_nic

version: 4.0.301

firmware-version: 4.0.406

bus-info: 0000:07:00.0

 

Update - Utility CD with firmware patch now included…

As you can see above, the firmware is out of date. To update the firmware you will need to boot from a Linux utility CD that has the appropriate driver, you then run a firmware update utility provided by HP.  To make this process easy I have created a bootable SLAX utility CD with the drivers pre-loaded. You can download the ISO from here. Once booted run the installer located in the root filesystem (ie: ./CP011471.scexe).

 

Let me know if you have any questions.


Posted under ESX 3.5 Tips, ESXi 3.5 Tips, Networking, Storage, vSphere, this blog has 1,151 views and 10 responses.

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading ... Loading ...

VMwareTips.com 2009 Holiday Giveaway Extravaganza - WINNERS

First of all, thank you very much to our sponsors for putting this together, EMC/IOMEGA and VEEAM.

Now the part you’ve all been waiting for.

Our third place winner, taking home some Veeam Backup and Recovery licenses is:

Gil Salazar from Tucson, AZ

Our second place winner, taking home a Veeam Management Suite license is:

Abe Lister from Glen Burnie, MD

(drum roll please)…and, our Grand Prize winner, taking home an IOMEGA IX4-100 is:

Herbert Tucker from Keyport, NJ

Once again, thank you to the sponsors and a very special Thank You to all the participants. Thank you for making VMwareTips.com so successful in 2009.   Happy New Year and here’s to a very prosperous 2010.


:: January 4, 2010 by Rick Scherer

Posted under Good Reading, this blog has 542 views and no responses.

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading ... Loading ...

Vote for VMwareTips.com

Wow, I cannot believe it is already 2010. I believe this year is going to bring some great things. For me it will bring a new home and the beginning of a new chapter in my life, I can’t wait!

I also feel it’s going to be the year of VDI and that virtualization is really going to hit the world even harder than it has already. I’m excited to see what’s in store for I.T. this year, and I’ll be even more excited to blog about it.

With that said, Eric Siebert at vSphere-Land.comis looking to update his Top Bloggers list. I’ve held positions #11 on the list for quite some time, and I’d love to maintain (or even go up on the list).  So please, go vote for VMwareTIps.com on his website, click here to vote.

Thank you for taking the time to vote, I really appreciate all of my readers visiting the site as well as e-mailing me comments and suggestions. It is because of you that the site has grown to over 85,000 visits per month from over 20,000 unique visitors. Cheers to you!


:: January 4, 2010 by Rick Scherer

Tags: ,

Posted under Good Reading, this blog has 497 views and one response.

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading ... Loading ...

ESX 4.0 Update 1A

A new patch has been made available for ESX 4.0 Update 1, this is called Update 1A. It only affects ESX and not ESXi. Here is an except of the alert put out by VMware:

ESX 4.0, Update 1, Alert: Upgrading ESX 4.0 to 4.0 U1 can fail or time out and leave the host in an unusable state if using HP Systems Insight Management Agent. ESX 4.0 Update 1a (a re-release of ESX 4.0 Update1) that addresses this issue is available. Please read KB article (ID 1016070) before proceeding with the upgrade.

As I said above, this patch is listed as ESX 4.0 Update 1A and can be found on the VMware Downloads website, or from within VMware Update Manager.


:: December 11, 2009 by Rick Scherer

Tags: , , , ,

Posted under vSphere, this blog has 1,734 views and one response.

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading ... Loading ...

Strange vCenter 4.0 U1 and ESXi 4.0 U1 SSL Issue

Last week I came across a problem that really stumped me, it even stumped the Tier-1 and Tier-2 support at VMware.  I’m posting the symptoms on here in a hope that someone else has experienced this issue and can share some light.

How about a little background on the environment, vCenter Server 4.0 U1 and multiple ESX(i) hosts (3.5, 4.0, 4.0 U1).   The vCenter Server as well as a number of ESXi 4.0 hosts were upgraded to U1 a couple days after it was released,  this problem however happened ~8 days after the upgrade.

Symptom 1: All ESX(i) hosts disconnect from vCenter Server, however, they are still online and no VMs went down.  Within 15 minutes all hosts appear to be reconnected.

Symptom 2: After the hosts reconnect, the ESX hosts appear to be functioning normally. However, the ESXi hosts display an error on the Overview tab as well as in the Events tab; “Unable to Synchronize with host that is unavailable.”

Symptom 3: Random VMotions start, for no apparent reason (DRS engaged, yet no constraints causing DRS to be invoked).  However, these VMotions fail at 10% due to the fact that the source and destination host is not available.

Symptom 4: /var/log/messages file displays errors with keywords: [VpxdVmomi] Error getting vpxa info: SSL Exception: Unexpected EOF From hosts, blacklisting showing up.   — I apologize for paraphrasing.

So, all this starts happening and I start investigating….pulling logs, restarting vCenter, and just sit there stumped.  I did notice that the rui.crt on the vCenter server expired, but back in 2008.  I went ahead and renewed the certificate and even restarted the entire vCenter server.  No luck.  I engaged VMware Support and their Tier-1 and Tier-2 support were stumped,  nothing even showed up in their internal database on this issue.

Then it all disappeared.  Roughly 90 minutes after it started, the problem just went away and everything was good.

Have you seen this issue?  What were your troubleshooting steps?  Did you resolve it or figure out the resolution?


:: December 9, 2009 by Rick Scherer

Tags: , , , , ,

Posted under vSphere, this blog has 1,469 views and 4 responses.

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading ... Loading ...

Pre-Order VMware vSphere 4 Administration Instant Reference

VMware vSphere 4 Administration Instant Reference, written by Scott Lowe, Jase McCarty and Matthew Johnson is now available for pre-order on Amazon.com. I was fortunate enough to work on this book as a technical editor, and must say that it is the perfect vSphere quick reference book for both the beginner vSphere admin as well as the seasoned veteran. Be one of the first to get your own copy, and order it now for under $20, it makes the perfect holiday gift!


:: December 9, 2009 by Rick Scherer

Tags: , ,

Posted under Good Reading, Training, vSphere, this blog has 1,032 views and no responses.

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading ... Loading ...

VMware Releases VI3 Update 5

Sometime yesterday ESX(i) Update 5 finally hit VMware Update Manager, about 2 days after the official announcement and release on the VMware website. This announcement includes updates for ESX, ESXi and vCenter Server. In addition to Update 5 being released there were about 20 additional updates made available for ESX(i), including 16 which were marked as critical.

The following information provides highlights of some of the enhancements available in this release of VMware ESX Server, this information can be found in the VMware ESX(i) 3.5 U5 Release Notes:

Enablement of Intel Xeon Processor 3400 Series– Support for the Intel Xeon processor 3400 series has been added. Support includes Enhanced VMotion capabilities. For additional information on previous processor families supported by Enhanced VMotion, see Enhanced VMotion Compatibility (EVC) processor support (KB 1003212).

Driver Update for Broadcom bnx2 Network Controller– The driver for bnx2 controllers has been upgraded to version 1.6.9. This driver supports bootcode upgrade on bnx2 chipsets and requires bmapilnx and lnxfwnx2tools upgrade from Broadcom. This driver also adds support for Network Controller - Sideband Interface (NC-SI) for SOL (serial over LAN) applicable to Broadcom NetXtreme 5709 and 5716 chipsets.

Driver Update for LSI SCSI and SAS Controllers – The driver for LSI SCSI and SAS controllers is updated to version 2.06.74. This version of the driver is required to provide a better support for shared SAS environments.

Newly Supported Guest Operating Systems – Support for the following guest operating systems has been added specifically for this release:

For more complete information about supported guests included in this release, see the VMware Compatibility Guide: http://www.vmware.com/resources/compatibility/search.php?deviceCategory=software.

  • Windows 7 Enterprise (32-bit and 64-bit)
  • Windows 7 Ultimate (32-bit and 64-bit)
  • Windows 7 Professional (32-bit and 64-bit)
  • Windows 7 Home Premium (32-bit and 64-bit)
  • Windows 2008 R2 Standard Edition (64-bit)
  • Windows 2008 R2 Enterprise Edition (64-bit)
  • Windows 2008 R2 Datacenter Edition (64-bit)
  • Windows 2008 R2 Web Server (64-bit)
  • Ubuntu Desktop 9.04 (32-bit and 64-bit)
  • Ubuntu Server 9.04 (32-bit and 64-bit)

Newly Supported Management Agents – See VMware ESX Server Supported Hardware Lifecycle Management Agents for current information on supported management agents.

Newly Supported Network Cards –This release of ESX Server supports HP NC375T (NetXen) PCI Express Quad Port Gigabit Server Adapter.

Newly Supported SATA Controllers – This release of ESX Server supports the Intel Ibex Peak SATA AHCI controller.

In addition to the enhancements found in ESX(i) 3.5 U5, there is also one lonely enhancement made to vCenter Server 2.5 U5:

Support for High Consolidation in VMware HA Clusters- VirtualCenter 2.5 Update 5 includes significant performance and scalability improvements to VMware HA. Use VirtualCenter 2.5 Update 5 for environments with more than 35 virtual machines per host in an HA cluster.
For information on the ESX Server host settings required for this scalability improvement, see ESX Server host settings required for environments with up to 80 virtual machines per host in an HA Cluster (KB 1012002).

Updating your ESX servers can and should be done with VMware Update Manager. To upgrade your vCenter Server installation you’ll need to download the installation ISO or ZIP from the VMware website and perform an in-place upgrade. Be sure to create a backup of your vCenter Server database then follow the steps in the Installation Guide.


:: December 6, 2009 by Rick Scherer

Tags: , , , , ,

Posted under ESX 3.5 Tips, ESXi 3.5 Tips, vCenter, this blog has 1,621 views and no responses.

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading ... Loading ...

VMware View 4 Available for Download

For my last post before leaving for the holiday I’d like to announce that VMware View 4 is available for download.

One prerequisite is that you’re running on VMware vSphere 4.0 Update 1 or at least Update 3 of the VI 3.5 Suite.

The most anticipated feature (IMO) with View 4 is full PCoIP support, which brings a full rich desktop experience regardless of the connection type (LAN or WAN). This truly means that virtualized desktops are a viable option for almost any environment now.  Another amazing feature of PCoIP is the ability to support up to four monitors so now even my desktop could be a virtual one.

Here are a few of the other new features found in the VMware View 4 Release Notes:

VMware View 4  Release Notes
What’s New

Enhanced single sign-on – The Log in as current user feature is integrated with Active Directory and smart cards to help simplify the process of logging in to a VMware View desktop.

Restricted entitlements – Administrators can control user access to virtual desktops based on the View Connection Server being used for authentication.

Smart card policies – Administrators can set group policies to force desktop disconnection and require reconnection when users remove smart cards.

Domain filtering – You can use vdmadmin.exe to control the accessibility of domains and traverse trust relationships more quickly.

You can cleanly delete View desktops using scripts.

You can log in to View desktops using user principal names (UPN).

You can explicitly configure IP addresses to override those supplied by the View Agent when accessing a desktop.

Mixed Active Directory and Kerberos authentication is supported.

From viewing the VMware HCL it appears that there are a number of Thin Clients that already have full support for PCoIP and View 4.

Another topic of discussion on Twitter was the Guest O/S support matrix, there were concerns that Windows 7 wouldn’t be supported as a Guest.  From what I’ve read in KB 1015591 it appears that there is full support for Windows 95/98, Windows 2000 Professional, Windows XP Professional, Windows Vista Ultimate, Business or Enterprise, and Windows 7.

So, go download your trial today and experience a true rich experience that PCoIP can provide.


:: November 25, 2009 by Rick Scherer

Tags: , ,

Posted under View, this blog has 1,759 views and 2 responses.

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading ... Loading ...

VMware vSphere 4.0 Update 1 Released

This is a couple days past due, but VMware has released Update 1 of their flagship vSphere product line.

One of the key drivers of this update is to provide support for VMware View 4 which was just released for download as well.

It also provides support for Windows 2008 R2 and Windows 7, for both as a Guest Operating System and base O/S for the vSphere Client.  This resolves the freeze issue in both of those O/S as discussed in my previous article.

Another hot update is the full support to utilize the pvSCSI adapter for your boot disk on Windows 2003 and Windows 2008.

Here are a few of the other items from the What’s New section of the Release Notes:

VMware vSphere 4.0 Update 1 ESX Release Notes
What’s New

Enhanced Clustering Support for Microsoft Windows – Microsoft Cluster Server (MSCS) for Windows 2000 and 2003 and Windows Server 2008 Failover Clustering is now supported on an VMware High Availability (HA) and Dynamic Resource Scheduler (DRS) cluster in a limited configuration. HA and DRS functionality can be effectively disabled for individual MSCS virtual machines as opposed to disabling HA and DRS on the entire ESX/ESXi host. Refer to the Setup for Failover Clustering and Microsoft Cluster Service guide for additional configuration guidelines.

Improved vNetwork Distributed Switch Performance Several performance and usability issues have been resolved resulting in the following:

  • Improved performance when making configuration changes to a vNetwork Distributed Switch (vDS) instance when the ESX/ESXi host is under a heavy load
  • Improved performance when adding or removing an ESX/ESXi host to or from a vDS instance

Increase in vCPU per Core Limit The limit on vCPUs per core has been increased from 20 to 25. This change raises the supported limit only. It does not include any additional performance optimizations. Raising the limit allows users more flexibility to configure systems based on specific workloads and to get the most advantage from increasingly faster processors. The achievable number of vCPUs per core depends on the workload and specifics of the hardware. For more information see the Performance Best Practices for VMware vSphere 4.0 guide.

Enablement of Intel Xeon Processor 3400 Series – Support for the Xeon processor 3400 series has been added. For a complete list of supported third party hardware and devices, see the VMware Compatibility Guide.

Resolved Issues In addition, this release delivers a number of bug fixes that have been documented in the Resolved Issues section.

Updating your environment to U1 isn’t that difficult, for vCenter Server simply download the installation files from VMware and install like any previous version.  Ensure that you choose the option to maintain your existing database, or else you’ll lose all of your data.

For updating your ESX hosts, simply use Update Manager or the Host Update Utility to perform these upgrades.

More information on Update 1 can be found here.


:: November 25, 2009 by Rick Scherer

Tags: ,

Posted under vSphere, this blog has 2,729 views and 2 responses.

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading ... Loading ...
Page 1 of 2012345»...Last »