VMware infrastructure upgrade is one of the my 2019 first quarter goals. Upgrading vCSA and replace all the esxi hosts with the new hardwares. I am near completion. I am documenting the process for my own reference as well as for others who wish to refer to it.

Part 1: Install a new vCSA 6.7 update 1
I was having an issue with vCenter running 6.5. Every five seconds or so an non-existant AD account had login failure coming from vCenter. Nothing was registered on any of the log files in vCSA. This provided me an opportunity to do a brand new appliance installation.

Installed vCenter Appliance 6.7 U1b which was straight forward. I just had to create a new data center object, new cluster and a new cluster and migrate all the vms.

Part 2: Install new hardware to replace the esxi hosts.

1. Hardware prep – firmware updates, NIC connections. Connect SAN cables only after installing ESXi.
2. Make sure ports on access switches are configured properly for all vm management, vm network, fault tolerance and vmotion.
3. Hardware are from HPE, so used HPE customized iso to install ESXi hosts.
4. Configured the networking of the new hosts to match the old ones. The most important aspects being the portgroup names and VLANs.
   – VM network protgroup name must exactly match the old one. kernel and management portgroups are not required to match their names.
   – If your current networking is on a distributed switch, you can simply add the new hosts to the existing on
5. Added the news hosts to storage array host clusters and configure zoning in SAN switches.
6. Ensured that the new hosts can access the same shared storage as the original cluster.
7. Created a new cluster object and added the new hosts to it.
8. Before adding the host, enabled ssh from dcui or else adding host to the vCenter will fail.
9. When convenient, power down the VMs and migrate them to the new cluster.
10. When fully evacuated, remove the original hosts from the vCenter’s inventory.
11. Add the licenses from the original hosts to the new hosts.

If your VM network has less than three physical uplink, do not enable “Beacon Probing” in Teaming and Failure for network failure detection. Following issues occurred when I enabled the above:
1. Ping DUP!

ping 10.0.1.5 ( a linux vm )
PING 10.0.1.5 ( 10.0.1.5 ) 56(84) bytes of data.
64 bytes from 10.0.1.5: icmp_seq=1 ttl=64 time=0.407 ms
64 bytes from 10.0.1.5: icmp_seq=1 ttl=63 time=1.43 ms (DUP!)

2. As soon as I powered VMs on a new cluster, hosts complained that network connectivity lost and network redundancy failed.