Issues with Controller replacement and Firmware upgrade on HP MSA 2312fc
Recently I was pulled into an issue involving an HP MSA 2312fc array. The device had a failed Controller B and for some reason it was also affecting the A Controller. This was causing the OS on the servers receiving storage from the device to loose connection to the vdisks. The way things are suppose to work with the MSA as in any array is for the other controller to take ownership and continue to provide connections to the vdisks. For some reason this did not happen.
When looking at the management console on the MSA 2312 you could clearly see that controller B was failed and that A had taken ownership over of any vdisks. But something with the fail over did not fully complete or corrupted something because it did not work as expected. Even with removing the failed controller from the device the vdisks were still not usable by the servers.
Once the failed part was replaced the configuration was copied to the replaced controller and service was restored. Then it was time to make sure the firmware levels matched on both controllers and update accordingly.
In the past I had been warned to proceed with caution about updating firmware levels on MSA arrays. I had read several horror stories about them getting stuck in loops during the update process. We did encounter this when the controllers tried to sync them selves to match the firmware levels. I was told by an HP Support Engineer that it’s best to disable this auto sync feature when it comes to updating firmware to prevent these loops. It can then be enabled after the updates are done on both controllers.
Below are a few steps to prevent the automatic firmware update from the partner controller. The first image below shows you how to access the Firmware options from the Advanced Settings area of the Configuration menu.
The next image is showing the option to turn on or off the Partner Firmware Update option. This determines if the firmware on a new controller is auto updated when it is inserted. The HP support engineer recommended that we disable this option prior to installing the new controller. If you did not follow this you can always come back and change and they wait for the current update to fail. This is what we had to do that ended up stopping the continuous loop that was happening.
The last image just shows what you will see if you log into the management controller that is being updated.
About Brian Suhr
Brian is a VCDX5-DCV and a Sr. Tech Marketing Engineer at Nutanix and owner of this website. He is active in the VMware community and helps lead the Chicago VMUG group. Specializing in VDI and Cloud project designs. Awarded VMware vExpert status 6 years for 2016 - 2011. VCP3, VCP5, VCP5-Iaas, VCP-Cloud, VCAP-DTD, VCAP5-DCD, VCAP5-DCA, VCA-DT, VCP5-DT, Cisco UCS Design