When looking at the management console on the MSA 2312 you could clearly see that controller B was failed and that A had taken ownership over of any vdisks. But something with the fail over did not fully complete or corrupted something because it did not work as expected. Even with removing the failed controller from the device the vdisks were still not usable by the servers.

Once the failed part was replaced the configuration was copied to the replaced controller and service was restored. Then it was time to make sure the firmware levels matched on both controllers and update accordingly.

In the past I had been warned to proceed with caution about updating firmware levels on MSA arrays. I had read several horror stories about them getting stuck in loops during the update process. We did encounter this when the controllers tried to sync them selves to match the firmware levels. I was told by an HP Support Engineer that it’s best to disable this auto sync feature when it comes to updating firmware to prevent these loops. It can then be enabled after the updates are done on both controllers.

Below are a few steps to prevent the automatic firmware update from the partner controller. The first image below shows you how to access the Firmware options from the Advanced Settings area of the Configuration menu.

The next image is showing the option to turn on or off the Partner Firmware Update option. This determines if the firmware on a new controller is auto updated when it is inserted. The HP support engineer recommended that we disable this option prior to installing the new controller. If you did not follow this you can always come back and change and they wait for the current update to fail. This is what we had to do that ended up stopping the continuous loop that was happening.

The last image just shows what you will see if you log into the management controller that is being updated.

About Brian

Brian is a Technical Architect for a VMware partner and owner of this website. He is active in the VMware community and is helps lead the Chicago VMUG group. This blog Virtualize Tips was started to document and remember things that I come across while working with tech.

Mail | Web | Twitter | LinkedIn | More Posts (170)

So it is with that preface that I arrive at discussing the SSSU: HP’s Storage Scripting System Utility. The premise of this utility is to allow command line access to an EVA. The idea is, at least from HP’s perspective, to use it for repetitious activities that would be otherwise tiresome in Command View. Additionally, you can also issue interactive commands like restarting a controller or changing mastership for disks between controllers manually. HP bundles the SSSU installer with Command View’s installer package. It carries the same versioning as Command View (so in my case, 9.1.0).

SSSU seems like a really great idea. I love the command line (being very partial to a bash shell ), but SSSU isn’t a real CLI in that sense. The first round of trouble is in the output: SSSU generates XML based output that a human can’t simply open up and read. This throws out a simple way to grab information that would otherwise take a lot of time via CV. This posting is an overview of SSSU, and I will write a follow up post that goes more into depth on what you can do with the output to make it actually usable in conjunction with Microsoft’s Logparser tool. Sad I know. You can pony up big $ to get actual tools but hey, in tight times, stretching the imagination is better than stretching the wallet. I’ll also put up some of the scripts I use to gather data on disks and such (ex: easy way to track disk size and growth).

At this point, let’s get SSSU fired up and play around a bit. To launch, log into your CV server and run that fancy icon on the desktop. One quick gripe: no way for me to install this on my own laptop to just run and connect with. To get around that, I copy over from the CV install to my local machine the SSSU.exe file. It runs great and it’s just that less than 2 meg file. Upon launching it, you get to login.

You get three parts to a login:

You can login with a domain account without any issue, so you don’t have to create a whole separate user account list. One thing to note is that although you can fire up multiple SSSU windows to one array, you can only execute commands to it one at a time (serial baby!), so when one command is running in one window, your other windows would hang if you tried to execute a command).

After logging in, you are sitting at a default path, so to speak. That being that you have not selected your array.

Hitting ? and enter will list off all the commands available. This can also be used to find out what options each command has available.

Object names are case sensitive and are organized in a root tree system. Commands and objects with spaces in their names require double quotes. The first thing to do is select an EVA to work with, using the term select system.

You can flip around using the select system command. At this time there isn’t a way to issue group commands (outside of scripting, and even then it’s not a group command, in that sense).

Root Structure of Objects:

\Hosts

“\Disk Groups”

“\Virtual Disks”

\Hardware

Note that you can create objects (like folders) under all but \Hardware. Objects created are purely for human organization. The EVA doesn’t care if you have them or not (though if you do create them you need to be aware of the “path” to them.

A complete listing of the commands can be found in the SSSU reference doc, found here:

http://bizsupport1.austin.hp.com/bc/docs/support/SupportManual/c02493404/c02493404.pdf

This link is for the latest (I believe rolled into Command View 9.3) as of this date (9.23.2010).

My advice would be to look through the command reference to understand what the commands do. Listing them all out here in a blog post would be silly. And a lot of work, and I’m lazy, so….

As promised before, I will follow up this post with a more in-depth guide for where SSSU is handy and probably some more complaints on how it’s irritating. J

About Mike Mills

Mail | More Posts (20)

So as you can see, we have a total Capacity (which is correct and accurate), and the Occupancy. The information listed is correct, and it’s refreshing. Now, for the problematic Disk Group (and please ignore the name as it was before my time).

Here we see the correctly calculated Capacity, but lo! What’s this? What on earth is going on with that Occupancy? It seemingly continues to grow a bit here and there as we take snapshots of various. HP doesn’t know what’s wrong either. When we upgrade from Command View 9.0 to 9.1, it did not fix the issue either. The big problem here is that we have to figure out how much space is actually used since we’ll never get alerted. It is a pain. I can turn to Replication Solutions Manager to obtain the correct size by simply adding all the luns (easy way to view them, as opposed to Command View), so that’s the workaround. Not very satisfying if you consider how much money was spent on these beasts.

About Mike Mills

Mail | More Posts (20)

Launched from the shortcut on the Windows host, or is accessible via a web browser at https://servername:2372

You are presented with the login screen:

It is here that you will also see the version number (down below). We have not yet upgraded this server to version 9.2 as we are building a VM to run Command View from and retire this physical box.

After login, you are greeted by the overview page. Here you will see all the EVAs listed, as well as the stats for your overall environment. The first thing of note is that two of my EVAs have bang lights on them, indicating something is amiss. I’ll investigate both as part of my next posting.

On the top right, there are some hyperlinks, your login id, and the ip of the Command View server. Most are self explanatory. Server Options provides for a place to enter license codes, setup RSM relationships, and a few other features. I seldom visit this page, so like once a year?

Moving right along, let’s take a look at a healthy EVA, in this case DS-SAN-2:

As you can see from the above, a healthy EVA has a number of folders beneath it. It breaks out into many subfolders. On the right hand side are the numbers. It would be here that you can get the logs for HP support (I’ll cover that in a separate blog post). You can see the current capacity level, view the Version level (this is the XCS code release running on the EVA. 6220 is the latest for the 8100 series). The left column is broken down like such:

Virtual Disks – this is where the luns (vdisks in EVA-speak) are listed. The folder structure is entirely man-made. That is to say, it’s for human organizational purposes (and plays an important role for setting up RSM jobs. More on that will be covered on a separate posting).

Hosts – here is where hosts are setup in Command View. You will provide the host’s OS, and it’s WWN’s for fiber cards. Hosts MUST be setup on every single EVA that you want to present disks from. Annoying, I know.

Disk Groups – these are comprised of physical disks. Best practice says to build these in multiples of 8, and of all the same speed and size. You can choose to not follow these and your performance will suck majorly. I’ve worked on rebuilding two of the 4 EVAs that had improperly constructed Disk Groups. It is PAINFUL to correct, but I’m glad I did. I will cover that also as a separate blog post. FYI: The ungrouped disks folder is for disks that have failed or been ungrouped on purpose. Ungrouping takes time as the EVA moves data from the drive to free it up to be removed or replaced.

Data Replication – it is here that you can create DR groups. This allows you to replicate (synchronous or asynchronously) between EVA arrays. A replication group is comprised of 1 or more vdisks. Sounding like a broken record, I will have a separate posting on replication.

Hardware – it is here that you can check out the status of the hardware. Both controllers are listed, as are all the disks. If there are hardware issues (per a bang light) then you can come here to find out why. The status of failed items is usually fairly straightforward and understandable as to what happened and what should be done.

About Mike Mills

Mail | More Posts (20)

So, as an overview for the environment, we currently have 3 8100 series and one 4400 series. EVAs arrays use two controllers, and our models are active/active. Our VMware environment at work is currently a VI3 schmorgesborg and we are planning an update (well migration) to vSphere. For SAN switches, we have older Mcdata’s that are workhorses! We manage them using EFCM software (also java, sigh).

HP provides Command View for EVA as the management tool for the EVAs. This installs and runs on Windows, and is now supported running in a VM as of the current release (9.2). Replication is handled with Business Copy and Continuous Access (separately licensed features). Automation of replication is handled using Replication Solutions Manager, which is a java-based App. :-/

For my next post, I’ll be giving a walkthrough of Command View and discuss how the EVAs we have are configured. Lots of screenshots too!

About Mike Mills

Mail | More Posts (20)

The configuration was a breeze just like the initial setup. The disks were placed evenly across the disk shelves and appeared into the console as expected. A couple of clicks later I had a vDisk setup and then created to volumes and assigned them to my hosts once I had created the Datastores from the first Host.

If you need to know more about how to setup these you can read my initial post on how to setup a HP MSA 2300 series array here.

About Brian

Brian is a Technical Architect for a VMware partner and owner of this website. He is active in the VMware community and is helps lead the Chicago VMUG group. This blog Virtualize Tips was started to document and remember things that I come across while working with tech.

Mail | Web | Twitter | LinkedIn | More Posts (170)

First thing you will need to do is to download the proper EMC grab version to support your hosts. At the time of writing this its a version 1.2.1 and is supports both ESX 3 and 4. Proceed on over to http://powerlink.emc.com and download it from the support programs area.

Once you have the file you will need to upload it to your host with something like WinSCP. I always upload it into the /tmp folder and unzip it there. It will create a folder called “emcgrab”. You can use the following command to do the unzip in case your not familiar with what to do.

tar -xvf emcgrab_ESX_vSphere_v.1.2.1.tar

Next thing is to move into the emcgrab directory that was created. From within there you will need to execute the following command. If you read the help file included in the directory it will explain some options to supress some annoying confirmation screens about the licensing and such.

./emcgrab.sh -nomsg

Once the program starts to run depending on the options you used it will prompt you to confirm and read the licensing. After you pass that part it will ask you a string of questions about your contact details and some questions about your environment. These are not necessary to complete it you are using these grabs in house. If you plan on sending these to EMC then I would advise to fill them out.

Once the script completes it will ask you if you want to run vm-support to collect the VMware support logs along with the EMCgrabs. This is up to you, if you have a need for them go ahead. Once the script finished it will place the zipped up file in the Output folder and you can pull off the file with WinSCP.

In closing it’s not necessary to be running Powerpath on your hosts to collect these grabs.

About Brian

Brian is a Technical Architect for a VMware partner and owner of this website. He is active in the VMware community and is helps lead the Chicago VMUG group. This blog Virtualize Tips was started to document and remember things that I come across while working with tech.

Mail | Web | Twitter | LinkedIn | More Posts (170)