Posted by mike on Mar 1, 2011 in HP, vCenter Server, VMware, vSphere | 1 comment
Here at 64k, in our smaller cube near the vending machines, we storage-oriented folks like to mull over ideas big and small, 4k at a time. We also deal in a great number of puns, so consider yourself warned. Today, in our maiden voyage, I’d like to talk about some of my experience with HP’s line of EVA storage arrays. As many of our readers know, the EVA line is a middle tier offering from HP. Though likely to be usurped in the near future by 3PAR’s goodies, I am not here to begin that debate. Rather, let us delve into a few common gotcha’s that can be overlooked in environments where EVAs live.
ONE]
The tight rope act begins with the storage array, our bright and shiny EVA. At a fundamental level, an EVA is comprised of two controllers. The operating environment of the EVA is such that it can, in a semi-intelligent fashion, manage vdisk ownership between the two controllers itself. By default, vdisks are set to no preference for a failover/mode setting at the time of creation. This means the EVA will decide which controllers get which vdisks when it (the EVA itself) boots. Every vdisk is assigned to a controller (and only one controller). If the non-owning controller is receiving the IO for a server(s) talking to a vdisk, it will after a period of time change the ownership of the vdisk. This will reduce the load crossing the mirror ports. While the EVA can run in this fashion, it is sub-optimal.
The other side of the tight rope of this balancing act is the hosts. IO can walk many paths from host to array, some optimal and others not. The start of such begins at the host’s adapter. If it is a dual port (or multiple single port) host, then you have all the more paths to choose from. Even in the case of a single port host, you can still cover multiple paths to arrive at the vdisk. The handling of the proper path comes in the form of multipathing software. From HP for Microsoft operating systems, we have Device Specific Module (DSM), which uses MS’s MPIO stack as its basis. HP makes specific DSM’s for each of its line of arrays. Without the MPIO stack, the host will see a drive presented once for each host port. In an 8×00 series array, that is 8! So clearly the MPIO software and HP’s DSM is needed for correct operation. The default install does not enable Adaptive Load Balance (ALB). This hampers read operations by not passing through the correct controller for a vdisk. Note that non-MS based operating systems (like VMware) have their own multipathing stacks. In the case of VMware ESX(i) 3.x, the options are fixed and mru. In the case of vSphere, we get round robin added to the mix. In pre-vSphere environments, the fixed path does not by default balance load across the host ports. You can end up with all your VM traffic running over one host port! Yikes!
TWO]
Now, to balance things out, let me start with the array. A good habit to get into involves understanding your environment from an IO perspective. You need to understand the profile, or workload, of your IO, so that you can balance between the controllers (among other things!). Make sure to capture your performance data using evaperf (or other tools) to allow you the view of your controller’s current load. As you add new vdisks, you can balance them by setting the failover/mode setting to the controller with failover + failback. This will allow the balancing to remain should you lose and regain a controller. Further, this specifies the controller for the vdisk in terms of mastership. This helps from the host side as the controller it needs to talk through is clearly defined. One thing to keep in mind also is the need to accept all load on one controller should failure occur. This should be something you are aware of via your performance data. A good rule of thumb is a controller should be no more than 30% ideally (at least in my experience). And as always, have the latest Command View and XCS code. One other thing to check for balance is to make sure the host ports are set to their top speed (4GB, except the very old EVA models) as well as properly balanced on the fabric (equal ports on both sides). One customer I came across had all ports from controller A on fabric A and all ports of controller B on fabric B! Definitely a big problem there!
For the host side, there is a bit more that can be done. There is some work to be done on the array as well, which I will address. The hosts should have the latest firmware, drivers, and software for their HBAs. Additionally, make sure you have the latest HP DSM software. Within the DSM software, you will want to enable Automatic Load Balancing. As I stated before, this is not enabled by default. To enable, just right click on each LUN (listed by WWN) that is listed and choose Enable ALB.
So, as a quick explanation: write requests from hosts will hit the controller that owns the vdisk in question, but that write will propagate over the mirror link into both controllers’ cache. This is in case a controller is lost, the write can still be committed. Read requests will hit whichever controller, and if it is the wrong controller, will have to travel over the mirror ports to the correct controller. This is sub-optimal, but is alleviated by enabling ALB. ALB communicates with the array and will always communicate its read requests through the owning controller. Very handy!
Now, from a VMware standpoint, let’s talk about fixed and then round robin (two most common multipathing situations found today). For Fixed, you will need to balance IO to your datastores over the host ports of the controllers. Also keep in mind which controller you selected at the array. As an example, if I have 8 datastores of average IO (no virtualized heavy apps) then I would want 4 datastores on each controller. To further balance, I would have each datastore talking over one of the host ports for each of the controllers (4 ports per controller x 2 controllers). The IO is evenly balanced. To set this, simply go into each datastore properties (via the VI Client) and pick the WWN for the corresponding host port). Under heavy IO circumstances, you may not be able to move your traffic to a different host port. Just try again at a later date. When it comes to round robin, the IO works a bit differently. Round Robin will send IO to each host port in turn after a certain amount of IOPS. In the HP best practices for vSphere on the EVA, it states to change this value to 1 (and thus pushing even IOPS over every host port visible). There was a bug which would, after a reboot of the ESX(i) host, reset this to a very high number. I have found in my experience that leaving it as-is seems to work fairly well. I would guess there is good reason that HP came up with that figure, and so at this point, with vSphere 4.1, I would suspect you could set this without issue.
Summary
Presented here are some of the findings I have come across in working with different customers. I figure that having these kinds of storage discussions can help to make for a very engaging conversation. Let me know what you think (and if I make any errors, which being human, am prone to!
read more
Posted by Brian on Feb 27, 2011 in Labs, VMware | 1 comment
These days it’s almost a necessity that if you want to continue to increase your technical skills you will need to build a home lab. With a lab at home you can learn new technology that you might not get the chance to work with at the office. In the safety of your Lab you can break things and fix them without any worries.
I was able to sit down and talk with Kendrick Coleman this month about his newly created home lab. I thought that Kendrick would be a great first victim for this column, since he just built his lab. Some of the choices that he picked for his equipment intrigued me and I wanted to learn more. In case you don’t know Kendrick he is a vArchitect for VCE and works with Service Providers in his role evangelizing the benefits of the VCE stack. I hope to find other interesting home lab stories for upcoming issues.
Why did you create your home lab?
To keep myself up to date with all things related to vSphere. Since joining VCE as a vArchitect my role is more of a Pre-Sales function. I don’t have that daily access to servers in a datacenter to test out features.
How long did it take you to assemble?
I really took about a month to do research on the parts that I purchased for the lab. I spoke with people online and read a lot of blogs that others wrote on their labs. I needed to decide if I was going to go with one really large server or two servers with shared storage model.
What do you want to test in the lab?
I’ve been working with the Uber Celerra VSA that Nick created and the Cisco UCS emulator. I will also be checking out the Uber UIM VM that Nick setup. A lot of my time at VCE will be centered around architecting VDI and vCloud Director solutions. So I will be working with these products to test out different solutions that will help me in my designs.
What type of equipment is on your wish list for future lab upgrades?
The one thing that I would really like to upgrade in the future would be to get another NAS device and add SSD’s into it so that I could test the difference in performance.
How often have you been using the lab?
I’ve been using it a ton lately but I also just finished building it. So my wife has already been complaining about my time spent with the lab.
Do you keep it running all of the time?
Yes it’s just a few feet from me as we are speaking and I can barely hear it. That was my goal by looking for products that are green. By selecting these parts I was able to keep the power consumption low and also build a lab that is very quiet. I built the shuttle systems without hard drives or optical drives this kept the cost down and I can barely hear them running.
Will you be running anything other than VMware products in the lab?
I only plan on running things that related to what I’m doing as a vArchitect. So things like a Nexus 1000V virtual switch, VMware products and EMC / Cisco offerings that tie in with VCE. I don’t think that I’ll be running anything else. I don’t plan on becoming an SQL DBA or a Microsoft guru or anything like that.
What is the best thing that you have learned so far?
The best thing so far was to take your time and make sure that you do your research on the products that you buy. You will want to make sure that the motherboard and parts will work with vSphere. I highly recommend that you choose a network card that is on the VMware HCL, while I was able to do some driver injection to get the onboard Nic to work I still have some issues with it.
You choose to run ESXi on both servers why?
I went with ESXi on both because it allowed me to go diskless on both servers and boot from USB sticks.
What method are you using to connect to the shared storage?
Right now I am using both iSCSI and NFS. This allows me to play with both methods and I recently setup CHAP authentication for the iSCSI. That’s something that I never did in the past. During this I encountered an error with the iSCSI and was able to take the time and correct it rather than just starting over.
Kendrick also wrote up an extensive post on his blog that talks about some of the technical setup steps that he went through. It’s a great read and I suggest you give it a look also at this link.
Brian is a Technical Architect for a VMware partner and owner of this website. He is active in the VMware community and is helps lead the Chicago VMUG group. This blog Virtualize Tips was started to document and remember things that I come across while working with tech.
Mail | Web | Twitter | LinkedIn | More Posts (170)
Posted by Brian on Feb 24, 2011 in Featured, Magazine, VMware | 2 comments
This is a project that I’ve been working on for a couple of months now and finally was able to bring it together. My dream for this is to produce something monthly if possible. I like the magazine format because it offers something different than just reading RSS feeds and blog posts. For the people that travel or use iPads or other tablets I hope this gives you something else of interest to read.
Initially the focus of the magazine will be heavily focused around VMware technology but as time passes I hope to be able to incorporate other Virtualization topics into the mix. The content will include interviews, how-to articles, reviews and community based topics.
This edition has articles that cover HP BladeSystem Matrix, Designing with VAAI, new VMware certifications, storage and networking articles and Kendrick Colemans home lab and several other topics.
You can download the Magazine in PDF Here or Here.
Brian is a Technical Architect for a VMware partner and owner of this website. He is active in the VMware community and is helps lead the Chicago VMUG group. This blog Virtualize Tips was started to document and remember things that I come across while working with tech.
Mail | Web | Twitter | LinkedIn | More Posts (170)
Posted by Brian on Feb 7, 2011 in VDI, View, VMware | 0 comments
With each year that passes Virtual Desktops or VDI continues to become a priority to more companies. For the last several years different consultants and media people are quick to declare this as the year of VDI. This has kind of become a joke among the admin community. There is no doubt that VDI is here and companies are deploying it.
Since the release of View 4.0 VMware seems to be showing that they are very serious about achieving the same level of dominance in the VDI market that they share in server virtualization. VMware quickly followed up with View 4.5 that brought many needed features to level the feature set with Citrix’s offerings.
To continue proof that they are committed to this Desktop race VMware has announce a Certification track for the Desktop. The new program is called the VMware Certified – Desktop program and is made up of three certification levels. This new approach adds a new lower level certification that was not available with vSphere certifications.
VMware Desktop Certifications
- VMware Certified Associate 4 – Desktop (VCA4-DT): is directed toward Virtualization System Administrators who seek to demonstrate their ability to manage, monitor and troubleshoot desktop deployments and VMware View 4.5 components, as well as their knowledge of adjacent, complementary technologies to VMware solutions.
Learn more.
- Coming soon! VMware Certified Professional 4 – Desktop (VCP4-DT) is directed toward Virtualization Systems Engineers who seek to demonstrate their ability to install and configure the VMware View environment and enable View Client users. Candidates must also be able to configure the vSphere environment, possess a deep understanding of VMware core components and their relation to storage and networking and be proficient in datacenter design methodologies.
- Coming soon! VMware Certified Advanced Professional – Desktop (VCAP-DT): is directed toward Virtualization Architects who seek to demonstrate their deep knowledge of VMware vSphere and VMware View design. Candidates will be able to expertly manage all aspects of the vSphere and View environments from installation and configuration to maintenance and troubleshooting. Candidates will also possess a deep knowledge of virtualization architectures and components and be able to architect VMware desktop solutions to meet specific customer business requirements.
Also as a first there will be no VMware course requirement to earn the VCA4-DT certification. Typically in the past the first level certifications have always required a VMware course. With more advanced levels leaving VMware courses as optional, figuring that most people able to acquire them would already possess the necessary skills. You can see from the image that VMware is recommending a list of courses that would help candidates achieve the knowledge needed to take the first level exam.
I have to say that this has been a long time coming and I’m pretty glad to see that they have finally arrived. With the demand for VDI growing companies are looking for ways to see employee’s prove they have the skills necessary to successfully implement VDI. Also vendors are sure to welcome these certifications as a way to bolster their VDI sales pitches and prove they are worthy of your business.
Brian is a Technical Architect for a VMware partner and owner of this website. He is active in the VMware community and is helps lead the Chicago VMUG group. This blog Virtualize Tips was started to document and remember things that I come across while working with tech.
Mail | Web | Twitter | LinkedIn | More Posts (170)
Posted by Brian on Feb 7, 2011 in VDI, View, VMware | 0 comments
I’ve started to hear whispers this week that VMware will be updating VMware View to version 4.6. The release of View 4.6 will be a minor update. If you have an active VMware support contract you can expect to be able to upgrade to VMware View 4.6 free of charge. The download is expect to be in the last couple of days of February.
Some of the expected new features are:
- 160+ Bug fixes
- Support for secure PCoIP tunneling
- Improvements in using Windows 7 SP1 RC as a remote desktop OS
- Better keyboard mapping support
- Enhanced USB device compatibility
Brian is a Technical Architect for a VMware partner and owner of this website. He is active in the VMware community and is helps lead the Chicago VMUG group. This blog Virtualize Tips was started to document and remember things that I come across while working with tech.
Mail | Web | Twitter | LinkedIn | More Posts (170)
Posted by Brian on Feb 4, 2011 in VMware, VMworld | 0 comments
The word is out and it looks like VMworld Europe 2011 is heading back to Copenhagen. I think that most that attended last year would agree that the show was great and had decent growth in size from the previous years. Sure most would probably like to have it in a more traveled place like London, but Copenhagen seems to be home for the European show for now. VMworld
Brian is a Technical Architect for a VMware partner and owner of this website. He is active in the VMware community and is helps lead the Chicago VMUG group. This blog Virtualize Tips was started to document and remember things that I come across while working with tech.
Mail | Web | Twitter | LinkedIn | More Posts (170)
