Virtualization Tips » ESXi

The difference of the Virtual Machine power and reset buttons

Brian — Wed, 15 Sep 2010 04:29:49 +0000

This is something that I did not pay much attention to but it’s one of those wacky things that you will notice at 4 AM while working a Disaster Recovery Drill. Anyways it struck me weird that the power controls and reset buttons available in the VM console are different than the ones available by right click the VM in the inventory list. The ones from the VM console require VMware tools to be installed and the ones from the tree list do not.

I really came to this conclusion while working on a DR drill this week for a client and we are using vSphere 4.1 for the drill. I had never ran into this with previous versions of vCenter server so I’ll have to take another look and see if this is something new to 4.1.

About Brian

Brian is a Technical Architect for a VMware partner and owner of this website. He is active in the VMware community and is helps lead the Chicago VMUG group. This blog Virtualize Tips was started to document and remember things that I come across while working with tech.

Mail | Web | Twitter | LinkedIn | More Posts (169)

VMware vSphere and security scan false positives on OpenSSH vulnerabilities

Brian — Thu, 09 Sep 2010 00:40:04 +0000

This is something that we get on a regular basis from the security team. When doing their regular security scans for compliance and vulnerabilities I always get a long list of ESX hosts. The scans normally come back and complain about an OpenSSH x11 vulnerability or an OpenSSH Memory and Buffer Overflow.

These seem to be False positives from the tool being used to scan the hosts. We always make sure that we have installed the necessary updates related to OpenSSH as VMware releases them. But the tool always comes back with these issues. It seems to stem from the fact that the tool looks at OpenSSH in generic terms and assumes that all vendors implement it in the same way. From the documents listed below VMware indicates that since ESX 3.x VMware no longer included the x11 packages with their products. I would recommend that you make sure you are up to date on your patches and if the scans still come back dirty that you should discuss this results with the Application vendor that created the scanning tool. You might find out that this is common and they are just false positives.

Links:

VMware ESX Server and Security Issues in OpenSSH

Security Response: SSH Version Installed with ESX Server May Be Vulnerable

About Brian

Mail | Web | Twitter | LinkedIn | More Posts (169)

What is VMware vCloud Datacenter services

Brian — Wed, 01 Sep 2010 04:28:34 +0000

So your head is probably spinning this week with all of the news coming out of VMworld 2010. I know it’s only day 2 and each evening my head was pounding after absorbing so much new information. Today was day 2 and the information fountain was turned up all the way with the Keynote speak, there was several new products announced. In this post I am covering vCloud Datacenter service and what it has to offer to your Service Catalog.

To break this down in simple terms. I see this as the public version of vCloud that provides you the ability to link up your internal private cloud build on vCloud Director. This service allows you to provision app’s and VM’s out in the public space when you need to. Maybe its because you are out of space on your private cloud or that you just want something out there for other reasons. The vCloud Datacenter services gives you the interoperability that you want with ease of use and VMware is promising the security that Corporations are demanding. This will all be provided by vCloud Director and the new vShield product family.

Here some of what VMware has to say about vCloud Datacenter services.

Built to predefined specifications and based on secure VMware cloud infrastructure technology, vCloud Datacenter Services provide multilevel, auditable security through SAS 70 Type II or ISO 27001 compliance. vCloud Datacenter Services also provide best-in-class virtual firewall capabilities, Layer 2 isolation, role-based access control and the ability to integrate with Active Directory. Access to end user activity logs keeps you in control and allows you to calibrate user access levels for enhanced end user security.

Because vCloud Datacenter Services are built upon the same, globally consistent foundation as your internal datacenter or private cloud, VMware vCloud Director and VMware vSphere, internal virtualized applications can be easily moved to a vCloud Datacenter Services without re-architecting or refactoring. Rather than being locked into a proprietary cloud platform as you may be with other providers, you can choose the vCloud Service Provider that best meets your needs and manage, move and operate your applications as if they were on site.

Link to vCloud Datacenter services at VMware

Here are some of the differences between what vCloud Datacenter Service has to offer compared to public clouds.

	vCloud Datacenter Services	Other public clouds
COMPATIBILITY AND ADMINISTRATION
Use existing internal VMs or vApps in the cloud	Yes	No
Familiar VMware infrastructure	Yes	No
Authenticate users against enterprise directory	Yes	No
Multi-user, role-based access control	Yes	No – one user per account
Identical GUI for internal and external clouds	Yes	No
Move applications between virtual data centers	Yes	No
PERFORMANCE
Predictable performance from resource allocation (committed VDC and dedicated VDC)	Yes	No – depends on other tenants’ use
Storage performance	5x	1x
NETWORKING AND SECURITY
Firewall per vApp and per organization	Yes	No – per VM
Full virtual layer 2 networking	Yes	No – L3 only
Auditable security with all logs provided	Yes	No
Optional physical segregation of resources	Yes	No

About Brian

Mail | Web | Twitter | LinkedIn | More Posts (169)

VMware vCloud Director rises from the shadows of project Redwood

Brian — Tue, 31 Aug 2010 16:45:36 +0000

You can now hear the sigh of relief as many bound by NDA to keep silent about Project Redwood. This morning brings the news of the official announcement from VMware about vCloud Director or vCD. This is the new VMware Cloud Infrastructure solution that will allow Corporations and Service Providers to build clouds and ITaaS ( IT as a Service ) consumption models. Below is a quote from the VMware press release about vCloud Director.

VMware vCloud(TM) Director: A new model for delivering and consuming
    infrastructure services
     VMware vCloud Director changes the way IT
    delivers infrastructure services and the way users access and consume
    them. By extending the resource pooling capabilities of VMware
    vSphere, VMware vCloud Director enables IT to create "virtual data
    centers" (VDCs) -- logical pools of compute, network and storage
    resources with defined management policies, SLAs and pricing. IT
    organizations can offer these VDCs -- along with catalogs of other
    infrastructure and application services such as virtual appliances,
    VMs, and OS images -- to users through fully automated self-service
    access.

So what is vCloud Director

To put it in the simplest terms it’s a layer that sits on top of vCenter server and abstracts all the resources that vCenter has under it’s control. You then combine all of these resources into large pools for your Customers or Tenants to consume. Also vCloud Director provides the Customer a Self Service portal to use.

So what exactly are the resources that vCloud Director abstracts from vCenter server? Below is a list of the resources and the vSphere term to bring it all together.

Compute resources = vSphere Clusters and Resource Pools
Network resources = dvSwitches and portgroups
Storage = Datastores ( VMFS and NFS )

These resources are then presented to you via the Self Service Portal of vCD. As an administrator you can use the vCloud Director Portal to split up and assign resources to Customers, Department or some other business division. These call also be referred to as an Organization, this sounds a lot like Lab Manager. The vCloud Director product was designed to work with both Enterprise and Service Provider clouds. The resources are divided up and assigned to a Virtual Datacenter or vDC. There are two types of vDC’s available withing vCloud Director.

Provider Virtual Datacenter ( Provider vDC )
Organization Virtual Datacenter ( org vDC )

The Provider Virtual Datacenter is the base for compute resources. When creating a Provider Virtual Datacenter you will need to select a resource pool. Nex you will need to associate at least one datastore with the Provider vDC, this might be all LUNs masked to your cluster. Duncan from Yellow Bricks layed out the following theory

Some of my colleagues described the Provider vDC as the object where you specify the SLA and I guess that explains the concept a bit more. So for instance you could have a Gold Provider vDC with 15K FC disks and N+2 redundancy for HA while your Silver Provider vDC just offers N+1 redundancy and runs on SATA disk… everything is possible.

Now that a Provider vDC was created you can create an Org vDC and associate the Org vDC to a vCD Organization. Its possible that an Organization can have multiple Org vDCs associated to it. For example it’s possible to have 3 Org vDCs owned by a single Organization across two Provider vDCs. Those provider vDCs could each have a different SLA.

So in my thought vCloud Director does seem be Lab Manager on steroids which is a phrase that I’ve heard before. Many of these ideas do seem to be based off of the Organizations, configurations and networking that Lab Manager was using.

Pricing and Availability
VMware vCloud Director is currently available and is licensed per VM starting at $150 per VM.

Build Secure, Multi-Tenant Clouds – VMware vCloud Director lets administrators group users by policy, such as a business unit, division or subsidiary. Each group has isolated virtual resources, independent LDAP-authentication, specific policy controls and unique catalogs. To ensure security and compliance in a cloud environment where multiple organizations share infrastructure resources, VMware vCloud Director includes VMware vShield perimeter protection, port-level firewall, and NAT and DHCP services.

vCloud Director Links

Download link for vCloud Director

vCloud Director Install and Configuration guide

vCloud Director User Guide

vCloud Director Administrators guide

About Brian

Mail | Web | Twitter | LinkedIn | More Posts (169)

VMware best practices for virtual machine snapshots

Brian — Tue, 03 Aug 2010 15:32:54 +0000

I will start this post off with the standard snapshot warning. Just a reminder that Snapshots are not backups, they are only a change log of the original virtual disk. You should not count on them as a backup. There are a number of different reasons that you might use a snapshot for. One of my most used reasons would be for a software upgrade I would use the snapshot to allow for an easy rollback to the machine state prior to the upgrade. If you have some other reasons leave a comment to share with others.

The maximum supported amount in a chain is 32. However, VMware recommends that you use only 2-3 snapshots in a chain.
Use no single snapshot for more than 24-72 hours.
- This prevents snapshots from growing so large as to cause issues when deleting/committing them to the original virtual machine disks. Take the snapshot, make the changes to the virtual machine, and delete/commit the snapshot as soon as you have verified the proper working state of the virtual machine.
- Be especially diligent with snapshot use on high-transaction virtual machines such as email and database servers. These snapshots can very quickly grow in size, filling datastore space. Commit snapshots on these virtual machines as soon as you have verified the proper working state of the process you are testing.|
If using a third party product that takes advantage of snapshots (such as virtual machine backup software), regularly monitor systems configured for backups to ensure that no snapshots remain active for extensive periods of time.
- Snapshots should only be present for the duration of the backup process.
- Snapshots taken by third party software (called via API) may not show up in the vCenter Snapshot Manager. Routinely check for snapshots via the command-line.
An excessive number of snapshots in a chain or snapshots large in size may cause decreased virtual machine and host performance.

You can find some more details from VMware on troubleshooting snapshots here.

About Brian

Mail | Web | Twitter | LinkedIn | More Posts (169)

Some cool new features in vSphere Command Line Interface CLI 4.1

Brian — Thu, 29 Jul 2010 20:18:48 +0000

Now that vSphere 4.1 has been out for a couple of weeks you’ve probably had some time to play with it in a lab. I’m sure you have also spent some time reading the release notes getting up to speed on the large list of new features that were released. After spending time myself getting familiar with many of the new options I wanted to dig in and see what was new with the Command Line Interface in 4.1. Since this is going to play a big part in how you will be managing ESXi hosts once you move your environment over to the platform of the future.

I have grabbed a list of the new commands added to vCLI 4.1, these command will help narrow the gap that had existed between what you could run on the ESX console (COS) and what you could do via the vCLI with an ESXi host. Notice the part at the end where it lists some of the commands that cannot be executed against a vCenter server for a host in lock down mode.

vicfg-hostops – Allows you to examine, stop, and reboot hosts and to instruct hosts to enter and exit maintenance mode.
vicfg-authconfig – Allows you to add an ESX/ESXi host to an Active Directory domain, remove the host, and list Active Directory domain information.
vicfg-ipsec – Supports IPsec setup.

vSphere CLI 4.1 also includes the following new functionality:

The following options have been added to esxcli:
- esxcli swiscsi session – Manage iSCSI sessions.
- esxcli swiscsi nic – Manage iSCSI network interfaces.
- esxcli swiscsi vmknic – List VMkernel network interfaces available for binding to particular iSCSI adapter.
- esxcli swiscsi vmnic – List available uplink adapters for use with a specified iSCSI adapter.
- esxcli vaai device – Display information about devices claimed by the VMware VAAI (vStorage APIs for Array Integration) Filter Plugin.
- esxcli corestorage – List devices or plugins. Used in conjunction with hardware acceleration.
- esxcli network – List active connections or list active ARP table entries.
- esxcli vms – List and forcibly stop virtual machines that do not respond to normal stop operations.
Some of the parity issues between vSphere CLI and the ESX service console have been resolved.
You can now run vCLI commands using SSPI (--passthroughauth) against both vCenter Server and ESX/ESXi systems.
Lockdown mode allows vSphere administrators to block direct access to ESXi systems. With lockdown mode enabled, all operations must go through a vCenter Server system. The following commands cannot run against vCenter Server systems and can therefore not be used in lockdown mode:
- vicfg-snmp
- vifs
- vicfg-user
- vicfg-cfgbackup
- vihostupdate
- vmkfstools
- esxcli
- vicfg-ipsec

If you want to run these commands against an ESXi system, turn off lockdown mode using the vSphere Client.

About Brian

Mail | Web | Twitter | LinkedIn | More Posts (169)

Use web browser to view vSphere Configuration and Log files

Brian — Thu, 29 Jul 2010 18:57:52 +0000

Sure this nothing earth shattering but it’s just something simple that can make your life easier. With a web browser and some links that I will provide below you can view some of the vSphere configuration files and messages from logs. This is probably the fastest way to get a view into your host with out having to SSH into the server or use another method. This method works for both vSphere 4.0 and 4.1 hosts and it works on both ESX and ESXi hosts.

You can view the VMware vSphere Configuration files from a browser using a link formatted like the following. https://hostname/host From that link you will need to authenticate to your host and then will be able to view a list of files from the host. In the list of files presented with be configuration files and some logs.

There is another page viewable with a web browser that will show you log messages from your ESX or ESXi host. Use the following syntax for the link. https://hostname/host/messages

About Brian

Mail | Web | Twitter | LinkedIn | More Posts (169)

Chicago VMware VMUG recap and notes

Brian — Thu, 29 Jul 2010 03:10:19 +0000

I would like to say a Big Thanks to everyone that attended today and a special thanks to our sponsor Compellent. Compellent gave a nice presentation about their technology and what they have accomplished in their 5 years since they released the first product. I will give a short break down of the different presentations today and if we are able to get the slide decks from the presenters we will publish them over on the Chicago VMUG blog.

In our first presentation of the day Chris Fox of VMware was in and gave an overview of the new features of vSphere 4.1. There were discussions about SIOC (Storage IO Control), NIOC (Network IO Control), VAAI the API’s for array integration were some of the most talked about features. It was also discussed that VMware ESX 4.1 classic will be the last release of the ESX flavor of Hypervisor. Sometime in 2011 there is expected to be the next major release of vSphere and it will only be available in ESXi flavor.

In the second session of the day Russ Taddiken of Compellent talked to us about their storage virtualization technology. Russ gave a presentation that explained many of the features that make Compellent a strong competitor in the storage market. He spoke about Storage Auto Tiering that has been a feature in their product for about 5 years. Some of the other points that stood out to me was CoPilot their support organization and the Portable Volume feature. With portable volume it allows for the initial data replication to be placed on an encrypted USB disk that can be shipped to a remote site that might have a slow link. You will then only have to replicate the changes rather then the entire amount. Russ also mentioned that Compellent will be in the 2nd round of Vendors that will be supporting VMware VAAI API for storage functions.

In the last session of the day Mark from VMware spent time to talk about migrating your ESX infrastructure to ESXi. He covered the different ways to convert your hosts over to VMware ESXi. There was discussion around some of the reasons for the VMware’s decision to move in the ESXi direction. An estimated 80% of patches that VMware released for the ESX classic version were related to the console (COS) due to it’s Linux base that it was built on. With ESXi the COS was removed and the amount of patching required is greatly reduced. VMware is also working in the direction of building the ability to have a stateless hypervisor. Mark spent some time showing some of the commands that are the vCLI versions of the console commands that many are used to using.

We had a pretty nice showing for this meeting and hope that our community continues to grow. We had a couple of higher profile members from the VMware community show up to the meeting. David Davis from Train Signal was in attendance at the meeting. David has created a large number for training videos from Train Signal as well as for his blog VMwareVideos.com. Thanks again to David and the Train Signal team for providing several copies of their VMware vSphere training videos that we were able to give away to our members. Also in attendance today was Justin Lauer of EMC and a vSpecialist from Chad Sakac’s vArmy Team. I’ve knowing Justin for a bit now and it’s always great to chat with him, his involvement in our VMUG community will help many.

Update: We have posted a few of the slide decks from the presentations today here.

Took a couple of quick photos with an iPhone today as I forgot my camera but will do a better job in the future.

About Brian

Mail | Web | Twitter | LinkedIn | More Posts (169)

Different ways to enable Tech Support Mode TSM on ESXi 4.1

Brian — Thu, 15 Jul 2010 02:59:14 +0000

For anyone that has been running or played with ESXi in the previous versions you should have a good idea of what Tech Support Mode is. The Tech Support Mode or TSM is a sort of simple version of the system console that was available on the classic versions of ESX. Except that the TSM mode is not Linux based and does not have all the capabilities that the old COS had. But you can now access Tech Support Mode locally or via SSH if you follow the instructions below to enable them. I have become very comfortable with the old console access and that’s probably my biggest complaint about having to use ESXi. I’ve been playing around with the vMA or virtual management appliance that can be used to remotely manage ESXi hosts in the lab but its just not the same. I guess it will become second nature the more that I use it since classic ESX will no longer be offered in ESX 5.0 when it is released in the future.

To enable local or remote TSM from the Direct Console User Interface (DCUI):

At the DCUI of the ESXi host, press F2 and provide credentials when prompted.
Scroll to Troubleshooting Options, and press Enter.
If you want to enable local TSM, select Local Tech Support and press Enter once. This allows users to login on the virtual console of the ESXi host.
If you want to enable remote TSM, select Remote Tech Support (SSH) and press Enter once. This allows users to login via SSH on the virtual console of the ESXi host.
Optionally, if you want to configure the timeout for TSM:
1. Select Modify Tech Support timeout and press Enter.
2. Enter the desired timeout value in minutes and press Enter.
Press Esc three times to return to the main DCUI screen.

To enable local or remote TSM from the vSphere Client:

Select the host and click the Configuration tab.
Click Security profile > Properties.
Click Local Tech Support or Remote Tech Support (SSH) and click Options.
Choose the desired startup policy and click Start, then click OK.
Verify that the daemon selected in step 3 shows as running in the Services Properties window.

To configure the TSM timeout value using the vSphere Client:

Select the host and click the Configuration tab.
Click Advanced Settings.
Change the UserVars.TSMTimeOut field to the desired value in minutes.
Click OK.

To access the local TSM:

At the main DCUI screen, press ALT+F1 simultaneously. This opens a virtual console window to the host.
Provide credentials when prompted.
Note: When typing the password, characters are not displayed on the console.

To access the remote TSM:

Open an SSH client.
Specify the IP address or domain name of the ESX host.
Notes:
- Directions may vary depending on what SSH client you are using. For more information, consult vendor documentation and support.
- By default, SSH works on TCP port 22.
Provide credentials when prompted.

About Brian

Mail | Web | Twitter | LinkedIn | More Posts (169)

Location of VMware log files for ESX, ESXi, SRM and vCenter

Brian — Wed, 02 Jun 2010 14:20:11 +0000

Whether your tracking down an issue on your own or collection data to submit a support request to VMware your gonna need to round up the necessary log files. I have collected and listed some of the main log locations from VMware and also linked to KB articles covering a full list of log file locations.

VMware ESX log files

You can see ESX logs: (KB Link)

From the Service Console
From the vSphere Client connected directly to the ESX host (click Home > Administration > System Logs)
From the VMware Infrastructure Client connected directly to the ESX host (click Administration > System Logs)

The vmkernel logs (which log everything related to the kernel/core of the ESX) are located at /var/log/vmkernel.

The vmkwarning logs (which log warnings from the vmkernel) are located at /var/log/vmkwarning.

The vmksummary logs (which provide a summary of system activities such as uptime, downtime, reasons for downtime) are located at /var/log/vmksummary.

The hostd log (which is the log of the ESX management service of the ESX) are located at /var/log/vmware/hostd.log.

The messages log (which log activity on the Service Console operating system) is located at /var/log/messages.

The VirtualCenter Agent log is located at /var/log/vmware/vmware/vpx/vpxa.log.

The Automatic Availability Manager (AAM) logs are located at /var/log/vmware/aam/vmware_-xxx.log.

The SW iSCSI logs are located at /var/log/vmkiscsid.log.

The System boot log is located at /var/log/boot-logs/sysboot.log.

VMware ESXi log files – (KB Article)

The vmkernel, vmkwarning, and hostd logs are located at /var/log/messages.

The Host Management service (hostd = Host daemon) log is located at /var/log/vmware/hostd.log\.

The VirtualCenter Agent log is located at /var/log/vmware/vmware/vpx/vpxa.log.

The System boot log is located at /var/log/sysboot.log.

The Automatic Availability Manager (AAM) logs are located at /var/log/vmware/aam/vmware_-xxx.log.

vCenter log files – (KB Article)

SRM log files – (KB Article)

The SRM configuration files are located at:

C:\Program Files\VMware\VMware Site Recovery Manager\config\extension.xml
C:\Program Files\VMware\VMware Site Recovery Manager\config\vmware-dr.xmlOr
C:\Program Files\VMware\VMware vCenter Site Recovery Manager\config\extension.xml
C:\Program Files\VMware\VMware vCenter Site Recovery Manager\config\vmware-dr.xml

The SRM Logs (on vCenter Server for connection with SRM and on SRM for SRM workflow) are located at:

%ALLUSERSPROFILE%\VMware\VMware Site Recovery Manager\Logs, which translates by default to C:\Documents and Settings\All Users\Application Data\VMware\VMware Site Recovery Manager\LogsOr
%ALLUSERSPROFILE%\VMware\VMware vCenter Site Recovery Manager\Logs, which translates by default to C:\Documents and Settings\All Users\Application Data\VMware\VMware vCenter Site Recovery Manager\Logs

The SRM Installation Logs (on the SRM Server, which may not be the vCenter Server) are located at C:\Documents and Settings\Administrator\Local Settings\Temp\1.

The location of the SRA Logs (on the SRM server) depends on the SRA type and vendor. They may be located in:

C:\Program Files\VMware\VMware vCenter Site Recovery Manager\scripts\SAN\*\logOr
C:\Program Files\\

About Brian

Mail | Web | Twitter | LinkedIn | More Posts (169)