Writing PowerCLI for Performance

I was building a piece of PowerCLU code to check on my vCD Deployed vAPPs for certain errors and because it’s me I was curious what performance difference there was between using array based operations verses piped operations in powershell. Here’s what I found out.

Here is the array based operation – time to completion 2 minutes 30 seconds.

$myVAPP = @(Get-CIVApp | Select-Object Org, Name, Status)
        foreach ($myVAPP.Name in $myVAPP)
            {
                IF ($myVAPP.Status -eq "FailedCreation")
                    {
                        Write-Host 'vAPP' $myVAPP.Name' - Status is' $myVAPP.Status
                    }
                ELSE
                    {
                            IF ($myVAPP.Status -eq "Unresolved")
                            {
                                Write-Host 'vAPP' $myVAPP.Name' - Status is' $myVAPP.Status
                            }
                            else
                            {
                                IF ($myVAPP.Status -eq "Unknown")
                                {
                                    Write-Host 'vAPP' $myVAPP.Name' - Status is' $myVAPP.Status
                                }
                                else
                                {
                                }
                            }                           
                    }
            }
           

Here is the pipe based operation which is recommended for reasons that I now can testify for first hand.

Get-CIVApp -Status "Unresolved", "FailedCreation", "Unknown", "Unrecognized" | Select-Object Org, Name, Status

This operation returned the same results as the array based operation in 14 seconds, that’s 9% of the original time of using an array…

vFabric Hyperic Server Heartbeat

I wrote a quick safety feature for my Hyperic server that monitors the Hyperic Server process every minute and if the Server stops or dies the script will attempt to restart it. Nothing fancy but it’s a nice little feature… plus you can change a couple lines and make one to monitor your Hyperic Agents as well…

#!/usr/bin/perl
# ---------- Hyperic 5.0.0 Server Heartbeat Check ------------
# This script is used to verify that the Hyperic 5.0.0 Server is running and
#    restarts it upon failure. To schedule it to run automatically every
#    minute on linux run the following:
#         crontab -e */1 * * * * <Path to this script>

# NOT SUPPORTED OR PROVIDED BY VMWARE AND HAS NO GUARENTEES

$cmd = "/opt/hyperic/server-5.0.0-EE/bin/hq-server.sh status";
$out = `$cmd`;
# print "Output $cmd\n";
# print "Output Check 1 - $out\n";
if (index($out, "HQ Server is not running") != -1)
        {
                # print "\nHQ Server is dead\n";
# Restart the HQ Service
                $cmd = "/opt/hyperic/server-5.0.0-EE/bin/hq-server.sh start" ;
                $out = `$cmd`;
                # print "Restarting the Service\n $out \n ";
        }
        else
        {
                # print "\nServer is running\n";
        }

Getting Started with vFabric Hyperic Monitoring and Alerting

During the past couple days I have been working with Hyperic to setup basic alerting functionality for things like disk space thresholds, Windows Services and memory usage by process. Here is a quick getting started to setup basic disk monitoring as well as an intro to the Hyperic Escalation Schemes which allow tiered alerting.

First off let’s create an Escalation Scheme. To do this go to Administration > Escalation Schemes Configuration.

Next Let’s go ahead and build the process of who gets alerted and when.

Now that we have setup the notification process let’s actually setup some alerts…For this example I will be setting up test disk space alerts on a subset of my servers. To get here you need to understand a few things:

1. Under the Resources Tab is where you are going to find all your things that can be monitored. Here comes the confusing part:

a. Platforms is the Server that your Hyperic Agent is running on.

b. Servers are things like .net, Apache Tomcat and MSSQL Server.

c. Services are things that you monitor like HTTP, Disks, CPU, RAM and Windows Services.

d. Compatible Groups/Clusters are groups of the same thing (ie, Disks only)/

e. Mixed Groups are groups that contain a mix of things like Disks and RAM.

Once you grasp this it will make your experience with Hyperic much easier.

Ok, so I have navigated to the Services Tab and filtered by “FileServer Mount” aka Disk information. Now I want to select my subset of disks and click on the Group button.

Next I click on "Add to a New Group”  to create a new group. Because this group contains all like items it will create a new Compatible Group.

Now you should see your new group.

From there click on the Alerts tab and click on Configure

Now I want to create a new alert against this group of servers. My normal if statement is “IF more THAN 0 of the Resources” because I want alerts if any of them go down.

Here’s an important note: if you select Total Bytes Avail or several other metrics your alert will NOT work by default, here’s why:

Go to Administration > Monitoring Defaults and find FileServer Mount and click Edit Metric Template.

This should bring up a screen like the below. Notice that the Default On is set to No for the Total Bytes Avail… if you build your alert on that setting and don’t check to make sure that valid data is coming in you might be lulled into a false sense of security… Word to the wise, make sure your monitors are green after you create them.

Congrats, you are now ready to use vFabric Hyperic to start basic monitoring in your environment.

Configuring VMware vCSA with Active Directory

I ran into 2 issues while using the vCenter Server Appliance and integrating it with AD and what I found I figured might help some of you in the future. Here goes:

Issue 1: vCSA fails to join the AD domain.

On the initial deployment of my vCSA I got an error while attempting to join my AD Domain that says “Error: Enabling Active Directory failed”. That’s it… nothing useful…

When this happens you can run the process manually and if it fails you will at least get an error code. To do so run /usr/sbin/vpxd_servicecfg with the following syntax:

vpxd_servicecfg ad write <username> <password in 'single’ quotes> <domain name>

like: 

vpxd_servicecfg ad write caleb ‘myTopSecretPa$$’ caleb.com

If it is successful the return code will be “VC_CFG_RESULT=0”. If that is the case you need to reboot your vCSA and you are ready to roll.

Issue 2: Configuring vCSA to use AD for user authentication with LDAPS.

1: Login to the web client (NGC) on your vCSA (https://<vCSA>:9443/vsphere-client.)

2. Go to Administration > Sign-On and Discovery > Configuration and click on the little green plus to add an identity source. Now you should see a screen like the below:

For the most part it is all pretty simple, the gotcha is the primary server URL and the “Choose Certificate” button.  For additional useful information look at the vmware documentation at http://pubs.vmware.com/vsphere-51/index.jsp?topic=%2Fcom.vmware.vsphere.security.doc%2FGUID-B23B1360-8838-4FF2-B074-71643C4CB040.html.

Next I need to get a certificate so I can talk to the DC. This part is pretty easy once you think about it for a minute. From my Windows workstation I open up MMC and add a Certificates snap-in for the Local Computer account. Next expand you certificates and navigate to the “Trusted Root Certification Authorities” folder and find the certificate that was issued by your Domain’s Certificate Authority. That is the certificate that we will use.

Right click on the proper certificate and go to All Tasks > Export. Change the format to Base-64 encoded X.509 .cer and save to your local desktop. Next from the vCSA “Add Identity Source” window click on the “Choose Certificate” and point it at the cert you just exported. Congratulations your domain logins will now work, just don’t forget you still need to assign them permissions.

Agentless / Network Monitoring with Hyperic

So… I’ve installed Hyperic and am using it to monitor a bunch of stuff (Disk Space, RAM, CPU, Services) on boxes that I have installed the Hyperic Agent on but I really want to just do a simple ping against a bunch of network devices for network monitoring…. Looking at the UI it is not at all obvious how I would accomplish such a task. The answer is you have to run the job against an agent, and here’s how you do that:

1. Install a Hyperic Agent that is going to do the actual pings for you. In my case I just installed it on my Hyperic Server. NOTE: You must install the agent as root

2. Browse to that “Platform”, i.e. the sever you installed the agent on.

3. In the little “Tools Menu” dropdown select “New Platform Service”

4. In the next window give this monitor a name and select the Service Type of “InetAddress Ping”:

5. The next screen is going to display a little banner that says “The resource has not been configured”. Click on the “Configuration Properties” hyperlink.

6. On this last step provide the FQDN or IP of the device that you want your agent to ping.

There you go, you are now ready to monitor your network connectivity using Hyperic. One last piece of advice; you might want to change the test interval to a smaller number than the default. You can do that from Administration > Monitoring Defaults > InetAddress Ping (click on the Edit Metric Template on the right side).

Changing VMware ESXi 5.1 Syslog settings via PowerCLI

Here is a quick and easy method to change the syslog settings on multiple ESX hosts using PowerCLI. First connect to your VC in PowerCLI and then run the below to see your current remote syslog server on each of the ESXi Hosts:

get-vmhost | Get-VMHostAdvancedConfiguration -Name Syslog.global.logHost

After getting that information you can then push out new changes:

#Get Each Host Connected to the vC
foreach ($myHost in get-VMHost)
{
    #Display the ESXi Host that you are applying the changes to
    Write-Host '$myHost = ' $myHost

    #Set the Syslog LogHost
    Set-VMHostAdvancedConfiguration -Name Syslog.global.logHost -Value 'server.domain.com,server2.domain.com' -VMHost $myHost
  
    #Use Get-EsxCli to restart the syslog service
    $esxcli = Get-EsxCli -VMHost $myHost
    $esxcli.system.syslog.reload()

    #Open the firewall on the ESX Host to allow syslog traffic
    Get-VMHostFirewallException -Name "syslog" -VMHost $myHost | set-VMHostFirewallException -Enabled:$true
}

For more information on set-VMHostAdvancedConfiguration take a look at https://www.vmware.com/support/developer/windowstoolkit/wintk40u1/html/Set-VMHostAdvancedConfiguration.html

Updating AD Lookup Account on VMware vCVA

 

I needed to update the service account password that my vCVA uses to authenticate with AD and I discovered something that was confusing for a while, my Configuration options under “Sign-On and Discovery” was missing… you can hit the little down arrow to expand the options and “Configuration” is just not there…

The answer makes sense but is not immediately apparent. You need to logout of the vCVA and login as root@localos or another local account and then the option will appear. From here you can change your configuration freely.