VMware vCenter Log Insight Series Part4 - Creating Analytics

After a bit since last post here's the new article in which we will create some basic analytics from our previously gathered logs.

Interactive Analytics tab in Log Insight is the place you want to go in order to retrieve event information, aggregate logs and present them in a fancy dashboard.
By default Log Insight displays latest logs collected from any connected source ordered in order of occurrence.

In the search box you can insert any string you want in order to search for matches across all logs while in the drop down box near the search button you can also choose the time frame in which to retrieve results, such as latest 5 minutes of data, latest hour, latest 6 hours, etc. The cool aspect of searches is that autocomplete will give you word suggestion and matched words in log entries will be highlighted in order to allow you to fast browse across results.

Filters are probably the most effective way to provide a satisfying search among the (probably) huge amount of collected logs. Filters can enourmously help you in finding the log entry you are searching for. Filters can be added by clicking the green plus icon Add Filter and they can match any generic text inside the log entry or some pre-determined field. The classical logical operators: contains, does not contain, starts with, match regex, etc. can also be used.

Fields used in filters can be easily created, and saved for later uses, by selecting a specific detail presented in a log entry. Select Extract field in order to extract that particular information from the log.

Matched results will be highlighted in green. In the right menu you can also modify the regular expression if needed and choose a proper name to save your extracted field. In this example I used device name as field name.

Extracted fields can now be used in queries.

Let's now produce some dashboard with our retrieved logs. Every log search will automatically produce a bar chart in the upper part of the page. By default Log Insight displays the count of events over time so the displayed bar indicates in a certain time like day, hour, or minute, the number of occurrences of that particular event.

Charts can also be customized, you can retrieve a certain metric grouped by a particular entity. In this example I asked for the average number of scsi latency events grouped by the filter I previously created (device name). With this chart I have a graphical view on the average count of scsi device related events and these are displayed grouped by the disk device name.

Charts aspect can also be modified to fit your style. Depending on the data you are extracting you can choose to represent data as a column like chart, a line chart, area, bar, pie or bubble based.

Once you are satisfied with the obtained result you can save your chart in a dashboard, which basically is a page on which you can have multiple charts grouped together. Before adding a chart to a dashboard you can choose chart title and optionally insert a brief description of what the cart represents.

Your dashboards are accessible by clicking Dashboards button on the top of the pages and by navigating in the left-sided menu.

This could be an example of a populated dashboard, a single pane of glass on which sysadmins can have informations at a rapid glance about how well things are going.

Other articles in this series:
VMware vCenter Log Insight Series Part1 - Introduction
VMware vCenter Log Insight Series Part2 - Installation and Configuration
VMware vCenter Log Insight Series Part3 - Collecting Logs
VMware vCenter Log Insight Series Part4 - Creating Analytics

VMware vCenter Log Insight Series Part3 - Collecting Logs

The most important step in analyzing logs is...collecting logs. In this post I will briefly guide you in how you can collect logs in vCenter Log Insight.
As previously said, Log Insight features a syslog collector, this means that it can ingests logs from literally anything: ESXi hosts, physical infrastructure, virtual machine guest O.S., application logs, etc.

While configuring ESXi hosts and vCenter log collection is an automated task that can be easily performed from within Log Insight itself as seen in previous post, other devices must be manually configured in order to send logs over to Log Insight.

Here's how to collect logs from other sources rather than ESXi hosts and vCenter Server:

1)Windows operating system:

vCenter Log Insight provides a native agent to be used on Windows to collect logs. Windows agent can be retrieved from Log Insight Administration -> Agents page. Agent is a small executable that can be installed on any Windows machine you want to retrieve logs from.

After agent installation has succeeded logs are immediately sent to Log Insight to be collected. Please check Administration -> Agents page to verify that your Windows server has been successfully registered.

Agent is installed by default in C:\ProgramData\VMware\Log Insight Agent. liagent.conf is the configuration file in which you can configure several parameters related on logs collecting such as what kind of logs to collect and what event severity to collect.
Windows agent can also be used to collect specific applications logs from software, residing on the same machine on which agent has been installed, such as Microsoft SQL logs, Apache logs, etc.
This can be done specifying in liagent.conf file the path where to retrieve application specific logs. For further informations regarding app logs collecting have a look at this Log Insight documentation page.

Agent configuration can also be performed in a centralized way via Log Insight Administration -> Agents page. These settings are server-side and will be applied to every client connected to Log Insight, these settings are more authoritative compared to the ones set on client-side, this means that if same setting exists both on client-side and on server-side, the latter is the one that is valid. Otherwise client and server-side settings are merged to create a resulting configuration.

2)Linux operating system:

We will use Rsyslog to send logs from Linux over to Log Insight. Rsyslog can also be used to create a centralized logging server when deploying Log Insight in an hub-spoke architecture.
Rsyslog installation and configuration is quite simple. On RHEL 6 and later is installed by default, on previous versions and on RHEL flavoured distros you can install it using:

 yum install rsyslog  

On Debian and similar distros, such as Ubuntu, installation can be performed with:

 apt-get install rsyslog  

Once installed configure it by editing /etc/rsyslog.conf file. I prefer to use UDP for log transferring even if TCP is also a viable protocol. To enable UDP modules on Rsyslog edit rsyslog.conf by removing comments ("#" at line start) on:

 $ModLoad imudp  
 $UDPServerRun 514  

Add this line at the end of file in order to send logs over to Log Insight:

 *.*  @Log_Insight_IP_Address_or_FQDN:514  

Save file and restart Rsyslog service. If needed add an exception to Linux firewall in order to open port 514 UDP.

 iptables -A INPUT -m state --state NEW -m udp -p udp --dport 514 -j ACCEPT  

Verify that Rsyslog will be automatically started on boot. On RHEL-like distros you can do this with:

 chkconfig rsyslog on  

3)Physical Devices:

Physical devices such as switches, routers, storage nodes and servers typically support sending logs to a remote destination. This of course vary depending on device model and manufacturer and the appropriate documentation must be consulted in order to find out how to offload logs to an external syslog.
This process is usually quite simple and can be done either via a graphical management console either via CLI.

As a general example here's how to configure an HP 29xx physical switch to send logs over to Log Insight.

Connect to switch management interface either via serial cable either via SSH. Enable offsite logging with the following command:

 logging <Log_Insight_IP_Address_or_FQDN>  

That's all. In next post we will get some useful informations by analyzing logs.

Other articles in this series:
VMware vCenter Log Insight Series Part1 - Introduction
VMware vCenter Log Insight Series Part2 - Installation and Configuration
VMware vCenter Log Insight Series Part3 - Collecting Logs
VMware vCenter Log Insight Series Part4 - Creating Analytics

VMware vCenter Log Insight Series Part2 - Installation and Configuration

In this second post regarding VMware vCenter Log Insight I will explain how to deploy it and perform initial setup and configuration.
As first thing bookmark the VMware vCenter Log Insight documentation.

Log Insight installation is a quite straightforward process. Download the OVA file from VMware.com and deploy it in the classic way.
During deployment you will be prompted to choose your deployment size. As discussed in previous article deployment size is related to how much data Log Insight is capable to ingest in a certain amount time. This obviously directly influences the requested CPU number and RAM size to support this certain kind of workload. Please note that the Extra Small configuration option is not supported in production but is only suited for demonstrations only.

Configure IP address, proper subnet mask, gateway, DNS and appliance root password.

Before continuing let's have a brief look at how Log Insight stores data. Once OVF has been deployde if you edit Log Insight virtual machine you will see something similar to this:

Three hard disks are used to store operating system data, application logs and, finally, logs and indexes received by Log Insight. By default, if Small configuration is used, application logs and data are stored in an LVM formatted 120GB disk.

/storage/core is where logs and indexes are stored. This volume can be increased by simply editing the virtual appliance hard disk size. When Log Insight is powered back on LVM will be automatically expanded and will reflect the new size.

Once Log Insight has booted completely point to it using a web browser and you can perform the initial configuration.

Here you can choose either to start a new deployment either to join an existing one. The latter option is used when adding worker nodes to an existing Log Insight cluster and we will see this in detail in another post so at this time I choose the former one.

Insert admin credentials.

Insert license key. You can skip this step at this time and insert the license later but a valid license key must inserted in order to access to collected logs and analytics.

Time configuration is an optional step but I suggest you to configure it to work properly since timestamps in log analysis can be a lifesaver.

SMTP is used by Log Insight to send alerts via email

When setup is complete you will be redirected to the main page of VMware vCenter Log Insight.

In order to get log data from vCenter Server and ESXi hosts connected to it you need to go to Administration -> vSphere and properly fill in the fields pointing to you vCenter Server then check both options in this page. This will automatically retrieve logs from vCenter and configure ESXi hosts to send logs to this instance of Log Insight.

NOTE: If you need to check if an ESXi host sends logs over to Log Insight just connect to it via SSH and run:

 esxcli system syslog config get  

To perform a manual configuration for sending logs from an ESXi host to Log Insight or to a generic syslog run the following command:

 esxcli system syslog config set --loghost='udp://<IP_OF_LOGINSIGHT:514>'  

That's all for now, in the next article we will start using vCenter Log Insight for analyzing logs.


Other articles in this series:
VMware vCenter Log Insight Series Part1 - Introduction
VMware vCenter Log Insight Series Part2 - Installation and Configuration
VMware vCenter Log Insight Series Part3 - Collecting Logs 
VMware vCenter Log Insight Series Part4 - Creating Analytics

VMware vCenter Log Insight Series Part1 - Introduction

It has been a while since my last post, so I decided to write some new content regarding my latest work. Recently I've been intensively working with VMware vCenter Log Insight 2.0 and I've to admit that this is a really valuable tool, so I decided to start a small post series regarding Log Insight.

In this first post I'll give you a brief introduction about this great piece of software.

Log Insight is an analytics tool that acts as a centralized logs server and is deployed in a VMware environment as a virtual appliance. The cool aspect of Log Insight is that it supports the collection of logs either from VMware infrastructure (i.e. ESXi hosts) either from physical infrastructure (i.e. physical servers, physical switches, etc.) either from application (i.e. virtual/physical machines guest operating systems).

Log Insight provides an easy to use interface to analyze, search, and aggregate informations from logs. It supports real-time complex queries against massive log datasets all accessible by using a web browser.

Log Insight use cases are:

-Identify issues during troubleshooting.
-Gain visibility across all infrastructure, either physical, either virtual, from a single log collection point.

Log Insight can be deployed following two reference architectures:

1)Centralized Log Management: where every log is sent directly over to Log Insight. This is best suited to be used in small to medium size environments.

2)Hub-Spoke: this is usually adopted in larger environments, even geographically dispersed, where all logs are first sent over to a syslog collector and only later are offloaded from the syslog collector over to VMware Log Insight.

Log Insight itself is designed to be scaled up and out. Scale-up capability is provided by supporting the size increase of the virtual appliance partition used for storing logs. Scale-out is achieved by native cluster support. Log Insight supports up to six nodes clustered to provide greater throughput and greater availability. A master node exists within the cluster and up to five slaves, referred as worker nodes, can exist.

During the installation of Log Insight you will be prompted to select the appliance size. According to the size of the infrastructure you are up to monitor you can either choose from Small to Large sizing deployment.

Small deployment requires 4 CPUs and 8GB of RAM and it supports the processing of up to 3GB of log files per day (with an average of 200 logs entries per second). Large deployment supports up to 113GB of log files per day with an average of 7500 log entries per second. It also requires 16CPUs and 32GB of RAM.

That's all for this first post, in the coming up articles I will guide you through installation and initial setup, configurations and analytics creation.

Other articles in this series:
VMware vCenter Log Insight Series Part1 - Introduction
VMware vCenter Log Insight Series Part2 - Installation and Configuration 
VMware vCenter Log Insight Series Part3 - Collecting Logs  
VMware vCenter Log Insight Series Part4 - Creating Analytics

VMware: alarm notifications using PowerCLI and IFTTT

IFTTT is the short name for If-This-Than-That, a website and an Android/iOS app, that automates task based on the triggering of a certain condition. The idea behind that is really simple yet really powerful: if a <condition> is verified then <do something>.
In this post we are gonna use PowerCLI and IFTTT to provide push notifications in case of an alarm or a warning is triggered in vSphere.

What we are up to do is basically:

-Use a PowerCLI script, intended to run as a scheduled task, to check if alarms or warnings are triggered in our vSphere environment.
-Generate an RSS Atom feed with all alarms/warnings (if any). The feed file must be reachable by the internet via an IP address/DNS name.
-Create a new recipe in IFTTT to check for our RSS feed updates.

If a new alarm or warning is triggered in vSphere the RSS feed will be updated, IFTTT will notify the change and will provide a push notification to the user informing that something that needs to be addressed is happening in vSphere.

Let's start with the PowerCLI script: it checks for any warning/alarm and generates as output an RSS Atom feed which will be monitored by IFTTT in search of updates. The idea is to add this script to a scheduled task which runs automatically and possibly frequently. The $OutputFile variable is where the RSS feed is generated. IFTTT would require an internet accessible feed so in my case I point $OutputFile to the root of one website running on IIS on the same machine that runs the script. Of course the website is publicly accessible.

This is the code, as usual you can also find it in my GitHub Page. IFTTT_Alarms.ps1



The output RSS feed is generated by concatenating three parts: the header which has the xml declaration and the opening tags, the body, which is the main part that is dynamically generated based on any alarm/warning triggered in vSphere, and the footer which closes the feed.

If no alarms or warnings are (hopefully) triggered in your vSphere environment the resulting feed will be like this:

 <?xml version="1.0" encoding="utf-8"?>  
 <feed xmlns="http://www.w3.org/2005/Atom">  
   <title>Triggered Alarms Feed</title>  
   <subtitle>This feed contains all triggered alarms and warnings</subtitle>  
   <link href="http://hostilecoding.blogspot.com" rel="self" />  
   <link href="http://hostilecoding.blogspot.com/" />  
   <updated>2014-06-13T22:38:45.0020183+02:00</updated>  
 </feed>  

If there is any alarm the resulting feed will be similar to this:

 <?xml version="1.0" encoding="utf-8"?>  
 <feed xmlns="http://www.w3.org/2005/Atom">  
      <title>Triggered Alarms Feed</title>  
      <subtitle>This feed contains all triggered alarms and warnings</subtitle>  
      <link href="http://hostilecoding.blogspot.com" rel="self" />  
      <link href="http://hostilecoding.blogspot.com/" />  
      <updated>2014-06-13T22:38:45.0020183+02:00</updated>  
 <entry>  
      <title>Alarm: 192.168.243.144</title>  
                <link href="http://hostilecoding.blogspot.com/" />  
                <link rel="alternate" type="text/html" href="http://hostilecoding.blogspot.com/"/>  
                <link rel="edit" href="http://hostilecoding.blogspot.com/"/>  
      <updated>2014-06-13T22:38:45.0020183+02:00</updated>  
      <summary>Alarm Triggered.</summary>  
                <content type="xhtml">  
                  <div xmlns="http://www.w3.org/1999/xhtml">  
                      <p>Network uplink redundancy lost</p>  
                  </div>  
                </content>  
                <author>  
                      <name>Administrator@vsphere.local</name>  
             </author>  
 </entry>  
 </feed>  

The <entry> part contains the informations about the alarm, which host the alarm refers to, description, and some other infos. Links are present because IFTTT was a bit sensitive to the feed format, if no links are in the feed IFTTT would not recognize it as having a correct syntax and will report an error during recipe creation.

Let's now jump to IFTTT recipe creation, I used Android version but I guess  the iOS counterpart would be identical.

Create a new recipe:

Look for RSS feed -> Neew Feed Item

Insert the URL of the RSS feed. I used feeburner.com to map the public IP address of my IIS server to a URL. I guess you could simply use your IP address.

Android Notifications -> Send Notification

This is the look of the complete recipe:

So, when an alarm/warning will be detected the PowerCLI script will update the feed, IFTTT will notice the new feed entry and will generate a push notification similar to this:

That's all!!

VMware: Automate support bundle collection using PowerCLI

When troubleshooting ESXi issues, especially when dealing with VMware technical support, you (almost) always will be asked to provide support bundles for ESXi host(s) and/or the managing vCenter Server.

Support Bundles can be easily generated by accessing vSphere Client -> File -> Export -> Export System Logs although the process is manual and potentially time consuming.

As almost everything in VMware environment even the support bundle collection process can be easily automated using PowerCLI.
The Get-Log PowerCLI cmdlet is used to retrieve support bundle either from vCenter Server either from ESXi hosts.

When connected to a vCenter Server running:

 Get-Log -Bundle -DestinationPath C:\Users\Paolo\Desktop  

will download in specified location support bundle for vCenter Server.

Support bundle will be named: vcsupport-<Generated_ID>.zip

To download support bundles for specific ESXi host(s) only:

 Get-Log -VMHost (Get-VMHost -Name 192.168.243.144) -Bundle -DestinationPath C:\Users\Paolo\Desktop  

where 192.168.243.144 is the IP address of ESXi host to download support bundle from.

This time file will be named: vmsupport-<Generated_ID>.tgz

To automate bundles retrieval for any ESXi host managed by a specific vCenter Server, including support bundle for vCenter itself, you can use the following script:



As usual you can also find the script above in my GitHub page: DownloadSupportBundles.ps1

Support bundle download is also achieved by simply connecting, using a web browser, to an ESXi host to the following address:

 https://<username>:<password>@<ESXi_host_IP_address>/cgi-bin/vm-support.cgi  

The previous step can be automated using PowerShell (not PowerCLI). This could be useful when in need to get support bundles rapidly using a PC/server where PowerCLI is not installed (only PowerShell is needed).



That's all!!

VMware: A PowerCLI module to massively backup/restore ESXi hosts configurations

Flexibility is one of the greatest advantages of ESXi. Almost every aspect can be customized and tuned using both basic and advanced configurations in order to achieve a custom tailored system.

Configuring settings is a time consuming process, networking configurations for virtual standard switches, iSCSI vmkernel, port binding, NTP configuration, etc.

In case of host reinstall you can save precious time by using a great PowerCLI cmdlet: Get-VMHostFirmware.

This cmdlet creates a tar compressed archive containing all ESXi host's configurations. Conversely, to recover a backupped configuration, Set-VMHostFirmware cmdlet is used.

In this blog post I provide a PowerCLI module that will allow you to backup, and eventually restore, ESXi hosts configurations.

This module uses Get-VMHostFirmware and Set-VMHostFirmware cmdlets introducing the possibility to pass more than a single host as source for backup or target for restore and automatically enters/exits each host into/from maintenance mode before and after a backup restore occurs.

Let's start by briefly explaining how to use the module:

Typically the first step is to connect to a vCenter Server via PowerCLI in order to be able to perform backup or restores of one or more vCenter registered hosts. PowerCLI connection to a single ESXi host is also supported but for obvious reasons you can backup/restore only that specific host.

Once downloaded the script provided below you will have a .psm1 file, which is the common extension for PowerShell modules, that must be imported into PowerCLI in order to use it.

 Import-Module C:\Users\Paolo\WindowsPowerShell\Modules\BackupRestore  

Where C:\Users\Paolo\WindowsPowerShell\Modules\ is the path of BackupRestore.psm1 file on your PC.

BackupRestore module introduces into PowerCLI two new functions: Backup-VMHost and Restore-VMHost.

Backup-VMHost requires as mandatory parameters:

-VMHost: backup source. IP address or FQDN of one or more ESXi hosts you want to backup configurations from.
-FilePath: location where configuration bundles will be saved.

The following example backups the configuration of host 192.168.243.143 and 192.168.243.144 then save their configurations into C:\Users\Paolo\Desktop.

 Backup-VMHost -VMHost 192.168.243.143,192.168.243.144 -FilePath C:\Users\Paolo\Desktop  

Restore-VMHost requires:

-VMHost: Restore destination. IP address or FQDN of one or more ESXi hosts you want to recover configurations to.

-FilePath: location where configuration bundles can be fetched in order to be restored on host(s)

-HostUsername: ESXi host username

-HostPassword: ESXi host password

The following command will first place each host into maintenance mode then restore configuration bundle on ESXi host 192.168.243.143 and 192.168.243.144 taking it from C:\Users\Paolo\Desktop folder.

By default configuration bundles are saved as configBundle-<ESXi_host_IP_address>.tar (for example: configBundle-192.168.243.143.tar) so Restore-VMHost function expects such named files to be present in source folder.
Finally it will wait a few minutes (3 by default), giving to each host time to perform a reboot, then removes host from maintenance mode.

 Restore-VMHost -VMHost 192.168.243.143,192.168.243.144 -FilePath C:\Users\Paolo\Desktop -HostUsername root -HostPassword vmware  

As usual this code is also available on my GitHub repository: BackupRestore.psm1



That's all!!

VMware: vCloud tenants reports using jQuery Mobile and PowerCLI

In this post we will create a simple HTML report using PowerCLI for Tenants and jQuery Mobile. jQuery Mobile provides an HTML5 user interface designed to allow the creation of web pages specifically suited for mobile devices with smaller screen compared to classic desktop/laptop PCs.

From PowerCLI perspective the following script is really simple, it retrieve some data from the vCloud environment then generates the HTML report by iterating HTML code (precisely <li> blocks) in order to create vApps list then finally it outputs the result as a single HTML by concatenating the composing elements.

As for jQuery Mobile perspective this is not complex at all, the resulting page structure is linear. In the header section there is the declaration of jQuery Mobile usage:

 <head>  
 <link rel="stylesheet" href="http://code.jquery.com/mobile/1.4.2/jquery.mobile-1.4.2.min.css" />  
 <script src="http://code.jquery.com/jquery-1.9.1.min.js"></script>  
 <script src="http://code.jquery.com/mobile/1.4.2/jquery.mobile-1.4.2.min.js"></script>  
 </head>  

The body is constructed in a block approach: several blocks with different functions while the body block is the main elements that is filled by different informations regarding cloud organization:

 <div data-role="page" id="home"> --Page  
   <div data-role="header">  
     HEADER BLOCK  
   </div>  
   <div data-role="content">  
     CONTENT BLOCK  
   </div>  
   <div data-role="footer">  
     FOOTER BLOCK  
   </div>  
 </div>  

The PowerCLI script is intended to be run by vCloud tenants, having enough privileges on the organization they manage, against a vCloud Director server. The expected output will be an HTML page that can be consumed by any HTML5 capable browser, but, as previously said, jQuery Mobile does its best when used on a mobile device like a tablet or a smartphone.

Results will be similar to these:

The PowerCLI code to create this report is the following. As usual you can also find it in my GitHub repository: Tenants Mobile Reports.ps1



That's all!!

VMware: PowerCLI for vCloud Tenants 101

PowerCLI for Tenants is a great way to offer scripting capabilities to your cloud users. It works identically to the "classic" PowerCLI but it offers a limited set of cmdlets specifically designed to provide tenants control over their cloud organizations.

This is an introductory post regarding PowerCLI for tenants in which I will explore some of the many useful cmdlets that allow users to control a cloud organization through PowerCLI.

First step is to install PowerCLI for Tenants. At this time version 5.1 release 2 is the latest available and can be downloaded here. If you already have the "classic" vSphere PowerCLI installed on your machine you need to remove it first in order to install the tenants version.

As usual, when dealing with PowerCLI, official documentation comes in handy.

In PowerCLI for tenants every user (tenant) connects to a vCloud Director server in order to run cmdlets against an organization the user is entitled to interact with. This means, for example, that different tenants can be connected at the same time to the same vCloud Director server interacting with different organizations.

Connection to an organization is performed using the following command:

Connect-CIServer -Server 192.168.243.50 -User User1 -Password MyPassword0! -Org HostileCoding

 Name              User              Org  
 ----              ----              ---  
 192.168.243.50         User1             HostileCoding  

where -Server is the vCloud Director server IP or FQDN and -Org is the organization name to connect to.

To retrieve an organization's details Get-Org cmdlet is used.

 Enabled     : True  
 CanPublish   : False  
 DeployedVMQuota : 8  
 StoredVMQuota  : 10  
 VdcCount    : 1  
 CatalogCount  : 1  
 VAppCount    : 1  
 Href      : https://192.168.243.50/api/admin/org/a3314e21-f6b8-4c53-9690-b44e96141b56  
 FullName    : HostileCoding  
 ExtensionData  : VMware.VimAutomation.Cloud.Views.AdminOrg  
 Description   : Organization Cloud  
 Id       : urn:vcloud:org:a3314e21-f6b8-4c53-9690-b44e96141b56  
 Name      : HostileCoding  

DeployedVMQuota and StoredVMQuota respectively count the maximum number of virtual machines that can be deployed and stored simultaneously by a member of this organization.

While Get-OrgVdc retrieves details about an organization:

 Href          : https://192.168.243.50/api/vdc/9fa21642-32a8-46b3-91ac-aedfc42a2937  
 AllocationModel     : ReservationPool  
 Enabled         : True  
 CpuUsedGhz       : 0  
 CpuLimitGhz       : 3  
 CpuAllocationGhz    : 3  
 CpuOverheadGhz     : 0  
 MemoryUsedGB      : 0.119140625  
 MemoryLimitGB      : 2  
 MemoryAllocationGB   : 2  
 MemoryOverheadGB    : 0  
 StorageUsedGB      : 10.25  
 StorageLimitGB     : 19.193359375  
 StorageAllocationGB   : -1  
 StorageOverheadGB    : -1  
 VAppCount        : 1  
 Status         : Ready  
 NetworkMaxCount     : 2  
 VMMaxCount       :  
 NicMaxCount       :  
 MemoryGuaranteedPercent :  
 CpuGuaranteedPercent  :  
 VMCpuCoreMHz      :  
 ThinProvisioned     :  
 UseFastProvisioning   :  
 ExtensionData      : VMware.VimAutomation.Cloud.Views.Vdc  
 Description       : Customer1 Virtual DataCenter  
 Id           : urn:vcloud:vdc:9fa21642-32a8-46b3-91ac-aedfc42a2937  
 Name          : VDC-Customer1  

Quite important are the resources limitation exposed by the previous command: StorageLimitGB, MemoryLimitGB, CpuLimitGhz, NetworkMaxCount, VMMaxCount indicate how many resources an organization is entitled to use from the cloud provider.

Get-OrgVdc retrieves a great amount of details regarding organizational's datacenter like: adopted allocation model, reservations, limits and resources used.

To retrieve organization's catalog(s) Get-Catalog cmdlet is used.

 Published     : False  
 Shared      : False  
 Created      : 4/28/2014 2:38:05 PM  
 Org        : HostileCoding  
 Owner       : system  
 VAppTemplateCount : 1  
 MediaCount    : 0  
 ExtensionData   : VMware.VimAutomation.Cloud.Views.AdminCatalog  
 Href       : https://192.168.243.50/api/admin/catalog/873e5007-896d-418b-ae0f-67e9131a1eb6  
 Description    :  
 Id        : urn:vcloud:catalog:873e5007-896d-418b-ae0f-67e9131a1eb6  
 Name       : Catalog1  

In vCloud Director user roles are extremely important because they grant certain capabilities to specific tenants. As you know vCD has five predefined user roles (excluding sysadmin role which is the global role of the user installing/managing the global vCD environment):

• Organization Administrator 
• Catalog Author
• vApp Author 
• vApp User 
• Console Access Only.

By being in one of these groups a tenant can or cannot perform several tasks. To retrieve to which role, with corresponding permissions, each user has been assigned to the Get-CIRole cmdlet is used:

Get-CIRole -User User1

 Name              ReadOnly Rights  
 ----              -------- ------  
 Organization Administrator   False  {Organization: Edit Properties, Orga...  

Then to retrieve detailed infos about a specific user:

Get-CIUser -Name User1

 Href      : https://192.168.243.50/api/admin/user/056c48fd-1b7f-4d48-8da6-81cb53ff0af3  
 StoredVMQuota  : 0  
 StoredVMCount  : 0  
 IM       :  
 DeployedVMQuota : 0  
 DeployedVMCount : 0  
 Phone      :  
 Org       : HostileCoding  
 LdapName    : User1  
 Locked     : False  
 IsLDapUser   : False  
 HasGroupRole  : False  
 External    : False  
 Enabled     : True  
 FullName    : User One  
 Email      : user.one@mycompany.com  
 ExtensionData  : VMware.VimAutomation.Cloud.Views.User  
 Name      : user1  
 Id       : urn:vcloud:user:056c48fd-1b7f-4d48-8da6-81cb53ff0af3  
 Description   :  

This informs us regarding the organization the user belongs, if it is enabled or not, as well as how many VMs has deployed or stored.

To retrieve vApps a user is entitled to access/manage the Get-CIVApp cmdlet is used.

 Href        : https://192.168.243.50/api/vApp/vapp-7591186a-1d3d-4a0c-b392-b2bd7d373210  
 ExtensionData   : VMware.VimAutomation.Cloud.Views.VApp  
 Enabled      : True  
 Status       : PoweredOn  
 SizeGB       : 5  
 CpuCount      : 1  
 MemoryAllocationMB : 256  
 MemoryAllocationGB : 0.25  
 InMaintenanceMode : False  
 Owner       : system  
 Org        : HostileCoding  
 Shared       : False  
 StorageLease    : 30.00:00:00  
 RuntimeLease    : 7.00:00:00  
 Description    :  
 Id         : urn:vcloud:vapp:7591186a-1d3d-4a0c-b392-b2bd7d373210  
 Name        : myFirstCloudvApp  

A vApp is, identically to the ones seen in vSphere, a container for one or more VMs. Here the concept is a little broader since a vApp in vCloud can also contain one or more networks to which the VMs in the vApp are connected to. If you have a look at the previous output several other infos are retrieved like the number of vCPUs, the amount of RAM memory, the power status and both the Storage and Runtime lease expressed in days.

To retrieve all VMs belonging to a specified vApp we use the Get-CIVM cmdlet. In this example only one VM is contained inside myFirstCloudvApp (yes I know, I selected wrong OS during initial VM creation)!

Get-CIVM -vApp myFirstCloudvApp

 ExtensionData  : VMware.VimAutomation.Cloud.Views.Vm  
 Status     : PoweredOn  
 Deleted     : False  
 GuestOsFullName : Red Hat Enterprise Linux 6 (64-bit)  
 CpuCount    : 1  
 MemoryMB    : 256  
 MemoryGB    : 0.25  
 OrgVdc     : VDC-Customer1  
 VApp      : myFirstCloudvApp  
 Description   : This is a simple configuration for Ubuntu Server  
 Href      : https://192.168.243.50/api/vApp/vm-e11d8857-6fa5-4fa1-ab52-b9  
          3c91a23c18  
 Id       : urn:vcloud:vm:e11d8857-6fa5-4fa1-ab52-b93c91a23c18  
 Name      : Ubuntu Minimal  

In the following posts I will provide some scripts to offer some useful capabilities to vCloud tenants.

That's all!!

VMware: Automate the manual deployment of vCloud Agent

While creating provider VDCs in vCloud Director sometimes occurs that not every host is being prepared successfully due to various reasons: something configured incorrectly, cannot place host into maintenance mode, cannot communicate with host, etc. This incapability to correctly dialogue with ESXi hosts leads to the missing deployment of vCloud agent on the hosts belonging to a provider VDC.

The best way to solve this is to fix the underlying problem. Let's be clear: when everything is fine vCloud Director agent deployment works smooth but when something is not the host preparation may become quite a pita. A way to fix this is to proceed with a manual deployment of vCloud agent but, as you reckon, this is quite uncomfortable when dealing with a large number of ESXi hosts.

Today's post goal is to provide a PowerCLI script that allows you to automate the manual process of vCloud agent deployment as well as a mass agent upgrade.

The following script will:

-Connect to a vCenter Server
-Download to your local machine the specified vCloud agent version from vCloud Director virtual machine
-For each host on which you need to install vCloud agent script will:
    -Place host into maintenance mode
    -Enable SSH
    -Check whether vCloud agent is installed or not. If already installed uninstall it first
    -Transfer vCloud agent over to ESXi host using Putty Secure Copy
    -Install vCloud agent
    -Disable SSH
    -Exit maintenance mode
   
Once vCloud agent is successfully installed ESXi host is seen as correctly prepared from vCloud Director.

This is the PowerCLI code. As always you can also find it in my GitHub repository. Automate vCloud Agent Deployment.ps1



Code is quite well commented but let me spend a few words on some cmdlets.

Initial part comprises variable declaration. $FileVcloudAgent is the version of vCloud agent that will be downloaded from vCloud Director virtual machine and later installed on all specified ESXi hosts. In the script I use vcloudagent-esx51-5.1.0-799577.zip file. As name states this version is for ESXi 5.1. If you have different ESXi versions you MUST choose the appropriate ones, like vcloudagent-esx55-5.5.0-1280396.zip if you are using ESXi 5.5 (with vCloud Director 5.5) or vcloudagent-esx50-5.1.0-799574.zip in case of ESXi 5.0, etc.

$puttyScpLocation points to the location of Putty Secure Copy (pscp.exe) that can be downloaded from here and must be placed somewhere on your local machine. Putty Secure Copy is used to copy vCloud agent over to /tmp/ folder of each ESXi host.

Copy-VMGuestFile is a PowerCLI cmdlet used to download specified file from vCloud Director virtual machine to your local machine. -GuestToLocal parameter states that we are downloading files from a VM. Au contraire -LocalToGuest would have been used.

Crucial commands of script above are the VIB uninstall/install ones. To perform a VIB installation on an ESXi host you must use esxcli commands using the following syntax:

 $esx=Get-EsxCli -VMHost <ESXi host on which use esxcli>  

esxcli commands can be executed from PowerCLI by exposing the esxcli functionalities. This is done using:

 $esx=Get-EsxCli -VMHost <ESXi host on which use esxcli>  

and later calling esxcli installation command:

 $esx.software.vib.install("/tmp/$($FileVcloudAgent)", $false, $false, $true, $false, $false)  

On which each parameter has the following meaning:

 Value:vim.EsxCLI.software.vib.install.InstallationResult  
    install(string[] depot, boolean dryrun, boolean force,  
    boolean maintenancemode, boolean noliveinstall, boolean  
    nosigcheck, string proxy, string[] vibname, string[]  
    viburl)  

That's all!!