Average CPU and Memory Use Per ESX Cluster
More stats for my capacity report, this time I want to know on a typical day in the month what is the average CPU and memory use like across my hosts and clusters. Note: this post is not aimed at troubleshooting performance issues, rather at a high level gives me a reasonable idea of the CPU and memory use in each cluster during peak and non-peak hours. By running this each month I can look at possible trends and where extra resource may be required.
Get-Stat is a very useful cmdlet in PowerCLI to extract performance data across multiple hosts or VM’s for which it is not really easy to do the same kind of thing in the VI client when looking at multiple machines. By using this script we can collect the data for each host and then aggregate it across them all to get an average figure.
Connect-VIServervirtualcenter | Out-Null$Clusters=Get-Cluster$PeakStart=Get-Date-year2009-month8-day27-hour8-minute0-sec0$PeakEnd=Get-Date-year2009-month8-day27-hour18-minute0-sec0$OffPeakStart=Get-Date-year2009-month8-day27-hour18-minute0-sec0$OffPeakEnd=Get-Date-year2009-month8-day28-hour8-minute0-sec0foreach ($Clusterin$Clusters){ $VMHosts=Get-Cluster$Cluster | Get-VMHost$PeakCPUStatTotal=0$OffPeakCPUStatTotal=0$PeakMemoryStatTotal=0$OffPeakMemoryStatTotal=0foreach ($VMHostin$VMHosts){ $PeakCPUStat=$VMhost | Get-Stat-Start$PeakStart-Finish$PeakEnd-statcpu.usage.average-IntervalSecs4500$PeakCPUStatAverage=$PeakCPUStat | Measure-Objectvalue-ave$PeakCPUStatTotal=$PeakCPUStatTotal+$PeakCPUStatAverage.average $OffPeakCPUStat=$VMhost | Get-Stat-Start$OffPeakStart-Finish$OffPeakEnd-statcpu.usage.average-IntervalSecs4500$OffPeakCPUStatAverage=$OffPeakCPUStat | Measure-Objectvalue-ave$OffPeakCPUStatTotal=$OffPeakCPUStatTotal+$OffPeakCPUStatAverage.average $PeakMemoryStat=$VMhost | Get-Stat-Start$PeakStart-Finish$PeakEnd-statmem.usage.average-IntervalSecs4500$PeakMemoryStatAverage=$PeakMemoryStat | Measure-Objectvalue-ave$PeakMemoryStatTotal=$PeakMemoryStatTotal+$PeakMemoryStatAverage.average $OffPeakMemoryStat=$VMhost | Get-Stat-Start$OffPeakStart-Finish$OffPeakEnd-statmem.usage.average-IntervalSecs4500$OffPeakMemoryStatAverage=$OffPeakMemoryStat | Measure-Objectvalue-ave$OffPeakMemoryStatTotal=$OffPeakMemoryStatTotal+$OffPeakMemoryStatAverage.average
} $PeakCPUStatResult\= \[math\]::round(($PeakCPUStatTotal/$VMHosts.count), 0) $OffPeakCPUStatResult\= \[math\]::round(($OffPeakCPUStatTotal/$VMHosts.count), 0) $PeakMemoryStatResult\= \[math\]::round(($PeakMemoryStatTotal/$VMHosts.count), 0) $OffPeakMemoryStatResult\= \[math\]::round(($OffPeakMemoryStatTotal/$VMHosts.count), 0) Write-Host"$Cluster has a Peak CPU Stat of $PeakCPUStatResult %"Write-Host"$Cluster has an Off Peak CPU Stat of $OffPeakCPUStatResult %"Write-Host"$Cluster has a Peak Memory Stat of $PeakMemoryStatResult %"Write-Host"$Cluster has an Off Peak Memory Stat of $OffPeakMemoryStatResult %" }
Update:
Thanks to Glenn Sizemore for the suggestion of using the cluster level counters. I have re-visited the script and updated based on those counters and can confirm it runs a lot quicker. Since the cluster CPU counter result is in MHz and my original script was based around percentages, we first of all have to calculate the total amount of MHz available in the cluster, after that its just a question of expressing the cpu.usagemhz.average as a percentage of the total. Memory is easier because the valule reported by the cluster with mem.usage.average is already a percentage.
Connect-VIServervirtualcenter | Out-Null$Clusters=Get-Cluster$PeakStart=Get-Date-year2009-month8-day27-hour8-minute0-sec0$PeakEnd=Get-Date-year2009-month8-day27-hour18-minute0-sec0$OffPeakStart=Get-Date-year2009-month8-day27-hour18-minute0-sec0$OffPeakEnd=Get-Date-year2009-month8-day28-hour8-minute0-sec0foreach ($Clusterin$Clusters){ $TotalCPUHZ=0$TotalCPUMHZ=0$VMHosts=$Cluster | Get-VMHostforeach ($VMHostin$VMHosts){ $View=$VMHost | Get-View$NumCpuCores=$View.Hardware.CpuInfo.NumCpuCores $Hz=$View.Hardware.CpuInfo.Hz $CPUHZ=$NumCpuCores*$Hz$TotalCPUHZ=$TotalCPUHZ+$CPUHZ } $TotalCPUMHZ=$TotalCPUHZ/1000000$PeakCPUStatAverage=$cluster | Get-Stat-Start$PeakStart-Finish$PeakEnd-Statcpu.usagemhz.average-IntervalSecs4500 | Measure-Objectvalue-ave$PeakCPUStatResult= [math]::round(($PeakCPUStatAverage.average /$TotalCPUMHZ*100), 0) $OffPeakCPUStatAverage=$cluster | Get-Stat-Start$OffPeakStart-Finish$OffPeakEnd-Statcpu.usagemhz.average-IntervalSecs4500 | Measure-Objectvalue-ave$OffPeakCPUStatResult= [math]::round(($OffPeakCPUStatAverage.average /$TotalCPUMHZ*100), 0) $PeakMemoryStatAverage=$cluster | Get-Stat-Start$PeakStart-Finish$PeakEnd-Statmem.usage.average-IntervalSecs4500 | Measure-Objectvalue-ave$PeakMemoryStatResult= [math]::round($PeakMemoryStatAverage.average, 0) $OffPeakMemoryStatAverage=$cluster | Get-Stat-Start$OffPeakStart-Finish$OffPeakEnd-Statmem.usage.average-IntervalSecs4500 | Measure-Objectvalue-ave$OffPeakMemoryStatResult= [math]::round($OffPeakMemoryStatAverage.average, 0) Write-Host"$Cluster has a Peak CPU Stat of $PeakCPUStatResult %“Write-Host”$Cluster has an Off Peak CPU Stat of $OffPeakCPUStatResult %“Write-Host”$Cluster has a Peak Memory Stat of $PeakMemoryStatResult %“Write-Host”$Cluster has an Off Peak Memory Stat of $OffPeakMemoryStatResult %" }
Of course the original script is still valid if you don’t have you hosts clustered, rather you simply have them grouped into folders. Although uncommon, this can occur if you have a particular workload spread across hosts which you wish to evaluate, but they were not clustered for whatever reason, cost being a typical exaxmple.
In this instance instead of:
$Clusters=Get-Cluster
Replace it with
$VMHosts=Get-Folder’Non-Clustered Hosts’ | Get-VMHost
Thanks again to Glenn for his tip about the cluster stats.
Update 28/02/2014:
Following some feedback in the comments it was high time I updated this for something more reusable. You can use the below function like this:
Get-Cluster 'Cluster01' | Get-ClusterAverageCpuMemory -PeakStartHour 8 -PeakEndHour 18 -Day 26 -Month 2 -Year 2014
Function:
function Get-ClusterAverageCpuMemory { <# .SYNOPSIS Get average CPU and Memory stats for a vSphere cluster
.DESCRIPTION Get average CPU and Memory stats for a vSphere cluster
.PARAMETER Name A vSphere Cluster object
.PARAMETER PeakStartHour Hour that Peak stats should be calcualted from, e.g. 8
.PARAMETER PeakEndHour Hour that Peak stats should be calcualted to, e.g. 18
.PARAMETER Day Day of the month, e.g. 10
.PARAMETER Month Month of the year, e.g. 8
.PARAMETER Year Year, e.g. 2013
.INPUTS System.Management.Automation.PSObject System.Int System.String
.OUTPUTS System.Management.Automation.PSObject.
.EXAMPLE PS> Get-ClusterAverageCpuMemory -Name Cluster01 -PeakStartHour 8 -PeakEndHour 18 -Day 10 -Month 8 -Year 2013
.EXAMPLE PS> Get-Cluster Cluster01 | Get-ClusterAverageCpuMemory -PeakStartHour 8 -PeakEndHour 18 -Day 10 -Month 8 -Year 2013
.NOTES Version: 1.0 - First draft Date: 28/02/2014 Tag: vsphere,cluster #> \[CmdletBinding()\]\[OutputType('System.Management.Automation.PSObject')\]
Param (
\[parameter(Mandatory=$true,ValueFromPipeline=$true)\] \[ValidateNotNullOrEmpty()\] \[PSObject\[\]\]$Name,
\[parameter(Mandatory=$true,ValueFromPipeline=$false)\] \[ValidateNotNullOrEmpty()\] \[Int\]$PeakStartHour,
\[parameter(Mandatory=$true,ValueFromPipeline=$false)\] \[ValidateNotNullOrEmpty()\] \[Int\]$PeakEndHour,
\[parameter(Mandatory=$true,ValueFromPipeline=$false)\] \[ValidateNotNullOrEmpty()\] \[Int\]$Day,
\[parameter(Mandatory=$true,ValueFromPipeline=$false)\] \[ValidateNotNullOrEmpty()\] \[Int\]$Month,
\[parameter(Mandatory=$true,ValueFromPipeline=$false)\] \[ValidateNotNullOrEmpty()\] \[Int\]$Year )
begin {
$DaysInMonth = \[DateTime\]::DaysInMonth($Year, $Month) $IsLeapYear = \[DateTime\]::IsLeapYear($Year)
if ($IsLeapYear){ $LastDayOfYear = 366 } else { $LastDayOfYear = 365 }
$PeakStart = Get-Date -Year $Year -Month $Month -Day $Day -Hour $PeakStartHour -Minute 0 -Second 0 $PeakEnd = Get-Date -Year $Year -Month $Month -Day $Day -Hour ($PeakEndHour - 1) -Minute 59 -Second 59 $OffPeakStart = Get-Date -Year $Year -Month $Month -Day $Day -Hour $PeakEndHour -Minute 0 -Second 0 if (((Get-Date -Year $Year -Month $Month -Day $Day).DayOfYear) -eq $LastDayOfYear){
$OffPeakEnd = Get-Date -Year ($Year +1) -Month 1 -Day 1 -Hour ($PeakStartHour -1) -Minute 59 -Second 59 } elseif ($Day -eq $DaysInMonth){
$OffPeakEnd = Get-Date -Year $Year -Month ($Month + 1) -Day 1 -Hour ($PeakStartHour -1) -Minute 59 -Second 59 } else {
$OffPeakEnd = Get-Date -Year $Year -Month $Month -Day ($Day + 1) -Hour ($PeakStartHour -1) -Minute 59 -Second 59 }
$OutputObject = @() }
process {
try { foreach ($Cluster in $Name){
if ($Cluster.GetType().Name -eq "string"){
try { $Cluster = Get-Cluster $Cluster -ErrorAction Stop } catch \[Exception\]{ Write-Warning "Cluster $Cluster does not exist" continue } }
elseif ($Cluster -isnot \[VMware.VimAutomation.ViCore.Impl.V1.Inventory.ClusterImpl\]){ Write-Warning "You did not pass a string or a Cluster object" continue }
$TotalCPUMHZ = $Cluster.TotalCPUGhz \* 1000
$PeakCPUStatAverage = $Cluster | Get-Stat -Start $PeakStart -Finish $PeakEnd -Stat cpu.usagemhz.average -IntervalSecs 4500 | Measure-Object value -Average $PeakCPUStatResult = \[math\]::round(($PeakCPUStatAverage.Average / $TotalCPUMHZ \*100), 0)
$OffPeakCPUStatAverage = $Cluster | Get-Stat -Start $OffPeakStart -Finish $OffPeakEnd -Stat cpu.usagemhz.average -IntervalSecs 4500 | Measure-Object value -Average $OffPeakCPUStatResult = \[math\]::round(($OffPeakCPUStatAverage.Average / $TotalCPUMHZ \*100), 0)
$PeakMemoryStatAverage = $Cluster | Get-Stat -Start $PeakStart -Finish $PeakEnd -Stat mem.usage.average -IntervalSecs 4500 | Measure-Object value -Average $PeakMemoryStatResult = \[math\]::round($PeakMemoryStatAverage.Average, 0)
$OffPeakMemoryStatAverage = $Cluster | Get-Stat -Start $OffPeakStart -Finish $OffPeakEnd -Stat mem.usage.average -IntervalSecs 4500 | Measure-Object value -Average $OffPeakMemoryStatResult = \[math\]::round($OffPeakMemoryStatAverage.Average, 0)
$hash = @{
Cluster = $Cluster PeakCPUStatResult = $PeakCPUStatResult OffPeakCPUStatResult = $OffPeakCPUStatResult PeakMemoryStatResult = $PeakMemoryStatResult OffPeakMemoryStatResult = $OffPeakMemoryStatResult
} $Object = New-Object PSObject -Property $hash $OutputObject += $Object } } catch \[Exception\]{
throw "Unable to get Cluster CPU and Memory Stats" } } end { Write-Output $OutputObject } }