CPU Utilisation Case Study

Configuration/Setup instructions can be found here: CPU Utilisation case study Configuration

The idea behind this wiki is to look at the CPU metrics from the baremetal host, the hypervisor and the VM itself to see the if the CPU utilisation metrics reported are consistent across all 3.

Two tests were conducted. One without isolating cpus from the OS and one with isolating CPUs from the OS on the baremetal host and pinning the vCPUs to those isolated cores. Results for the unisolated tests as expected have some unusual results...

The CPU plugins for collectd were configured not to report the CPU meterics per state - so just a single metric for utilization

The interval used for the test was for 1 second (this interval was used for the host and guest collectd cpu plugins, the virt plugin wouldn't go down to a 1 second configuration).

1 second was selected in order to get the timing differences between starting the plugin on the host and starting the plugin on the guest down to a lower value...

Tests were conducted with more/less an idle environment and will be repeated with a busy env.

Function used to convert nanoseconds virt metrics to percent:

static double cpu_ns_to_percent(unsigned int node_cpus,

unsigned long long cpu_time_old,

unsigned long long cpu_time_new) {

double percent = 0.0;

unsigned long long cpu_time_diff = 0;

double time_diff_sec = CDTIME_T_TO_DOUBLE(plugin_get_interval());

if (node_cpus != 0 && time_diff_sec != 0 && cpu_time_old != 0) {

cpu_time_diff = cpu_time_new - cpu_time_old;

percent = ((double)(100 * cpu_time_diff)) /

(time_diff_sec * node_cpus * NANOSEC_IN_SEC);

}

With Isolation (stress test test, 1 second interval)

Host View		VM View		Hypervisor View

Host View		VM View		Hypervisor View
silpixa00399126/cpu-10/percent-active-2017-03-07		ubuntu_vm/cpu-0/percent-active-2017-03-07		6c33ff2f-40d0-7a45-873b-7f3ead32413b/virt/virt_cpu_total-2017-03-07
epoch	value	epoch	value	epoch	value	percent
				1492431855	1343320000000.00
1492431856	78.35	1492431856	62.19512	1492431856	1344110000000.00	79.72
1492431857	100.00	1492431857	82.02247	1492431857	1345110000000.00	100.00
1492431858	100.00	1492431858	56.25	1492431858	1346110000000.00	100.00
1492431859	100.00	1492431859	62.5	1492431859	1347110000000.00	100.00
1492431860	100.00	1492431860	89.58333	1492431860	1348110000000.00	100.10
1492431861	100.00	1492431861	75.25773	1492431861	1349110000000.00	99.90
1492431862	23.00	1492431862	0	1492431862	1349310000000.00	20.00
1492431863	2.02	1492431863	0	1492431863	1349330000000.00	2.00
1492431864	1.00	1492431864	0	1492431864	1349350000000.00	2.00
1492431865	1.01	1492431865	19.19192	1492431865	1349390000000.00	4.00
1492431866	19.19	1492431866	0	1492431866	1349550000000.00	16.00
1492431867	0.99	1492431867	14.14141	1492431867	1349570000000.00	2.00
1492431868	82.83	1492431868	100	1492431868	1350420000000.00	84.92
1492431869	100.00	1492431869	100	1492431869	1351420000000.00	100.00
1492431870	100.00	1492431870	100	1492431870	1352420000000.00	100.00
1492431871	100.00	1492431871	100	1492431871	1353420000000.00	100.00
1492431872	100.00	1492431872	100	1492431872	1354420000000.00	100.10
1492431873	100.00	1492431873	100	1492431873	1355420000000.00	99.90
1492431874	100.00	1492431874	100	1492431874	1356420000000.00	100.00
1492431875	100.00			1492431875	1357420000000.00	100.10
1492431876	100.00			1492431876	1358420000000.00	99.90
1492431877	100.00			1492431877	1359420000000.00	100.00
1492431878	33.33			1492431878	1359730000000.00	31.00
1492431879	0.00			1492431879	1359730000000.00	0.00
1492431880	0.00			1492431880	1359750000000.00	2.00
1492431881	1.00			1492431881	1359760000000.00	1.00
1492431882	1.00			1492431882	1359770000000.00	1.00

With Isolation (stress test test, 10 second interval)

Note: stress was ran for 20 seconds in this case

Host View		VM View		Hypervisor View

Host View		VM View		Hypervisor View
silpixa00399126/cpu-10/percent-active-2017-03-07		ubuntu_vm/cpu-0/percent-active-2017-03-07		6c33ff2f-40d0-7a45-873b-7f3ead32413b/virt/virt_cpu_total-2017-03-07
epoch	value	epoch	value	epoch	value	percent
				1492433815	1398840000000.00
1492433825	77.56345	1492433833	65.57712	1492433825	1405960000000.00	71.44
1492433835	85.4	1492433843	40.22066	1492433835	1414630000000.00	86.70
1492433845	18.07229	1492433853	1.002004	1492433845	1416330000000.00	17.00
1492433855	1.705115	1492433863	1.202405	1492433855	1416530000000.00	2.00
1492433865	1.606426	1492433873	43.7	1492433865	1416730000000.00	2.00
1492433875	66.4	1492433883	100	1492433875	1423510000000.00	67.79
1492433885	100	1492433893	59.34066	1492433885	1433500000000.00	99.91
1492433895	37.21164	1492433903	0	1492433895	1437100000000.00	36.00
1492433905	1.606426	1492433903	0	1492433905	1437300000000.00	2.00
1492433915	1.803607	1492433913	0.901804	1492433915	1437490000000.00	1.90