The idea behind this wiki is to look at the CPU metrics from the baremetal host, the hypervisor and the VM itself to see the if the CPU utilisation metrics reported are consistent across all 3.
Two tests were conducted. One without isolating cpus from the OS and one with isolating CPUs from the OS on the baremetal host and pinning the vCPUs to those isolated cores.
The CPU plugins for collectd were configured not to report the CPU meterics per state - so just a single metric for utilization
The interval used for the test was for 1 second (this interval was used for the host and guest collectd cpu plugins, the virt plugin wouldn't go down to a 1 second configuration).
1 second was selected in order to get the timing differences between starting the plugin on the host and starting the plugin on the guest down to a lower value...
Tests were conducted with more/less an idle environment and will be repeated with a busy env.
Function used to convert nanoseconds virt metrics to percent:
static double cpu_ns_to_percent(unsigned int node_cpus,
unsigned long long cpu_time_old,
unsigned long long cpu_time_new) {
double percent = 0.0;
unsigned long long cpu_time_diff = 0;
double time_diff_sec = CDTIME_T_TO_DOUBLE(plugin_get_interval());
if (node_cpus != 0 && time_diff_sec != 0 && cpu_time_old != 0) {
cpu_time_diff = cpu_time_new - cpu_time_old;
percent = ((double)(100 * cpu_time_diff)) /
(time_diff_sec * node_cpus * NANOSEC_IN_SEC);
}
With Isolation
Core 10 and 11 were isolated from the OS using the isolcpus argument to grub command line and using virsh to Pin the vCPUs to those cores:
$ virsh vcpupin test 0 10
$ virsh vcpupin test 1 11
$ virsh vcpupin test
VCPU: CPU Affinity
----------------------------------
0: 10
1: 11
Host View | VM View | Virt view | ||||
---|---|---|---|---|---|---|
silpixa00399126/cpu-10/percent-active-2017-03-07 | ubuntu_vm/cpu-0/percent-active-2017-03-07 | 6c33ff2f-40d0-7a45-873b-7f3ead32413b/virt/virt_vcpu-0-2017-03-07 | ||||
epoch | value | epoch | value | epoch | value | percent |
1488880609.34 | 0 | 1488880609.14 | 0 | 1488880613.37 | 16590000000.00 | |
1488880610.34 | 0 | 1488880610.14 | 0 | |||
1488880611.34 | 0 | 1488880611.14 | 0 | |||
1488880612.34 | 0 | 1488880612.14 | 0 | |||
1488880613.34 | 0 | 1488880613.14 | 0.990099 | |||
1488880614.34 | 0.990099 | 1488880614.14 | 0 | |||
1488880615.34 | 0 | 1488880615.14 | 0 | |||
1488880616.34 | 0 | 1488880616.14 | 0 | |||
1488880617.34 | 0 | 1488880617.14 | 0 | |||
1488880618.34 | 0.990099 | 1488880618.14 | 0.990099 | |||
1488880619.34 | 0 | 1488880619.14 | 0 | |||
1488880620.34 | 0 | 1488880620.14 | 0 | |||
1488880621.34 | 0 | 1488880621.14 | 0 | |||
1488880622.34 | 0 | 1488880622.14 | 0 | |||
1488880623.34 | 0 | 1488880623.14 | 0 | 1488880623.36 | 16610000000.00 | 0.20 |
1488880624.34 | 0 | 1488880624.14 | 0 | |||
1488880625.34 | 0 | 1488880625.14 | 0 | |||
1488880626.34 | 0 | 1488880626.14 | 0 | |||
1488880627.34 | 1 | 1488880627.14 | 0 | |||
1488880628.34 | 1.980198 | 1488880628.14 | 0 | |||
1488880629.34 | 0 | 1488880629.14 | 0 | |||
1488880630.34 | 0 | 1488880630.14 | 0.990099 | |||
1488880631.34 | 0 | 1488880631.14 | 0 | |||
1488880632.34 | 0 | 1488880632.14 | 0 | |||
1488880633.34 | 0 | 1488880633.14 | 0 | 1488880633.36 | 16630000000.00 | 0.20 |
1488880634.34 | 0 | 1488880634.14 | 0 | |||
1488880635.34 | 0 | 1488880635.14 | 0 | |||
1488880636.34 | 0 | 1488880636.14 | 0 | |||
1488880637.34 | 0 | 1488880637.14 | 0 | |||
1488880638.34 | 1 | 1488880638.14 | 1 | |||
1488880639.34 | 1 | 1488880639.14 | 0 | |||
1488880640.34 | 0 | 1488880640.14 | 0 | |||
1488880641.34 | 0 | 1488880641.14 | 0 | |||
1488880642.34 | 0.990099 | 1488880642.14 | 0.990099 | |||
1488880643.34 | 0 | 1488880643.14 | 1.980198 | 1488880643.36 | 16660000000.00 | 0.30 |
1488880644.34 | 1 | 1488880644.14 | 1.980198 | |||
1488880645.34 | 0 | 1488880645.14 | 1.020408 | |||
1488880646.34 | 2.970297 | 1488880646.14 | 0 | |||
1488880647.34 | 0 | 1488880647.14 | 0 | |||
1488880648.34 | 1 | 1488880648.14 | 1.980198 | |||
1488880649.34 | 0 | 1488880649.14 | 0 | |||
1488880650.34 | 0 | 1488880650.14 | 0 | |||
1488880651.34 | 0.990099 | 1488880651.14 | 0 | |||
1488880652.34 | 0 | 1488880652.14 | 1 | |||
1488880653.34 | 1 | 1488880653.14 | 0 | 1488880653.36 | 16730000000.00 | 0.70 |
1488880654.34 | 1 | 1488880654.14 | 2.020202 | |||
1488880655.34 | 2.020202 | 1488880655.14 | 0 | |||
1488880656.34 | 0.990099 | 1488880656.14 | 0 | |||
1488880657.34 | 0 | 1488880657.14 | 0 | |||
1488880658.34 | 2 | 1488880658.14 | 1 | |||
1488880659.34 | 0 | 1488880659.14 | 0 | |||
1488880660.34 | 0 | 1488880660.14 | 0 | |||
1488880661.34 | 0 | 1488880661.14 | 0 | |||
1488880662.34 | 0 | 1488880662.14 | 0 | |||
1488880663.34 | 0 | 1488880663.14 | 0 | 1488880663.36 | 16780000000.00 | 0.50 |
1488880664.34 | 0 | 1488880664.14 | 0 | |||
1488880665.34 | 0 | 1488880665.14 | 0 | |||
1488880666.34 | 0 | 1488880666.14 | 1 | |||
1488880667.34 | 1.980198 | 1488880667.14 | 0.990099 | |||
1488880668.34 | 1.010101 | 1488880668.14 | 1.010101 | |||
1488880669.34 | 0 | 1488880669.14 | 0 | |||
1488880670.34 | 0 | 1488880670.14 | 0 | |||
1488880671.34 | 0 | 1488880671.14 | 0 | |||
1488880672.34 | 3.960396 | 1488880672.14 | 10.89109 | |||
1488880673.34 | 7 | 1488880673.14 | 0 | 1488880673.37 | 16910000000.00 | 1.30 |
1488880674.34 | 0 | 1488880674.14 | 0 | |||
1488880675.34 | 0 | 1488880675.14 | 0 | |||
1488880676.34 | 0 | 1488880676.14 | 0.990099 | |||
1488880677.34 | 0 | 1488880677.14 | 0 | |||
1488880678.34 | 1 | 1488880678.14 | 2 | |||
1488880679.34 | 0 | 1488880679.14 | 0 | |||
1488880680.34 | 0 | 1488880680.14 | 0 | |||
1488880681.34 | 1.960784 | 1488880681.14 | 0 | |||
1488880682.34 | 0 | 1488880682.14 | 0 | |||
1488880683.34 | 0 | 1488880683.14 | 0 | 1488880683.36 | 16940000000.00 | 0.30 |
1488880684.34 | 0 | 1488880684.14 | 0 | |||
1488880685.34 | 0 | 1488880685.14 | 0 | |||
1488880686.34 | 0 | 1488880686.14 | 0 | |||
1488880687.34 | 0 | 1488880687.14 | 0 | |||
1488880688.34 | 1 | 1488880688.14 | 1 | |||
1488880689.34 | 0 | 1488880689.14 | 0 | |||
1488880690.34 | 0 | 1488880690.14 | 0 | |||
1488880691.34 | 0 | 1488880691.14 | 0.990099 | |||
1488880692.34 | 0.990099 | 1488880692.14 | 0 | |||
1488880693.34 | 0 | 1488880693.14 | 1 | 1488880693.36 | 16960000000.00 | 0.20 |
1488880694.34 | 0 | 1488880694.14 | 0 | |||
1488880695.34 | 0 | 1488880695.14 | 0 | |||
1488880696.34 | 0 | 1488880696.14 | 0 | |||
1488880697.34 | 0.990099 | 1488880697.14 | 0.990099 | |||
1488880698.34 | 0 | 1488880698.14 | 0 | |||
1488880699.34 | 0 | 1488880699.14 | 0 | |||
1488880700.34 | 0 | 1488880700.14 | 0 | |||
1488880701.34 | 0 | 1488880701.14 | 0 | |||
1488880702.34 | 0.990099 | 1488880702.14 | 0 | |||
1488880703.34 | 0 | 1488880703.14 | 0 | 1488880703.36 | 16970000000.00 | 0.10 |
Without Isolation
Host View | VM View | Virt view | ||||
---|---|---|---|---|---|---|
silpixa00399126/cpu-10/percent-active-2017-03-07 | ubuntu_vm/cpu-0/percent-active-2017-03-07 | 6c33ff2f-40d0-7a45-873b-7f3ead32413b/virt/virt_vcpu-0-2017-03-07 | ||||
epoch | value | epoch | value | epoch | value | percent |
1488877698.94 | 2.020202 | 1488877698.64 | 1.010101 | 1488877697.98 | 628610000000.00 | |
1488877699.94 | 14.14141 | 1488877699.64 | 0 | |||
1488877700.94 | 0 | 1488877700.64 | 0 | |||
1488877701.94 | 0 | 1488877701.64 | 0 | |||
1488877702.94 | 0 | 1488877702.64 | 0 | |||
1488877703.94 | 2.941176 | 1488877703.64 | 0 | |||
1488877704.94 | 0 | 1488877704.64 | 0 | |||
1488877705.94 | 0 | 1488877705.64 | 0 | |||
1488877706.94 | 0 | 1488877706.64 | 0 | |||
1488877707.94 | 0 | 1488877707.64 | 0 | 1488877707.97 | 628760000000.00 | 1.50 |
1488877708.94 | 0 | 1488877708.64 | 0 | |||
1488877709.94 | 1.980198 | 1488877709.64 | 0 | |||
1488877710.94 | 0 | 1488877710.64 | 0 | |||
1488877711.94 | 0 | 1488877711.64 | 0 | |||
1488877712.94 | 0 | 1488877712.64 | 0 | |||
1488877713.94 | 0 | 1488877713.64 | 0 | |||
1488877714.94 | 0 | 1488877714.64 | 0 | |||
1488877715.94 | 0 | 1488877715.64 | 0 | |||
1488877716.94 | 0 | 1488877716.64 | 0 | |||
1488877717.94 | 0 | 1488877717.64 | 0 | 1488877717.96 | 628760000000.00 | 0.00 |
1488877718.94 | 2 | 1488877718.64 | 1.010101 | |||
1488877719.94 | 1 | 1488877719.64 | 0 | |||
1488877720.94 | 0 | 1488877720.64 | 0 | |||
1488877721.94 | 0.990099 | 1488877721.64 | 0 | |||
1488877722.94 | 0 | 1488877722.64 | 0 | |||
1488877723.94 | 0 | 1488877723.64 | 0 | |||
1488877724.94 | 1.980198 | 1488877724.64 | 0 | |||
1488877725.94 | 0 | 1488877725.64 | 0 | |||
1488877726.94 | 0 | 1488877726.64 | 0 | |||
1488877727.94 | 1 | 1488877727.64 | 0 | 1488877727.96 | 628790000000.00 | 0.30 |
1488877728.94 | 1 | 1488877728.64 | 1 | |||
1488877729.94 | 1.980198 | 1488877729.64 | 0 | |||
1488877730.94 | 0 | 1488877730.64 | 0 | |||
1488877731.94 | 0 | 1488877731.64 | 0 | |||
1488877732.94 | 0 | 1488877732.64 | 0 | |||
1488877733.94 | 0 | 1488877733.64 | 0 | |||
1488877734.94 | 0 | 1488877734.64 | 0 | |||
1488877735.94 | 0.990099 | 1488877735.64 | 0 | |||
1488877736.94 | 0 | 1488877736.64 | 0 | |||
1488877737.94 | 0.990099 | 1488877737.65 | 0 | 1488877737.96 | 628800000000.00 | 0.10 |
1488877738.94 | 8.080808 | 1488877738.64 | 2.020202 | |||
1488877739.94 | 2 | 1488877739.64 | 0 | |||
1488877740.94 | 0 | 1488877740.64 | 0 | |||
1488877741.94 | 0 | 1488877741.64 | 0 | |||
1488877742.94 | 0 | 1488877742.64 | 0 | |||
1488877743.94 | 0 | 1488877743.64 | 0 | |||
1488877744.94 | 0 | 1488877744.64 | 0 | |||
1488877745.94 | 0.990099 | 1488877745.64 | 0 | |||
1488877746.94 | 0 | 1488877746.64 | 0 | |||
1488877747.94 | 0 | 1488877747.64 | 0 | 1488877747.96 | 628830000000.00 | 0.30 |
1488877748.94 | 2 | 1488877748.64 | 0 | |||
1488877749.94 | 0 | 1488877749.64 | 0 | |||
1488877750.94 | 0 | 1488877750.64 | 0 | |||
1488877751.94 | 0 | 1488877751.64 | 0 | |||
1488877752.94 | 0 | 1488877752.64 | 0 | |||
1488877753.94 | 0 | 1488877753.64 | 1 | |||
1488877754.94 | 0 | 1488877754.64 | 0 | |||
1488877755.94 | 0 | 1488877755.64 | 0 | |||
1488877756.94 | 1 | 1488877756.64 | 0 | |||
1488877757.94 | 0.990099 | 1488877757.64 | 0 | 1488877757.97 | 628840000000.00 | 0.10 |
1488877758.94 | 1.010101 | 1488877758.64 | 1.010101 | |||
1488877759.94 | 2 | 1488877759.64 | 0 | |||
1488877760.94 | 0 | 1488877760.64 | 0 | |||
1488877761.94 | 0 | 1488877761.64 | 0 | |||
1488877762.94 | 0.990099 | 1488877762.64 | 0 | |||
1488877763.94 | 0 | 1488877763.64 | 0 | |||
1488877764.94 | 0 | 1488877764.64 | 0 | |||
1488877765.94 | 0 | 1488877765.64 | 0 | |||
1488877766.94 | 1 | 1488877766.64 | 0 | |||
1488877767.94 | 0 | 1488877767.64 | 0 | 1488877767.97 | 628880000000.00 | 0.40 |
1488877768.94 | 3 | 1488877768.64 | 1 | |||
1488877769.94 | 1.010101 | 1488877769.64 | 0 | |||
1488877770.94 | 1.960784 | 1488877770.64 | 0 | |||
1488877771.94 | 0 | 1488877771.64 | 0 | |||
1488877772.94 | 1 | 1488877772.64 | 0 | |||
1488877773.94 | 0 | 1488877773.64 | 0 | |||
1488877774.94 | 0.990099 | 1488877774.64 | 0 | |||
1488877775.94 | 1 | 1488877775.64 | 0 | |||
1488877776.94 | 0 | 1488877776.64 | 0 | |||
1488877777.94 | 0 | 1488877777.65 | 0 | 1488877777.96 | 628890000000.00 | 0.10 |
1488877778.94 | 3.960396 | 1488877778.64 | 1 | |||
1488877779.94 | 1 | 1488877779.64 | 0 | |||
1488877780.94 | 0 | 1488877780.64 | 0 | |||
1488877781.94 | 0 | 1488877781.64 | 0 | |||
1488877782.94 | 0 | 1488877782.64 | 0 | |||
1488877783.94 | 0 | 1488877783.64 | 0 | |||
1488877784.94 | 0.990099 | 1488877784.64 | 0 | |||
1488877785.94 | 0.990099 | 1488877785.64 | 0 | |||
1488877786.94 | 0 | 1488877786.64 | 0 | |||
1488877787.94 | 0 | 1488877787.64 | 0 | 1488877787.96 | 628900000000.00 | 0.10 |
1488877788.94 | 1 | 1488877788.64 | 0 | |||
1488877789.94 | 1.980198 | 1488877789.64 | 0 | |||
1488877790.94 | 0 | 1488877790.64 | 0 | |||
1488877791.94 | 0 | 1488877791.64 | 0 | |||
1488877792.94 | 0 | 1488877792.64 | 0 | |||
1488877793.94 | 1 | 1488877793.64 | 0 | |||
1488877794.94 | 0 | 1488877795.64 | 0 | |||
1488877795.94 | 1 | 1488877796.64 | 0 | |||
1488877796.94 | 1 | 1488877796.64 | 0 | |||
1488877797.94 | 0 | 1488877797.65 | 0 | 1488877797.96 | 628920000000.00 | 0.20 |