Collectd Metrics and Events

CPU

(A read plugin that retrieves CPU usage in Nanoseconds of as a percentage)

Interface

(A read plugin that retrieves Linux Interface statistics)

Memory

(A read plugin that retrieves memory usage statistics)

disk

(A read plugin that retrieves disk usage statistics)

Ping

(A read plugin that retrieves the RTT for a ping)

if pkg_recv > 1

latency_stddev = sqrt (((((double) pkg_recv) * latency_squared) - (latency_total * latency_total)) / ((double) (pkg_recv * (pkg_recv - 1))));

load

(A read plugin that retrieves system load for 1, 5 and 15 mins.)

OVS events

(A read plugin that retrieves events (like link status changes) from OVS.)

OVS Stats

(A read plugin that retrieves interface stats from OVS.)

The total number of packets received that were less than 64 octets in length (excluding framing bits but including FCS octets) and had either a bad Frame Check Sequence (FCS) with an integral number of octets (FCS Error) or a bad FCS with a non-integral number of octets (Alignment Error).

Hugepages

(A read plugin that retrieves the number of available and free hugepages on a platform as well as what is available in terms of hugepages per socket.)

Virtual memory makes it easy for several processes to share memory [TM1] .

Each process has its own virtual address space, which is mapped to physical memory by the operating system” [TM2] . The process views the virtual memory address space as a contiguous/linear address space, but in reality – the virtual addresses need to be mapped to physical addresses, this is typically done by the Memory Management Unit (MMU) on the CPU.

“There are two ways to enable the system to manage large amounts of memory:

• Increase the number of page table entries in the hardware memory management unit
• Increase the page size/use huge pages à to reduce the number of lookups

The first method is expensive, since the hardware memory management unit in a modern processor only supports hundreds or thousands of page table entries. Additionally, hardware and memory management algorithms that work well with thousands of pages (megabytes of memory) may have difficulty performing well with millions (or even billions) of pages. This results in performance issues: when an application needs to use more memory pages than the memory management unit supports, the system falls back to slower, software-based memory management, which causes the entire system to run more slowly.

Huge pages are blocks of memory that come in 2MB and 1GB sizes. The page tables used by the 2MB pages are suitable for managing multiple gigabytes of memory, whereas the page tables of 1GB pages are best for scaling to terabytes of memory” [TM3] 

More info on virtual memory and TLB lookups can be found:

1. http://www.tldp.org/LDP/tlk/mm/memory.html
2. https://lwn.net/Articles/253361/

processes

(A read plugin that  collects the number of processes, grouped by their state (e. g. running, sleeping, zombies, etc.). In addition to that, it can select detailed statistics about selected processes, grouped by name.)

virt

(A read plugin that uses virtualization API libvirt to gather statistics about virtualized guests on a system directly from the hypervisor, without a need to install collectd instance on the guest.)

Amount of memory which can be reclaimed by balloon without causing host swapping

RDT

(A read plugin that provides the last level cache utilization and memory bandwidth utilization)

NOTE: it's recommended to configure the interval for this plugin as 1 second

A higher IPC means that the processor can get more work done per unit time, which generally translates to faster application performance [0]

"Ideally every instruction a CPU gets should be read, executed and finished in one cycle, however that is never the case. The processor has to take the instruction, decode the instruction, gather the data (depends on where the data is), perform work on the data, then decide what to do with the result. Moving has never been more complicated, and the ability for a processor to hide latency, pre-prepare data by predicting future events or keeping hold of previous events for potential future use is all part of the plan. All the meanwhile there is an external focus on making sure power consumption is low and the frequency of the processor can scale depending on what the target device actually is". [1]

0 https://www.nextplatform.com/2016/03/31/examining-potential-hpc-benefits-new-intel-xeon-processors/

1 http://www.anandtech.com/show/9482/intel-broadwell-pt2-overclocking-ipc/3

Local Memory Bandwidth in Bytes per second, for a specified processor (socket), Resource Monitoring ID (RMID), and a given measurement time over the preceding measurement interval.

RMID may be mapped to Threads, Application processes, or VMs by the host OS or hypervisor.

Local refers to Memory Bandwidth used by RMID within the specified processor (socket).

https://software.intel.com/en-us/articles/memory-bandwidth-monitoring-proof-points

Remote Memory Bandwidth in Bytes per second, for a specified processor (socket), Resource Monitoring ID (RMID), and a given measurement time over the preceding measurement interval.

RMID may be mapped to Threads, Application processes, or VMs by the host OS or hypervisor.

RMID and Class of Service (CLOS) limits may be mapped to Threads, Application processes, or VMs by the host OS or hypervisor.

Question if units are in bytes or kibibits.

Al Morton after looking into this, I found that kilobytes is what is used for the cache.
In collectd we are reporting it simply in bytes. I updated the 4th column to reflect that as well as the definition.

dpdkstats

(A read plugin that retrieve stats from the DPDK extended

NIC stats API.)