Anuket Project

Intel PMU Executed Tests

Owned by Former user (Deleted)
Feb 01, 2018

System under test:

Default collectd configuration:

<Plugin intel_pmu>
HWCacheEvents true
KernelPMUEvents true
SWEvents true 
EventList "path_to_events_list"
HWSpecificEvents "L2_RQSTS.CODE_RD_HIT,L2_RQSTS.CODE_RD_MISS"
</Plugin>

Map of collectd.conf fields to events:

Based on OPNFV page:

https://wiki.opnfv.org/display/fastpath/Collectd+Metrics+and+Events

HWCacheEvents:

- L1-dcache-loads
- L1-dcache-load-misses
- L1-dcache-stores
- L1-dcache-store-misses
- L1-dcache-prefetches
- L1-dcache-prefetch-misses
- L1-icache-loads
- L1-icache-load-misses
- L1-icache-prefetches
- L1-icache-prefetch-misses
- LLC-loads
- LLC-load-misses
- LLC-stores
- LLC-store-misses
- LLC-prefetches
- LLC-prefetch-misses
- dTLB-loads
- dTLB-load-misses
- dTLB-stores
- dTLB-store-misses
- dTLB-prefetches
- dTLB-prefetch-misses
- iTLB-loads
- iTLB-load-misses
- branch-loads
- branch-load-misses

KernelPMUEvents:

- cpu-cycles
- instructions
- cache-references
- cache-misses
- branches
- branch-misses
- bus-cycles

SWEvents:

- cpu-clock
- task-clock
- context-switches
- cpu-migrations
- page-faults
- minor-faults
- major-faults
- alignment-faults
- emulation-faults

HWSpecificEvents:

Example:

 - "L2_RQSTS.CODE_RD_HIT,L2_RQSTS.CODE_RD_MISS"

Test cases

 
Summary
collectd.conf
Steps
Results
Status
1Verify intel_pmu compilation is skipped and collectd is successfully compiled and installed when jevents is installed without cFPIC flag-

  1. Build jevents without specifying cflag option.

    cd jevents
make
make install
  2. Build collectd without --with-libjevents argument.
    ./build.sh
./configure --enable-syslog --enable-logfile --prefix=/root/orest/install --enable-debug
make

    make install

  3. Make sure that collectd installation is completed and intel_pmu compilation skipped.

Collectd compilation fails and intel_pmu plugin compilation is not skipped.

FAIL

Internal JIRA filed

2Verify intel_pmu compilation is not skipped and collectd is successfully compiled and installed when jevents is installed with cFPIC flag-

  1. Build jevents with cflag option.

    cd jevents 
make CFLAGS+='-fPIC'
make install CFLAGS+='-fPIC'
  2. Build collectd without --with-libjevents argument.
    ./build.sh
./configure --enable-syslog --enable-logfile --prefix=/root/orest/install --enable-debug
make

    make install

  3. Make sure that collectd installation is completed and intel_pmu is compiled.
Collectd installation is completed and intel_pmu is compiled successfully.PASS
3Verify only HWCacheEvents is collected when related field is set to True.
<Plugin intel_pmu>
 HWCacheEvents true
 KernelPMUEvents false
 SWEvents false
</Plugin>

 

  1. Run collectd with write plugin enabled as well.
  2. Wait 2 read intervals.
  3. Stop collectd
  4. Get intel_pmu statistics
  5. Make sure that events related only to HWCacheEvents are collected.
HWCacheEvents are collected.PASS
4Verify only KernelPMUEvents is collected when related field is set to True.
<Plugin intel_pmu>
 HWCacheEvents false
 KernelPMUEvents true
 SWEvents false
</Plugin>
  1. Run collectd with write plugin enabled as well.
  2. Wait 2 read intervals.
  3. Stop collectd
  4. Get intel_pmu statistics
  5. Make sure that events related only to KernelPMUEvents are collected.
KernelPMUEvents are collectd.PASS
5Verify only SWEvents is collected when related field is set to True.
<Plugin intel_pmu>
 HWCacheEvents false
 KernelPMUEvents false
 SWEvents true
</Plugin>
  1. Run collectd with write plugin enabled as well.
  2. Wait 2 read intervals.
  3. Stop collectd
  4. Get intel_pmu statistics
  5. Make sure that events related only to SWEvents are collected.
SWEvents are collected.PASS
6Verify only HWSpecificEvents is collected when related field is set to specific values.
<Plugin intel_pmu>
 HWCacheEvents false
 KernelPMUEvents false
 SWEvents false
 HWSpecificEvents "L2_RQSTS.CODE_RD_HIT"
</Plugin>
  1. Make sure that events are downloaded using pmu-tool event_download.py
  2. Run collectd with write plugin enabled as well.
  3. Wait 2 read intervals.
  4. Stop collectd
  5. Get intel_pmu statistics
  6. Make sure that events related only to HWSpecificEvents are collected.

HAA-1820

PASS
7Verify that all metrics are collected when all fields have enabled flags.
<Plugin intel_pmu>
 HWCacheEvents true
 KernelPMUEvents true
 SWEvents true
 HWSpecificEvents "L2_RQSTS.CODE_RD_HIT,L2_RQSTS.CODE_RD_MISS"
</Plugin>
  1. Make sure that events are downloaded using pmu-tool event_download.py
  2. Run collectd with write plugin enabled as well.
  3. Wait 2 read intervals.
  4. Stop collectd
  5. Get intel_pmu statistics
  6. Make sure that events related to HWCacheEvents , KernelPMUEvents, SWEvents, HWSpecificEvents are collected.

HAA-1820

PASS
8

Verify number of counters for HWCacheEvents matches number of events defined in OPNFV metrics page.

 

<Plugin intel_pmu>
 HWCacheEvents true
 KernelPMUEvents false
 SWEvents false
 HWSpecificEvents ""
</Plugin>
  1. Run collectd with write plugin enabled as well.
  2. Wait 2 read intervals.
  3. Stop collectd
  4. Get intel_pmu statistics
  5. Verify number of counters for HWCacheEvents matches number of events defined in OPNFV metrics page.

Number of counters for HWCacheEvents matches number of events defined in OPNFV metrics page.

 

PASS
9Verify number of counters for KernelPMUEvents matches number of events defined in OPNFV metrics page.
<Plugin intel_pmu>
 HWCacheEvents false
 KernelPMUEvents true
 SWEvents false
 HWSpecificEvents ""
</Plugin>
  1. Run collectd with write plugin enabled as well.
  2. Wait 2 read intervals.
  3. Stop collectd
  4. Get intel_pmu statistics
  5. Verify number of counters for KernelPMUEvents matches number of events defined in OPNFV metrics page.
Number of counters for KernelPMUEvents matches number of events defined in OPNFV metrics page.PASS
10Verify number of counters for SWEvents matches number of events defined in OPNFV metrics page.
<Plugin intel_pmu>
 HWCacheEvents false
 KernelPMUEvents false
 SWEvents true
 HWSpecificEvents ""
</Plugin>
  1. Run collectd with write plugin enabled as well.
  2. Wait 2 read intervals.
  3. Stop collectd
  4. Get intel_pmu statistics
  5. Verify number of counters for SWEvents matches number of events defined in OPNFV metrics page.
Number of counters for SWEvents matches number of events defined in OPNFV metrics page.PASS
11Verify that HWCacheEvents counters are incremented.
<Plugin intel_pmu>
 HWCacheEvents true
 KernelPMUEvents false
 SWEvents false
 HWSpecificEvents ""
</Plugin>
  1. Run collectd with write plugin enabled as well.
  2. Wait 10 read intervals.
  3. Stop collectd
  4. Get intel_pmu statistics
  5. Verify counters related to HWCacheEvents are incremented.
Internal JIRA FiledFAIL
12Verify that KernelPMUEvents counters are incremented.
<Plugin intel_pmu>
 HWCacheEvents false
 KernelPMUEvents true
 SWEvents true
 HWSpecificEvents ""
</Plugin>
  1. Run collectd with write plugin enabled as well.
  2. Wait 10 read intervals.
  3. Stop collectd
  4. Get intel_pmu statistics
  5. Verify counters related to KernelPMUEvents are incremented.
 PASS
13Verify that SWEvents counters are incremented.
<Plugin intel_pmu>
 HWCacheEvents false
 KernelPMUEvents false
 SWEvents true
 HWSpecificEvents ""
</Plugin>
  1. Run collectd with write plugin enabled as well.
  2. Wait 10 read intervals.
  3. Stop collectd
  4. Get intel_pmu statistics
  5. Verify counters related to SWEvents are incremented.
 PASS
14Verify that HWSpecificEvents counters are incremented.
<Plugin intel_pmu>
 HWCacheEvents false
 KernelPMUEvents false
 SWEvents false
 HWSpecificEvents "L2_RQSTS.CODE_RD_HIT,L2_RQSTS.CODE_RD_MISS"
</Plugin>
  1. Run collectd with write plugin enabled as well.
  2. Wait 10 read intervals.
  3. Stop collectd
  4. Get intel_pmu statistics
  5. Verify counters related to HWSpecificEvents are incremented.
 PASS
15Verify values reported for HWCacheEvents are in range of actual values reported by perf tool.
<Plugin intel_pmu>
 HWCacheEvents true
 KernelPMUEvents false
 SWEvents false
 HWSpecificEvents ""
</Plugin>
  1. Run collectd and perf tool for HWCacheEvents simultaneously.(interval is set to 1 second)
  2. Wait 20 seconds.
  3. Stop collectd.
  4. Get last 5 values of HWCacheEvents and perf data.
  5. Verify perf values are in range of collectd values.
 PASS
16Verify values reported for KernelPMUEvents are in range of actual values reported by perf tool.
<Plugin intel_pmu>
 HWCacheEvents false
 KernelPMUEvents true
 SWEvents false
 HWSpecificEvents ""
</Plugin>
  1. Run collectd and perf tool for KernelPMUEvents simultaneously.(interval is set to 1 second)
    perf stat -A -C 1,5,20 -e <events> sleep 20
  2. Wait 20 seconds.
  3. Stop collectd.
  4. Get last 5 values of KernelPMUEvents and perf data.
  5. Verify perf values are in range of collectd values.
 PASS
17Verify values reported for SWEvents are in range of actual values reported by perf tool
<Plugin intel_pmu>
 HWCacheEvents false
 KernelPMUEvents false
 SWEvents true
 HWSpecificEvents ""
</Plugin>
  1. Run collectd and perf tool for SWEvents simultaneously.(interval is set to 1 second)
    perf stat -A -C 1,5,20 -e <events> sleep 20
  2. Wait 20 seconds.
  3. Stop collectd.
  4. Get last 5 values of SWEvents and perf data.
  5. Verify perf values are in range of collectd values.
 PASS
18Verify values reported for HWSpecificEvents are in range of actual values reported by perf tool
<Plugin intel_pmu>
 HWCacheEvents false
 KernelPMUEvents false
 SWEvents false
 HWSpecificEvents "L2_RQSTS.CODE_RD_HIT,L2_RQSTS.CODE_RD_MISS"
</Plugin>
  1. Run collectd and perf tool for HWSpecificEvents simultaneously.(interval is set to 1 second)
    perf stat -A -C 1,5,20 -e <events> sleep 20
  2. Wait 20 seconds.
  3. Stop collectd.
  4. Get last 5 values of HWSpecificEvents and perf data.
  5. Verify perf values are in range of collectd values.
Internal JIRA FiledFAIL
19Verify intel_pmu counters are reported per each core and per all cores
<Plugin intel_pmu>
 HWCacheEvents {}
 KernelPMUEvents {}
 SWEvents {}
 HWSpecificEvents {}
</Plugin>
  1. Set HWCacheEvents to true in collectd.conf
  2. Wait 10 seconds.
  3. Stop collectd.
  4. Verify that values are reported per core and for all cores.
  5. Repeat 1-4 steps for KernelPMUEvents, SWEvents, HWSpecificEvents.
 PASS
20Verify intel_pmu counters are not reported for unsupported events per core and per all cores.
<Plugin intel_pmu>
 HWCacheEvents true
 KernelPMUEvents false
 SWEvents false
 HWSpecificEvents ""
</Plugin>
  1. Set HWCacheEvents to true in collectd.conf
  2. Make sure that perf list doesn't have some event that is in the list of collectd events.
  3. Wait 10 seconds.
  4. Stop collectd.
  5. Verify that unsupported events are not collected.

Unsupported events are collected per all cores.

 

PASS

 

21Config-less testing - invalid HWCacheEvents field
<Plugin intel_pmu>
 HWCacheEvents incorrect
 KernelPMUEvents true
 SWEvents false
 HWSpecificEvents ""
</Plugin>

 

 

Values to test =

['None', '', '1', 'incorrect']
  1. Set HWCacheEvents to incorrect value in collectd.conf
  2. Start collectd
  3. Verify collectd is not started.
  PASS
22Config-less testing - invalid KernelPMUEventsfield
<Plugin intel_pmu>
 HWCacheEvents true
 KernelPMUEvents invalid_value
 SWEvents false
 HWSpecificEvents ""
</Plugin>
 

Values to test =

['None', '', '1', 'incorrect']
  1. Set KernelPMUEvents to incorrect value in collectd.conf
  2. Start collectd
  3. Verify collectd is not started.
 PASS
23Config-less testing - invalid SWEventsfield
<Plugin intel_pmu>
 HWCacheEvents true
 KernelPMUEvents false
 SWEvents invalid_value
 HWSpecificEvents ""
</Plugin>
 

Values to test =

['None', '', '1', 'incorrect']
  1. Set SWEvents to incorrect value in collectd.conf
  2. Start collectd
  3. Verify collectd is not started.
 PASS
24Config-less testing - invalid HWSpecificEventsfield
<Plugin intel_pmu>
 HWCacheEvents true
 KernelPMUEvents false
 SWEvents false
 HWSpecificEvents 
</Plugin>

Values to test =

['']
  1. Set HWSpecificEvents to incorrect value in collectd.conf
  2. Start collectd
  3. Verify collectd is not started.
 PASS
25Config-less testing - invalid field name
<Plugin intel_pmu>
 HWCacheEvents true
 KernelPMUEvents false
 SWEvents false
 HWSpecificEvents ""
 IncorrectField true
</Plugin>

Values to test =

['None', '', '1', 'true']
  1. Set invalid field name in collectd.conf
  2. Start collectd
  3. Verify collectd is not started.
 PASS