Table of Contents
OPNFV Release D – MS2 – Functest Plan for the Barometer Project – version 1.0
OPNFV Release E (Euphrates)
Functest Test Case E (Euphrates)
The intention of the Barometer test in the OPNFV E release is simply to extend the test suite that was developed in the D release with new tests for the following features:
Theme: OPNFV Platform Equivalence | Test Jira |
|---|---|
RDT Support | |
Platform Legacy Support (IPMI) | |
RAS Support | |
PMU Support | |
Libvirt Support | |
vSwitch Health | |
PCIe Error reporting | |
SNMP write plugin | |
Aodh plugin | |
Gnocchi Support | |
ETSI TST008 Alignment | |
| clock speed and power state (CPU + platform) | |
| Plugin rename to barometer as part of Apex Support | |
| Integration with Apex | Covered by all the other testing JIRAs |
| Euphrates Stretch Goals | |
| collectd agent extensions | |
Host Health Support | |
Functest Test Case Extension Proposal
Project: Barometer (formerly known as SFQM – Software Fastpath Service Quality Metrics)
Authors/Contributors: Calin Gherghe (Intel Corp)
Test Case Name: BarometerCollectd
Test Type: External (i.e. test code will be kept in the project repository)
Test Tier: features
Test Constrains:
...
os-nosdn-bar-noha
os-nosdn-bar-ha
os-nosdn-kvm_ovs_dpdk_bar-noha
os-nosdn-kvm_ovs_dpdk_bar-ha
Installer(s): Apex
Test Scope:
The test is intended to validate individually the collectd plugins which are part of the Barometer project, therefore is constituted as a test-suite.
...
The following files are proposed to be patched:
1. Dockerfile (/home/opnfv/repos/functest/docker/Dockerfile). Add the following line in order
to clone the test code from the external repository into the Functest container:
RUN git clone https://gerrit.opnfv.org/gerrit/BarometerCollectd
${repos_dir}/BarometerCollectd
2. exec_test.sh (/home/opnfv/repos/functest/ci/exec_test.sh). Add the following to enable
running the test in CI or CLI:
"BarometerCollectd")
python ${FUNCTEST_REPO_DIR}/testcases/features/BarometerCollectd.py
;;
3. testcases.yaml (/home/opnfv/repos/functest/ci/testcases.yaml). Add the following block to
declare the test and its constrains:
name: BarometerCollectd
criteria: 'status == "PASS"'
blocking: false
description: >-
Test suite for the Barometer project. Separate tests verify the proper configuration and
functionality of the following collectd plugins: Ceilometer, Intel RDT, Hugepages, Memory
RAS, OVS Stats and Events, BIOSand the plugins mentioned above
dependencies:
installer: 'fuel'
scenario: ' os-nosdn-kvm_ovs-noha, os-nosdn-kvm_ovs_dpdk-noha, os-nosdn-kvm_ovsha, os-nosdn-kvm_ovs_dpdk-ha’
4. config_functest.yaml (/home/opnfv/repos/functest/ci/config_functest.yaml). Add the
following line to define the repository absolute path:
dir_repo_BarometerCollectd: /home/opnfv/repos/BarometerCollectdApex'
2. Updating the Test cases in the barometer repo to test the new plugins: [barometer.git] / baro_tests /
High Level Testing Procedure:
For Each Compute Node
...
Get configuration (Parse collectd.conf or plugin-specific conf file)
...
For Each Output Plugin
...
Verify plugin is enabled by default
Verify (sample) metrics (and/or events) are sent
Wait Interval
Verify metrics (and/or events) timestamps are updated
Log Result
For Each Input Plugin
Use Output Plugin if PASS, otherwise use CSV (local files)
Verify plugin is enabled by default
Verify (if any) dependent modules are loaded
Induce events (as applicable)
Verify ALL corresponding metrics and/or events are dispatched
Wait Interval
Induce events (as applicable)
Verify ALL metrics and/or events timestamps are updated
Log Result
Done
Done
Done
Testing Assumptions (Same assumptions for the D release):
...
- The metrics’ actual values are NOT validated for correctness (at least in this release). Only the
...
- metrics list and
...
- time stamps are verified.
...
- If the Output plugin(s) verification fails, the CSV plugin will be enabled and (collectd service
...
- restarted) in order to be able to proceed with the Input plugins testing
...
- Log messages will be generated at every
...
- step
...
- Failure of ANY plugin test would generate an Overall FAIL Result (unless a success_rate criteria is
...
- adopted)
...
- Any change in configuration (such as enabling the CSV plugin if necessary) would be restored
...
- upon the tests’ exit.
...
- Testing will exit when all loops are completed once. No additional runtime is allotted for
...
- checking potential stress failures (such as monitoring over longer time for: memory leaks,
...
- traffic congestion, latency issues, disk space, database chocking, etc)
OPNFV Release D (Danube)
MS2: Detailed test case descriptions communicated to test project teams
Due: 11/22/2016
Detailed Milestone Description (per OPNFV Danube Wiki): Each feature project team will complete a
test plan, describing the test cases that they intend to use to validate their project, including pass/fail
criteria, and will share this plan with the test working group and other team members and solicit
feedback. The test plan will also describe dependencies on any new test capability that is not currently
available through existing test frameworks and how these dependencies will be satisfied. Project teams
are encouraged to attend the Test Working Group meeting, or to setup separate meetings to discuss
requirements
New Functest Test Case Proposal
Project: Barometer (formerly known as SFQM – Software Fastpath Service Quality Metrics)
Authors/Contributors: Calin Gherghe (Intel Corp)
Test Case Name: BarometerCollectd
Test Type: External (i.e. test code will be kept in the project repository)
Test Tier: features
Test Constrains:
...