This page provides information about the Dovetail Test Tool. The Dovetail Test Tool is designed to leverage existing OPNFV test projects (such as functest and yardstick) to automate the execution of Dovetail Test Areas and Test Cases in a way that satisfies compliance testing requirements, and generate required reports for use by the OPNFV compliance verification program.
Dovetail Test Tool Design
Test cases from upstream projects will be validated and work in conjunction with each other as an organic whole. To achieve this goal, we need bunch of scripts to communicate with these test cases. The scripts can execute the test cases under certain circumstances, and then analysis variety results to make a human-friendly report .
This diagram will present the dependencies between each module in dovetail tool:
.png?version=1&modificationDate=1481860067000&cacheVersion=1&api=v2)
Component | Description | Status |
config files | basiclly the same env config as the config in functest/yardstick, used to indicate SUT information | Done |
Testcase definition | testcase definition files, mapping to the testcase from functest/yardstick being executed. these testcases comprise the certification | Done |
testsuite definition | testsuite definition files, to determine the scenario of the SUT, and a collection of testcases | Done, basic definition |
parser | parse, load and check input files(config,certification,testcases) if they are valid | Done, basic function |
Testcase manager | Testcase management, execution status , etc | Done |
| Test runner | different runner for different test case type. | Done |
container manager | all stuff around the container, like pull image, start container and exec cmd in container | Done |
runner | main entry for running through the whole validating procedure | Done |
report | manage test results and generate reports from these results | Done |
dashboard | presentation of test report and show the details of each testcase | Optional |
.
Dovetail Test Definition Files
Dovetail uses two levels of definition files to specify what exact test cases are to be executed in a given test run or test suite specification.
Test Suite Definition File
This file defines a suite that is comprised of test cases which may be a partial list of all existing test cases. It gives the suite a name that is more convenient for reference.
- compliance_set: test suite name, the example is the test cases used for compliance
- testcase_list: a list of test case referenced to be in the named test suite
Test Case Definition File
- dovetail.ipv6.tc001: name of the test case, comprised of 3 parts, the dovetail project, belongs to which test area, and the serial number
- objective: a brief description of the test case, explains the objective of this test case
- validate: scripts and configurations for the validation of the test case
- type: which method used to validate, support 3 ways: functest/yardstick/shell
- testcase: the input parameter if "type" is one of the upstream testing project(functest/yardstick)
- pre_condition: scripts that should be run before the test
- cmds: how to run the test
- post_condition: scripts that should be run after the test
- sub_testcase_list: use to check if these sub test case passed or not
