...
Test cases from upstream projects will be validated and work in conjunction with each other as an organic whole. To achieve this goal, we need bunch of scripts to communicate with these test cases. The scripts can execute the test cases under certain circumstances, and then analysis variety results to make a human-friendly report .
Component | Description | Status |
config | basiclly the same env config as the config in functest/yardstick, used to indicate SUT information | Done |
Testcase file | testcase definition files, mapping to the testcase from functest/yardstick being executed. these testcases comprise the certification | Done |
testsuite | testsuite definition files, to determine the scenario of the SUT, and a collection of testcases | Done, basic definition |
parser | parse, load and check input files(config,certification,testcases) if they are valid | Done, basic function |
Testcase keeper | Testcase management, execution status , etc | Done |
| Test Runner | different runner for different test case type. | Done |
Container manager | all stuff around the container, like pull image, start container and exec cmd in container | Done |
runner | main entry for running through the whole validating procedure | Done |
report | manage test results and generate reports from these results | Done |
dashboard | presentation of test report and show the details of each testcase | Optional |
.
Dovetail Test Definition Files
Dovetail uses two levels of definition files to specify what exact test cases are to be executed in a given test run or test suite specification.
Test Suite Definition File
This file defines a suite that is comprised of test cases which may be a partial list of all existing test cases. It gives the suite a name that is more convenient for reference.
- certification_basic: test suite name
- testcase_list: a list of test case referenced to be in the named test suite
Test Case Definition File
- dovetail.feature.tc003: name of the test case, comprised of 3 parts, the dovetail project, belongs to which feature, and the serial number
- objective: a brief description of the test case, explains the objective of this test case
- scripts: scripts and configurations for the execution of the test case
- type: upstream testing project
- testcase: the input parameter for the execution of the upstream testing scripts
- sub_testcase_list: a list of real working test, indicated what's inside of this test case
This diagram will present the dependencies between each module in dovetail tool:
| Gliffy | ||||
|---|---|---|---|---|
|
.