...

• config: basiclly the same env config files as the config files in functest/yardstick, used to indicate SUT information
• testcases: testcase definition files, mapping to the testcase from functest/yardstick being executed. these testcases comprise the certification
• certification: certification definition files, to determine the scenario of the SUT, and a collection of testcases.
• parser: parse, load and check input files(config,certification,testcases) if they are valid.
• downloader: get functest/yardstick repos if they are not existed.
• runner: execute testcases from functest/yardstick.
• report: generate reports from test results.
• dashboard: presentation of test report and show the details of each testcase.

.

Definitions

certification scenario definition

• certification_basic: scenario name
• testcase_list: a list of test case referenced to test case defined in dovetail

Image Added

testcase definition

• dovetail.feature.tc003:  name of the test case,  comprised of 3 parts, the dovetail project, belongs to which feature, and the serial number
• description: a brief description of the test case, explains the objective of this test case
• scripts: scripts and configurations for the execution of the test case
  • type: upstream testing project
  • testcase: the input parameter for the execution of the upstream testing scripts
  • sub_testcase_list: a list of real working test, indicated what's inside of this test case

Image Added

Testcase report analysis

As our certification reports come from the report of test cases, we have to check those test cases one by one. And the most important data is the test case details. Test case details include its duration, test case names , etc. So i make a table to show the details of testcases.

...

yardstick report analysis: