...

Test cases  from upstream projects will be validated and work in conjunction with each other as an organic whole. To achieve this goal,  we need bunch of scripts to communicate with these test cases. The scripts can execute the test cases under certain circumstances, and then analysis variety results to make a human-friendly report . 

Testcase report analysis

 As our certification reports come from the report of test cases, we have to check those test cases one by one. And the most important data is the test case details. Test case details include its duration, test case names , etc. So i make a table to show the differences between test cases.

functest report analysis

...

Test-case	duration	Sub-testcase	Details on success	Details on fail
ping
healthcheck
tempest_smoke_serial	Y	Y	N	Y
vping_ssh	Y	N	N	N
vping_userdata	Y	N	N	N
rally_sanity	Y	Y	N	N
odl	Y	Y	Y	Y
onos	Y	Y	Y	N
moon
multisite
Tempest_full_parallel
Onos_sfc	Y	N	N	N
vims	Y	Y	Y	N

...