Versions Compared

Version Old Version 65 New Version 66
Changes made by

Dan Xu

Dan Xu

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

 

this wiki page is used to record the running histroy for the dovetil tool.this may cover the runnning in the plugfest, CI and vendors lab.

...

opnfv/dovetail:cvp.0.2.0 (commit ID 5ddf932dc28bcc47169c3091267d57ba5a99f9b2)

opnfv/yardstick:danube.3.0

opnfv/functest:cvp.0.2.0 (commit ID 3d03bbcfc45d00d4ce995d8aabff5808accb0687)

opnfv/testapi:cvp.0.2.0 (commit ID a7f82c093fab2ad19c7ffb0a81ec5756e91e73ae)

...

ImageUpdates and problems found and fixedJIRAGerrit
opnfv/dovetail:cvp.0.2.0
  1. Update IPv6 tc018-tc025: give more details and modify some misunderstanding about these test cases
  2. Delete the ERROR log when it is not https
  3. Adapt Yardstick to copy yardstick.conf to /etc/yardstick
  4. Fix the docker version to be 17.03.0-ce rather than the latest one
  1. DOVETAIL-461
  2. DOVETAIL-462
  3. DOVETAIL-466
  4. DOVETAIL-467
  1. https://gerrit.opnfv.org/gerrit/#/c/37229/
  2. https://gerrit.opnfv.org/gerrit/#/c/37449/
  3. https://gerrit.opnfv.org/gerrit/#/c/37737/
  4. https://gerrit.opnfv.org/gerrit/#/c/37813/
opnfv/yardstick:danube.3.0
  1. Support ignoring server certificate
  2. Add debug log for recovering service failure
  3. Improve method to kill keystone process
None
  1. https://gerrit.opnfv.org/gerrit/#/c/36377/
opnfv/functest:cvp.0.2.0NoneNoneNone
opnfv/testapi:cvp.0.2.0
  1. role based access control and result upload
  2. testapi tox bugfix
  3. some bugfix
  1. None
  2. None
  3. RELENG-281 
    RELENG-212 
    FUNCTEST-694 
    RELENG-213  
  1. https://gerrit.opnfv.org/gerrit/#/c/37053/
    https://gerrit.opnfv.org/gerrit/#/c/37839/
  2. https://gerrit.opnfv.org/gerrit/#/c/37459/
  3. https://gerrit.opnfv.org/gerrit/#/c/37765/
    https://gerrit.opnfv.org/gerrit/#/c/37723/
    https://gerrit.opnfv.org/gerrit/#/c/37711/

...

1.3 cvp.0.3.0 (2017/7/21)

1.3.1 Docker images used:

opnfv/dovetail:cvp.0.3.0 (commit ID 7ed1cf189676693d84c9d48bb964c178b16bae76)

opnfv/yardstick:danube.3.1

opnfv/functest:cvp.0.2.0 (commit ID 3d03bbcfc45d00d4ce995d8aabff5808accb0687)

opnfv/testapi:cvp.0.3.0 (commit ID 4e0b74b0141e3ae5d250006f3d75fcc880738cc9)

1.3.2 Updates and problems found and fixed in this version

ImageUpdates and problems found and fixedJIRAGerrit
opnfv/dovetail:cvp.0.3.0
  1. Add docs about vping test cases (test scope and test specification)
  2. Add 2 vping test cases (vping.tc001->vping_userdata, vping.tc002->vping_ssh) into proposed_tests
  3. Bugfix: Build docker image failed because of lacking of packages
  4. Add resiliency.tc001 into proposed_tests
  1. None
  2. DOVETAIL-446
  3. DOVETAIL-475
  4. DOVETAIL-474
  1. https://gerrit.opnfv.org/gerrit/#/c/36253/
  2. https://gerrit.opnfv.org/gerrit/#/c/37853/
  3. https://gerrit.opnfv.org/gerrit/#/c/38573/
  4. https://gerrit.opnfv.org/gerrit/#/c/38713/
opnfv/yardstick:danube.3.1NoneNoneNone
opnfv/functest:cvp.0.2.0NoneNoneNone
opnfv/testapi:cvp.0.3.0
  1. Add Linux Foundation ID as account provider
  2. Upload multi results in one time
  3. Submit results to CVP to review
  4. Results are grouped by test id
  5. Add details for each test
NoneNone

 

1.4 cvp.0.4.0 (2017/8/4)

1.4.1 Docker images used:

opnfv/dovetail:cvp.0.4.0 (commit ID a8d7665ca783684bc10d7c3dbbc606e30f083bc7)

opnfv/yardstick:danube.3.1

opnfv/functest:cvp.0.2.0 (commit ID 3d03bbcfc45d00d4ce995d8aabff5808accb0687)

opnfv/testapi:cvp.0.4.0

opnfv/bottlenecks:cvp.0.4.0 (commit ID 3592f9016821a603930ebda7f4112abf732e6750)

1.4.2 Updates and problems found and fixed in this version

ImageUpdates and problems found and fixedJIRAGerrit
opnfv/dovetail:cvp.0.4.0
  1. Bugfix: wrong command in function 'check_https_enabled'
  2. Use opnfv/functest:cvp.0.5.0 and do some adaptions in dovetail tool
  3. Create a tar file for all logs
  4. Add tempest test cases into proposed_tests
  5. Fix testapi base_url for public_url
  6. Add user guide for results operations on web
  7. Add nginx to host static files
  1. DOVETAIL-482
  2. DOVETAIL-480
  3. DOVETAIL-479
  4. DOVETAIL-478
  5. None
  6. DOVETAIL-471
  7. DOVETAIL-485
  1. https://gerrit.opnfv.org/gerrit/#/c/39393/
  2. https://gerrit.opnfv.org/gerrit/#/c/39221/
  3. https://gerrit.opnfv.org/gerrit/#/c/38891/
  4. https://gerrit.opnfv.org/gerrit/#/c/38835/
  5. https://gerrit.opnfv.org/gerrit/#/c/38823/
  6. https://gerrit.opnfv.org/gerrit/#/c/38065/
  7. https://gerrit.opnfv.org/gerrit/#/c/39489/
opnfv/yardstick:danube.3.1Bugfix: cannot run task if without yardstick.conf in danubeYARDSTICK-776https://gerrit.opnfv.org/gerrit/#/c/38955/
opnfv/functest:cvp.0.2.0
  1. Remove functest repo and only python module have been installed
  2. Functest Docker image will not host any OS images
  3. Add a dedicated file to configure extra params in tempest.conf

  1. FUNCTEST-836

  2. None

  3. FUNCTEST-860

  1. https://gerrit.opnfv.org/gerrit/#/c/35813/
    https://gerrit.opnfv.org/gerrit/#/c/36163/
  2. https://gerrit.opnfv.org/gerrit/#/c/37373/
  3. https://gerrit.opnfv.org/gerrit/#/c/39019/
opnfv/testapi:cvp.0.4.0
  1. log browsing in directory style
  
opnfv/bottlenecks:cvp.0.4.0   

 

1.5 cvp.0.5.0 (2017/8/18)

1.5.1 Docker images used:

opnfv/dovetail:cvp.0.5.0 (commit ID 474795c11c556dcd325559853fe7ed436a69418c)

opnfv/yardstick:danube.3.2 (commit ID e9338f29a70a38a5838498adf79ec91f92d085d1)

opnfv/functest:cvp.0.5.0  (commit ID 13067de652d6ccdcfc85f0f5e65841b644d9a6e9)

opnfv/bottlenecks:cvp.0.4.0 (commit ID 3592f9016821a603930ebda7f4112abf732e6750)

opnfv/testapi:cvp.0.5.0 

opnfv/dovetail:nginx.cvp.0.5.0

1.5.2 Updates and problems found and fixed in this version

ImageUpdates and problems found and fixedJIRAGerrit
opnfv/dovetail:cvp.0.5.0
  1. Merge addendum doc and workflow doc
  2. Update user guide docs
  3. Remake aarch64 patch for Dovetail container
  4. Remove some tempest test cases that are not too common
  5. Add additional Tempest API test cases into test case tempest.tc005
  6. persist db file in host for mongo container
  7. add nginx to host static files
  8. add a sample tempest_conf.yaml file into userconfig
  9. Add a patch to set ha SLA to be 30 seconds
  10. Modify logs
  11. Bugfix: the config files in useeconfig will cover user's config
  12. Bugfix: functest only keeps the last test case's log
  13. Bugfix: pre_condition cmds is just executed once
  14. Bugfix: No cirros-0.3.5-x86_64-disk.img
  15. Merge all test specification docs about test area tempest and defcore
  1. None
  2. DOVETAIL-493
  3. ARMBAND-286
  4. None
  5. DOVETAIL-495
  6. DOVETAIL-488
  7. DOVETAIL-485
  8. DOVETAIL-484
  9. DOVETAIL-496
  10. DOVETAIL-498
  11. DOVETAIL-492
  12. DOVETAIL-490
  13. DOVETAIL-489
  14. DOVETAIL-487
  15. DOVETAIL-431
    DOVETAIL-473

  1. https://gerrit.opnfv.org/gerrit/#/c/35161/
    https://gerrit.opnfv.org/gerrit/#/c/34643/
  2. https://gerrit.opnfv.org/gerrit/#/c/40369/
  3. https://gerrit.opnfv.org/gerrit/#/c/39073/
  4. https://gerrit.opnfv.org/gerrit/#/c/40963/
  5. https://gerrit.opnfv.org/gerrit/#/c/40745/
  6. https://gerrit.opnfv.org/gerrit/#/c/39851/
  7. https://gerrit.opnfv.org/gerrit/#/c/39489/
  8. https://gerrit.opnfv.org/gerrit/#/c/39463/
  9. https://gerrit.opnfv.org/gerrit/#/c/41251/
  10. https://gerrit.opnfv.org/gerrit/#/c/41285/
  11. https://gerrit.opnfv.org/gerrit/#/c/40733/
  12. https://gerrit.opnfv.org/gerrit/#/c/40051/
  13. https://gerrit.opnfv.org/gerrit/#/c/39925/
  14. https://gerrit.opnfv.org/gerrit/#/c/39719/
  15. https://gerrit.opnfv.org/gerrit/#/c/34649/
    https://gerrit.opnfv.org/gerrit/#/c/39389/
    https://gerrit.opnfv.org/gerrit/#/c/37989/
    https://gerrit.opnfv.org/gerrit/#/c/39459/
    https://gerrit.opnfv.org/gerrit/#/c/39585/
    https://gerrit.opnfv.org/gerrit/#/c/39223/
    https://gerrit.opnfv.org/gerrit/#/c/39229/
    https://gerrit.opnfv.org/gerrit/#/c/39227/
    https://gerrit.opnfv.org/gerrit/#/c/38379/
    https://gerrit.opnfv.org/gerrit/#/c/39225/

 

 

opnfv/yardstick:danube.3.2NoneNoneNone
opnfv/functest:cvp.0.5.0NoneNoneNone
opnfv/testapi:cvp.0.5.0   
opnfv/dovetail:nginx.cvp.0.5.0   
opnfv/bottlenecks:cvp.0.4.0   

1.6 cvp.0.6.0 (2017/9/8)

1.6.1 Docker images used:

opnfv/dovetail:cvp.0.6.0 (commit ID 0aacdceeeac62333651ec5440fb32af965c7689e)

opnfv/yardstick:danube.3.2 (commit ID e9338f29a70a38a5838498adf79ec91f92d085d1)

opnfv/functest:cvp.0.5.0  (commit ID 13067de652d6ccdcfc85f0f5e65841b644d9a6e9)

opnfv/bottlenecks:cvp.0.4.0 (commit ID 3592f9016821a603930ebda7f4112abf732e6750)

opnfv/testapi:cvp.0.6.0 

opnfv/dovetail:web.cvp.0.6.0

opnfv/dovetail:api.cvp.0.6.0

 

...

3. Running against scenarios by owners

 

-kvmha    
scenariosinstallerownerscvp.0.1.0cvp.0.2.0cvp.0.3.0cvp.0.4.0cvp.0.5.0cvp.0.6.0
Apex Scenarios Currently Test Against
os-nosdn-nofeature-ha   Apex     

https://build.opnfv.org/ci/view/dovetail/job/dovetail-apex-huawei-pod4-proposed_tests-danube/4/console

PASS/total

defcore: 204/205

HA: 7/8

vping: 2/2

ipv6: 25/25

tempest: 4/6

SDNVPN: 0/4 no SDN feature

 
Compass Scenarios Currently Test Against
os-nosdn-Apex       
 os-odl_l3-fdio-ha  Apex       
os-odl_l2-nofeature-ha Compass  

https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetalvirtual-proposed_tests-danube/163176/consoleFull

PASS/total

defcore: 204/205

HA: 87/8 tc007, SLA not passed

ipv6: 25/25

SDNVPN: 0/4 no SDN feature

https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/191157/consoleFull

PASS/total

defcore: 204/205

HA: 8/8

ipv6: 1725/25

SDNVPN: 0/4 no SDN feature

https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtualbaremetal-proposed_tests-danube/309196/consoleFull

PASS/total

defcore: 204/205

HA: 78/8vping: 2/2

ipv6: 25/25

SDNVPN: 0/4 no SDN feature

os-nosdn-nofeature-ha Compass 

resiliency: 1/1

https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetalvirtual-proposed_tests-danube/304335/consoleFull

PASS/total

defcore: 205204/205

HA: 8/8

vping: 2/2

ipv6: 2425/25

tempest: 2/6

SDNVPN: 0/4 no SDN feature

 

resiliency: 1/1

https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtualbaremetal-proposed_tests-danube/176288/consoleFull

PASS/total

defcore: 204205/205

HA: 78/8 tc007, SLA not passed

vping: 2/2

ipv6: 25/25SDNVPN

: tempest: 3/6

SDNVPN: 0/4 no SDN feature

https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/157331/consoleFull

PASS/total

defcore: 204205/205

HA: 8/8

vping: 2/2

ipv6: 25/25

tempest: 5/5

SDNVPN: 0/4 no SDN feature

os-odl_l2-nofeature-ha Compass  

https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/196163/consoleFull

PASS/total

defcore: 204/205

HA: 8/8

ipv6: 25/25

SDNVPN: 0/4 no SDN feature

https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtualbaremetal-proposed_tests-danube/335191/consoleFull

PASS/total

defcore: 204/205

HA: 8/8

vping: 2/2

ipv6: 2517/25

SDNVPN: 0/4 no SDN feature

resiliency: 1/1

https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetalvirtual-proposed_tests-danube/288309/consoleFull

PASS/total

defcore: 205204/205

HA: 87/8

vping: 2/2

ipv6: 25/25tempest: 3/6

SDNVPN: 0/4 no SDN feature

 os-onos-nofeature-ha Compass 

https://resiliency: 1/1

https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtualbaremetal-proposed_tests-danube/177304/consoleFull

PASS/total

defcore: 150205/171205

HA: 78/8 tc007, SLA not passed

vping: 2/2

ipv6: 1724/25 tc018-tc025 fails

tempest: 2/6

SDNVPN: 0/4 no SDN feature

 
os-odl_l3-nofeature-ha Compass 

https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetalvirtual-proposed_tests-danube/160178/consoleFull

PASS/total

defcore: 198204/205

HA: 87/8 tc007, SLA not passed

ipv6: 17/25 tc018-tc025 fails

SDNVPN: 0/4 no SDN feature

https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/195159/consoleFull

PASS/total

defcore: 198204/205

HA: 8/8

ipv6: 17/25 tc018-tc025 fails

SDNVPN: 0/4 no SDN feature

https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtualbaremetal-proposed_tests-danube/307198/consoleFull

PASS/total

defcore: 198204/205

HA: 78/8

vping: 2/2

ipv6: 17/25 tc018-tc025 fails

SDNVPN: 0/4 no SDN feature

os-odl_l3-nofeature-ha Compass 

resiliency: 0/1

https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/312243/consoleFull

PASS/total

defcore: 198204/205

HA: 58/8

vping: 2/2

ipv6: 17/25 tc018-tc025 fails

tempest: 1/6

SDNVPN: 0/4 no SDN feature

 

resiliency: 0/1

https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtualbaremetal-proposed_tests-danube/178311/consoleFull

PASS/total

defcore: 204205/205

HA: 78/8 tc007, SLA not passed

vping: 2/2

ipv6: 17/25 tc018-tc025 fails

tempest: 2/6

SDNVPN: 0/4 no SDN featurehttps://build

 
Fuel Scenarios Currently Test Against
os-nosdn-nofeature-ha

Fuel

(zte-pod1)

      

https://build.opnfv.org/ci/view/dovetail/job/dovetail-fuel-compasszte-baremetalpod1-proposed_tests-danube/1592/consoleFull

PASS/total

defcore: 204205/205

HA: 8/8

vping: 2/2

ipv6: 1725/25 tc018-tc025 fails

tempest: 4/5

SDNVPN: 0/4 no SDN feature

os-odl_l2-bgpvpn-ha Fuel 

https://build.opnfv.org/ci/view/dovetail/job/dovetail-compassfuel-baremetal-proposed_tests-danube/19845/consoleFull

PASS/total

defcore: 204205/205

HA: 8/8

ipv6: 17/25 tc018-tc025 fails

SDNVPN: 03/4 no SDN featuresdnvpn.tc001 "

https://build.opnfv.org/ci/view/dovetail/job/dovetail-compassfuel-baremetal-proposed_tests-danube/24352/consoleFullconsole

PASS/total

defcore: 204205/205

HA: 8/8vping: 2/2

ipv6: 17/25 tc018-tc025 fails

SDNVPN: 0/4 no SDN featureresiliency: 0/14/4

https://build.opnfv.org/ci/view/dovetail/job/dovetail-fuel-baremetal-proposed_tests-danube/71/consoleFull

PASS/total

defcore: 205/205

HA: 8/8

ipv6: 17/25 tc018-tc025 fails

SDNVPN: 4/4

   
Other Scenarios
os-nosdn-kvm-ha    Apex       
os-odl_l3-fdio-ha  Apex       

os-onos-nofeature-ha 

Don't test against this scenario since Compass will not release this scenario anymore.

Compass 

https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetalvirtual-proposed_tests-danube/311177/consoleFull

PASS/total

defcore: 205150/205171

HA: 87/8

vping: 2/2

tc007, SLA not passed

ipv6: 17/25 tc018-tc025 fails

tempest: 2/6

SDNVPN: 0/4 no SDN feature

 os-odl_l2-bgpvpn-ha Fuel 

https://build.opnfv.org/ci/view/dovetail/job/dovetail-fuelcompass-baremetal-proposed_tests-danube/45160/consoleFull

PASS/total

defcore: 205198/205

HA: 8/8

ipv6: 17/25 tc018-tc025 fails

SDNVPN: 30/4 sdnvpn.tc001 "no SDN feature

https://build.opnfv.org/ci/view/dovetail/job/dovetail-fuelcompass-baremetal-proposed_tests-danube/52195/consoleconsoleFull

PASS/total

defcore: 205198/205

HA: 8/8

ipv6: 17/25 tc018-tc025 fails

SDNVPN: 4/40/4 no SDN feature

https://build.opnfv.org/ci/view/dovetail/job/dovetail-fuelcompass-baremetalvirtual-proposed_tests-danube/71307/consoleFull

PASS/total

defcore: 205198/205

HA: 7/8

vping: 2/82

ipv6: 17/25 tc018-tc025 fails

SDNVPN: 40/4

   os-nosdn-nofeature-haFuel(ZTE-POD1)      

no SDN feature

resiliency: 0/1

https://build.opnfv.org/ci/view/dovetail/job/dovetail-fuelcompass-ztebaremetal-pod1-proposed_tests-danube/2312/consoleFull

PASS/total

defcore: 205198/205

HA: 85/8

vping: 2/2

ipv6: 2517/25 tc018-tc025 fails

tempest: 41/56

SDNVPN: 0/4 no SDN feature

 
os-odl_l2-nofeature-haFuel(ZTE-POD1)       
os-nosdn-ovs-ha Fuel       
os-odl_l3-nofeature-ha Fuel       
os-nosdn-kvm_ovs_dpdk-haFuel       

...