Versions Compared

Version Old Version 44 New Version 45
Changes made by

Dan Xu

Dan Xu

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

 

this wiki page is used to record the running histroy for the dovetil tool.this may cover the runnning in the plugfest, CI and vendors lab.

Table of Contents

1. Version change logs

For each dovetail release version, the docker images used are introduced first, then problems found and fixed are listed as well as the updates.

1.1 cvp.0.1.0  (2017/06/23)

1.1.1 Docker images used:

opnfv/dovetail:cvp.0.1.0 (commit ID 041a51d6d3ba5f2d9eabcf3501abb9af30ebc394)

opnfv/yardstick:danube.2.0

opnfv/functest:danube.2.0

opnfv/testapi:cvp.0.1.0 (Commit ID 056b7ccefe38b13e4b4eb11ff7e259e254e35c32)

1.1.2 Updates and problems found and fixed in this version

...

  1. https enabled checking
  2. sdnvpn path fix
  3. log improvement
  4. docker image tag with cvp.0.2.0

...

  1. DOVETAIL-456
  2. DOVETAIL-458
  3. DOVETAIL-450
  4. DOVETAIL-447

 

this wiki page is used to record the running histroy for the dovetil tool.this may cover the runnning in the plugfest, CI and vendors lab.

Table of Contents

1. Version change logs

For each dovetail release version, the docker images used are introduced first, then problems found and fixed are listed as well as the updates.

1.1 cvp.0.1.0  (2017/06/23)

1.1.1 Docker images used:

opnfv/dovetail:cvp.0.1.0 (commit ID 041a51d6d3ba5f2d9eabcf3501abb9af30ebc394)

opnfv/yardstick:danube.2.0

opnfv/functest:danube.2.0

opnfv/testapi:cvp.0.1.0 (Commit ID 056b7ccefe38b13e4b4eb11ff7e259e254e35c32)

1.1.2 Updates and problems found and fixed in this version

ImageUpdates and problems found and fixedJIRAGerrit
opnfv/dovetail:cvp.0.1.0
  1. https enabled checking
  2. sdnvpn path fix
  3. log improvement
  4. docker image tag with cvp.0.2.0
  1. DOVETAIL-456
  2. DOVETAIL-458
  3. DOVETAIL-450
  4. DOVETAIL-447
  1. https://gerrit.opnfv.org/gerrit/#/c/36541/
  2. https://gerrit.opnfv.org/gerrit/#/c/36785/
  3. https://gerrit.opnfv.org/gerrit/#/c/36615/
  4. https://gerrit.opnfv.org/gerrit/#/c/36775/
opnfv/yardstick:danube.2.0
  1. Bugfix: Monitor command in tc019 may not show the real nova-api service status
  2. Bugfix: "monitor_multi" type monitor in HA test case cannot get "max_recover_time"
  3. Improvement: Terminate openstack service's process using "kill" command
  4. Improvement: "monitor_process" type monitor pass criteria
  5. Bugfix: test.dbf file not deleted after execution of Disk I/O Block High Availability test
  1. YARDSTICK-655
  2. YARDSTICK-657
  3. YARDSTICK-659
  4. YARDSTICK-660
  5. YARDSTICK-696
  1. https://gerrit.opnfv.org/gerrit/#/c/34817/
  2. https://gerrit.opnfv.org/gerrit/#/c/35251/
  3. https://gerrit.opnfv.org/gerrit/#/c/36215/
  4. https://gerrit.opnfv.org/gerrit/#/c/35329/
  5. https://gerrit.opnfv.org/gerrit/#/c/36541//36759/
opnfv/functest:danube.2.0 
  1. SDNVPN test cases are not enable in Functest
None

https://gerrit.opnfv.org/gerrit/#/c/

36785

35051/2

https://gerrit.opnfv.org/gerrit/#/c/

36615

36777/3

  • https://gerrit.opnfv.org/gerrit/#/c/36775/
    		• opnfv/yardstick:danube.2.0
    1. Bugfix: Monitor command in tc019 may not show the real nova-api service status
    2. Bugfix: "monitor_multi" type monitor in HA test case cannot get "max_recover_time"
    3. Improvement: Terminate openstack service's process using "kill" command
    4. Improvement: "monitor_process" type monitor pass criteria
    5. Bugfix: test.dbf file not deleted after execution of Disk I/O Block High Availability test
    1. YARDSTICK-655
    2. YARDSTICK-657
    3. YARDSTICK-659
    4. YARDSTICK-660
    5. YARDSTICK-696
  • https://gerrit.opnfv.org/gerrit/#/c/34817/
  • https://gerrit.opnfv.org/gerrit/#/c/35251/
  • https://gerrit.opnfv.org/gerrit/#/c/36215/
    • https://gerrit.opnfv.org/gerrit/#/c/35329/opnfv/testapi:cvp.0.1.0
    1. user management and some bugfix
    		NoneNone


    1.2 cvp.0.2.0 (2017/07/07)

    1.2.1 Docker images used:

    opnfv/dovetail:cvp.0.2.0 (commit ID 5ddf932dc28bcc47169c3091267d57ba5a99f9b2)

    opnfv/yardstick:danube.3.0

    opnfv/functest:cvp.0.2.0 (commit ID 3d03bbcfc45d00d4ce995d8aabff5808accb0687)

    opnfv/testapi:cvp.0.2.0 (commit ID a7f82c093fab2ad19c7ffb0a81ec5756e91e73ae)

    1.2.2 Updates and problems found and fixed in this version

    1.2.1 Docker images used:

    opnfv/dovetail:cvp.0.2.0 (commit ID 5ddf932dc28bcc47169c3091267d57ba5a99f9b2)

    ...

    None
    ImageUpdates and problems found and fixedJIRAGerrit
    opnfv/dovetail:cvp.0.2.0
    1. Update IPv6 tc018-tc025: give more details and modify some misunderstanding about these test cases
    2. Delete the ERROR log when it is not https
    3. Adapt Yardstick to copy yardstick.conf to /etc/yardstick
    4. Fix the docker version to be 17.03.0-ce rather than the latest one
    1. DOVETAIL-461
    2. DOVETAIL-462
    3. DOVETAIL-466
    4. DOVETAIL-467
    1. https://gerrit.opnfv.org/gerrit/#/c/36759/
    		opnfv/functest:danube.2.0 
    1. SDNVPN test cases are not enable in Functest
    1. /#/c/37229/
    2. https://gerrit.opnfv.org/gerrit/#/c/
    35051
    1. 37449/
    2
    2. https://gerrit.opnfv.org/gerrit/#/c/
    36777
    1. 37737/
    3
    opnfv/testapi:cvp.0.1.0
    1. user management and some bugfix
    		  

    ...

    1. https://gerrit.opnfv.org/gerrit/#/c/37813/
    opnfv/yardstick:danube.3.0
    1. Support ignoring server certificate
    2. Add debug log for recovering service failure
    3. Improve method to kill keystone process
    		None
    1. https://gerrit.opnfv.org/gerrit/#/c/36377/
    opnfv/functest:cvp.0.2.0

    ...

    NoneNoneNone
    opnfv/testapi:cvp.0.2.0

    ...

    1.2.2 Updates and problems found and fixed in this version

  • DOVETAIL-461
  • DOVETAIL-462
  • DOVETAIL-466
    • DOVETAIL-467
    ImageUpdates and problems found and fixedJIRAGerrit
    opnfv/dovetail:cvp.0.2.0
    1. Update IPv6 tc018-tc025: give more details and modify some misunderstanding about these test cases
    2. Delete the ERROR log when it is not https
    3. Adapt Yardstick to copy yardstick.conf to /etc/yardstick
    4. Fix the docker version to be 17.03.0-ce rather than the latest one
    1. role based access control and result upload
    2. testapi tox bugfix
    3. some bugfix
    1. None
    2. None
    3. RELENG-281 
      RELENG-212 
      FUNCTEST-694 
      RELENG-213  
    		opnfv/testapi:cvp.0.2.0
    1. role based access control and result upload
    2. testapi tox bugfix
    3. some bugfix
    		 
    1. https://gerrit.opnfv.org/gerrit/#/c/37053/
      https://gerrit.opnfv.org/gerrit/#/c/37839/
    2. https://gerrit.opnfv.org/gerrit/#/c/37459/
    3. https://gerrit.opnfv.org/gerrit/#/c/3722937765/
      https://gerrit.opnfv.org/gerrit/#/c/3744937723/
      https://gerrit.opnfv.org/gerrit/#/c/37737/
    4. https://gerrit.opnfv.org/gerrit/#/c/37813/
    		opnfv/yardstick:danube.3.0
    1. Support ignoring server certificate
    2. Add debug log for recovering service failure
    3. Improve method to kill keystone process
    1. https://gerrit.opnfv.org/gerrit/#/c/36377/
    opnfv/functest:cvp.0.2.0None  
    1. https://gerrit.opnfv.org/gerrit/#/c/37053/37711/

    1.3 cvp.0.3.0 (2017/7/21)

    1.3.1 Docker images used:

    opnfv/dovetail:cvp.0.3.0 (commit ID 7ed1cf189676693d84c9d48bb964c178b16bae76)

    opnfv/yardstick:danube.3.1

    opnfv/functest:cvp.0.2.0 (commit ID 3d03bbcfc45d00d4ce995d8aabff5808accb0687)

    opnfv/testapi:cvp.0.3.0 (commit ID 4e0b74b0141e3ae5d250006f3d75fcc880738cc9)

    1.3.2 Updates and problems found and fixed in this version

    ImageUpdates and problems found and fixedJIRAGerrit
    opnfv/dovetail:cvp.0.3.0
    1. Add docs about vping test cases (test scope and test specification)
    2. Add 2 vping test cases (vping.tc001->vping_userdata, vping_tc002->vping_ssh) into proposed_tests
    3. Bugfix: Build docker image failed because of lacking of packages
    4. Add resiliency.tc001 into proposed_tests
    1. None
    2. DOVETAIL-446
    3. DOVETAIL-475
    4. DOVETAIL-474
    1. https://gerrit.opnfv.org/gerrit/#/c/

    ...

    1. 36253/
    2. https://gerrit.opnfv.org/gerrit/#/c/

    ...

    1. 37853/
    2. https://gerrit.opnfv.org/gerrit/#/c/

    ...

    1. 38573/
    2. https://

    ...

    1.3 cvp.0.3
    opnfv/yardstick:danube.3.1NoneNoneNone
    opnfv/functest:cvp.0.2.0NoneNoneNone
    opnfv/testapi:cvp.0.3.0  

     

    1.4 cvp.0.4.0 (2017/

    7

    8/

    21

    4)

    1.3.1 Docker images used:

    opnfv/dovetail:cvp.0.34.0 (commit ID 7ed1cf189676693d84c9d48bb964c178b16bae76)

    opnfv/yardstick:danube.3.1

    opnfv/functest:cvp.0.2.0 (commit ID 3d03bbcfc45d00d4ce995d8aabff5808accb0687)

    opnfv/testapi:cvp.0.3.0 (commit ID 4e0b74b0141e3ae5d250006f3d75fcc880738cc9)

    1.3.2 Updates and problems found and fixed in this version

    ImageUpdates and problems found and fixedJIRAGerritopnfv/dovetail

    4.0 (commit ID )

    opnfv/bottlenecks:cvp.0.

    3

    4.0

    1. Add docs about vping test cases (test scope and test specification)
    2. Add 2 vping test cases (vping.tc001->vping_userdata, vping_tc002->vping_ssh) into proposed_tests
    3. Bugfix: Build docker image failed because of lacking of packages
    1. None
    2. DOVETAIL-446
    3. DOVETAIL-475
  • https://gerrit.opnfv.org/gerrit/#/c/36253/
  • https://gerrit.opnfv.org/gerrit/#/c/37853/
    • https://gerrit.opnfv.org/gerrit/#/c/38573/

    (commit ID)

    1.3.2 Updates and problems found and fixed in this version

    ImageUpdates and problems found and fixedJIRAGerrit
    opnfv/dovetail:cvp.0.4.0   
    opnfv/yardstick:danube.3.1None   
    opnfv/functest:cvp.0.2.0None   
    opnfv/testapi:cvp.0.3.4.0   
    opnfv/bottlenecks:cvp.0.4.0   

     


    2. Running by CI

    in order to make the dovetail tool  more stable, the dovetail has been integreated in the CI for daily running.

    ...

    scenariosinstallerownerscvp.0.1.0cvp.0.2.0cvp.0.3.0
    os-nosdn-nofeature-ha   Apex    
    os-nosdn-kvm-ha    Apex    
     os-odl_l3-fdio-ha  Apex    
    os-odl_l2-nofeature-ha Compass  

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/163/consoleFull

    PASS/total

    defcore: 204/205

    tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_attach_detach_volume  FAIL

    This can be solved by manually set "volume_device_name" in refstack configuration file

    HA: 8/8

    ipv6: 25/25

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/191/consoleFull

    PASS/total

    defcore: 204/205

    tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_attach_detach_volume  FAIL

    This can be solved by manually set "volume_device_name" in refstack configuration file

    HA: 8/8

    ipv6: 17/25

    SDNVPN: 0/4 no SDN feature

    os-nosdn-nofeature-ha Compass 

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtual-proposed_tests-danube/176/consoleFull

    PASS/total

    defcore: 204/205

    tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_attach_detach_volume  FAIL

    This can be solved by manually set "volume_device_name" in refstack configuration file

    HA: 7/8 tc007, SLA not passed

    ipv6: 25/25

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/157/consoleFull

    PASS/total

    defcore: 204/205

    tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_attach_detach_volume  FAIL

    This can be solved by manually set "volume_device_name" in refstack configuration file

    HA: 8/8

    ipv6: 25/25

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/196/consoleFull

    PASS/total

    defcore: 204/205

    tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_attach_detach_volume  FAIL

    This can be solved by manually set "volume_device_name" in refstack configuration file

    HA: 8/8

    ipv6: 25/25

    SDNVPN: 0/4 no SDN feature

    os-onos-nofeature-ha Compass 

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtual-proposed_tests-danube/177/consoleFull

    PASS/total

    defcore: 150/171

    HA: 7/8 tc007, SLA not passed

    ipv6: 17/25 tc018-tc025 fails

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/160/consoleFull

    PASS/total

    defcore: 198/205

    HA: 8/8

    ipv6: 17/25 tc018-tc025 fails

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/195/consoleFull

    PASS/total

    defcore: 198/205

    HA: 8/8

    ipv6: 17/25 tc018-tc025 fails

    SDNVPN: 0/4 no SDN feature

    os-odl_l3-nofeature-ha Compass 

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtual-proposed_tests-danube/178/consoleFull

    PASS/total

    defcore: 204/205

    tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_attach_detach_volume  FAIL

    This can be solved by manually set "volume_device_name" in refstack configuration file

    HA: 7/8 tc007, SLA not passed

    ipv6: 17/25 tc018-tc025 fails

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/159/consoleFull

    PASS/total

    defcore: 204/205

    tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_attach_detach_volume  FAIL

    This can be solved by manually set "volume_device_name" in refstack configuration file

    HA: 8/8

    ipv6: 17/25 tc018-tc025 fails

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/198/consoleFull

    PASS/total

    defcore: 204/205

    tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_attach_detach_volume  FAIL

    This can be solved by manually set "volume_device_name" in refstack configuration file

    HA: 8/8

    ipv6: 17/25 tc018-tc025 fails

    SDNVPN: 0/4 no SDN feature

    os-nosdn-ovs-ha Fuel    
    os-odl_l2-bgpvpn-ha Fuel 

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-fuel-baremetal-proposed_tests-danube/45/consoleFull

    PASS/total

    defcore: 205/205

    HA: 8/8

    ipv6: 17/25 tc018-tc025 fails

    SDNVPN: 3/4 sdnvpn.tc001 "

    VPN provides connectivity between Neutron subnets" fails

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-fuel-baremetal-proposed_tests-danube/52/console

    PASS/total

    defcore: 205/205

    HA: 8/8

    ipv6: 17/25 tc018-tc025 fails

    SDNVPN: 4/4

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-fuel-baremetal-proposed_tests-danube/7071/consoleFull

    PASS/total

    defcore: 205/205

    HA: 8/8

    ipv6: 17/25 tc018-tc025 fails

    SDNVPN: 34/4sdnvpn.tc001 "VPN provides connectivity between Neutron subnets" fails

    os-odl_l2-nofeature-haFuel(ZTE-POD1)    
    os-odl_l3-nofeature-ha Fuel    
    os-nosdn-nofeature-haFuel    
    os-nosdn-kvm_ovs_dpdk-haFuel    

    ...