Versions Compared

Version Old Version 78 New Version Current
Changes made by

Dan Xu

Dan Xu

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

 

this wiki page is used to record the running histroy for the dovetil tool.this may cover the runnning in the plugfest, CI and vendors lab.

Table of Contents

1. Version change logs

For each dovetail release version, the docker images used are introduced first, then problems found and fixed are listed as well as the updates.

1.1 cvp.0.1.0  (2017/06/23)

1.1.1 Docker images used:

opnfv/dovetail:cvp.0.1.0 (commit ID 041a51d6d3ba5f2d9eabcf3501abb9af30ebc394)

opnfv/yardstick:danube.2.0

opnfv/functest:danube.2.0

opnfv/testapi:cvp.0.1.0 (Commit ID 056b7ccefe38b13e4b4eb11ff7e259e254e35c32)

1.1.2 Updates and problems found and fixed in this version

...

  1. https enabled checking
  2. sdnvpn path fix
  3. log improvement
  4. docker image tag with cvp.0.2.0

...

 

this wiki page is used to record the running histroy for the dovetil tool.this may cover the runnning in the plugfest, CI and vendors lab.

Table of Contents

1. Version change logs

For each dovetail release version, the docker images used are introduced first, then problems found and fixed are listed as well as the updates.

1.1 cvp.0.1.0  (2017/06/23)

1.1.1 Docker images used:

opnfv/dovetail:cvp.0.1.0 (commit ID 041a51d6d3ba5f2d9eabcf3501abb9af30ebc394)

opnfv/yardstick:danube.2.0

opnfv/functest:danube.2.0

opnfv/testapi:cvp.0.1.0 (Commit ID 056b7ccefe38b13e4b4eb11ff7e259e254e35c32)

1.1.2 Updates and problems found and fixed in this version

ImageUpdates and problems found and fixedJIRAGerrit
opnfv/dovetail:cvp.0.1.0
  1. https enabled checking
  2. sdnvpn path fix
  3. log improvement
  4. docker image tag with cvp.0.2.0
  1. DOVETAIL-456
  2. DOVETAIL-458
  3. DOVETAIL-450
  4. DOVETAIL-447
  1. https://gerrit.opnfv.org/gerrit/#/c/36541/
  2. https://gerrit.opnfv.org/gerrit/#/c/36785/
  3. https://gerrit.opnfv.org/gerrit/#/c/36615/
  4. https://gerrit.opnfv.org/gerrit/#/c/36775/
opnfv/yardstick:danube.2.0
  1. Bugfix: Monitor command in tc019 may not show the real nova-api service status
  2. Bugfix: "monitor_multi" type monitor in HA test case cannot get "max_recover_time"
  3. Improvement: Terminate openstack service's process using "kill" command
  4. Improvement: "monitor_process" type monitor pass criteria
  5. Bugfix: test.dbf file not deleted after execution of Disk I/O Block High Availability test
  1. YARDSTICK-655
  2. YARDSTICK-657
  3. YARDSTICK-659
  4. YARDSTICK-660
  5. YARDSTICK-696
  1. https://gerrit.opnfv.org/gerrit/#/c/34817/
  2. https://gerrit.opnfv.org/gerrit/#/c/35251/
  3. https://gerrit.opnfv.org/gerrit/#/c/36215/
  4. https://gerrit.opnfv.org/gerrit/#/c/35329/
  5. https://gerrit.opnfv.org/gerrit/#/c/36759/
opnfv/functest:danube.2.0SDNVPN test cases are not enable in FunctestNone

https://gerrit.opnfv.org/gerrit/#/c/35051/2

https://gerrit.opnfv.org/gerrit/#/c/36777/3

opnfv/testapi:cvp.0.1.0user management and some bugfixNoneNone


1.2 cvp.0.2.0 (2017/07/07)

1.2.1 Docker images used:

opnfv/dovetail:cvp.0.2.0 (commit ID 5ddf932dc28bcc47169c3091267d57ba5a99f9b2)

opnfv/yardstick:danube.3.0

opnfv/functest:cvp.0.2.0 (commit ID 3d03bbcfc45d00d4ce995d8aabff5808accb0687)

opnfv/testapi:cvp.0.2.0 (commit ID a7f82c093fab2ad19c7ffb0a81ec5756e91e73ae)

1.2.2 Updates and problems found and fixed in this version

ImageUpdates and problems found and fixedJIRAGerrit
opnfv/dovetail:cvp.0.2.0
  1. Update IPv6 tc018-tc025: give more details and modify some misunderstanding about these test cases
  2. Delete the ERROR log when it is not https
  3. Adapt Yardstick to copy yardstick.conf to /etc/yardstick
  4. Fix the docker version to be 17.03.0-ce rather than the latest one
  1. DOVETAIL-461
  2. DOVETAIL-462
  3. DOVETAIL-466
  4. DOVETAIL-467
  1. https://gerrit.opnfv.org/gerrit/#/c/37229/
  2. https://gerrit.opnfv.org/gerrit/#/c/37449/
  3. https://gerrit.opnfv.org/gerrit/#/c/37737/
  4. https://gerrit.opnfv.org/gerrit/#/c/37813/
opnfv/yardstick:danube.3.0
  1. Support ignoring server certificate
  2. Add debug log for recovering service failure
  3. Improve method to kill keystone process
None
  1. https://gerrit.opnfv.org/gerrit/#/c/36377/
opnfv/functest:cvp.0.2.0NoneNoneNone
opnfv/testapi:cvp.0.2.0
  1. role based access control and result upload
  2. testapi tox bugfix
  3. some bugfix
  1. None
  2. None
  3. RELENG-281 
    RELENG-212 
    FUNCTEST-694 
    RELENG-213  
  1. https://gerrit.opnfv.org/gerrit/#/c/37053/
    https://gerrit.opnfv.org/gerrit/#/c/37839/
  2. https://gerrit.opnfv.org/gerrit/#/c/37459/
  3. https://gerrit.opnfv.org/gerrit/#/c/37765/
    https://gerrit.opnfv.org/gerrit/#/c/37723/
    https://gerrit.opnfv.org/gerrit/#/c/37711/

 

1.3 cvp.0.3.0 (2017/7/21)

1.3.1 Docker images used:

opnfv/dovetail:cvp.0.3.0 (commit ID 7ed1cf189676693d84c9d48bb964c178b16bae76)

opnfv/yardstick:danube.3.1

opnfv/functest:cvp.0.2.0 (commit ID 3d03bbcfc45d00d4ce995d8aabff5808accb0687)

opnfv/testapi:cvp.0.3.0 (commit ID 4e0b74b0141e3ae5d250006f3d75fcc880738cc9)

1.3.2 Updates and problems found and fixed in this version

ImageUpdates and problems found and fixedJIRAGerrit
opnfv/dovetail:cvp.0.3.0
  1. Add docs about vping test cases (test scope and test specification)
  2. Add 2 vping test cases (vping.tc001->vping_userdata, vping.tc002->vping_ssh) into proposed_tests
  3. Bugfix: Build docker image failed because of lacking of packages
  4. Add resiliency.tc001 into proposed_tests
  1. None
  2. DOVETAIL-446
  3. DOVETAIL-475
  4. DOVETAIL-474
  1. https://gerrit.opnfv.org/gerrit/#/c/36253/
  2. https://gerrit.opnfv.org/gerrit/#/c/37853/
  3. https://gerrit.opnfv.org/gerrit/#/c/38573/
  4. https://gerrit.opnfv.org/gerrit/#/c/38713/
opnfv/yardstick:danube.3.1NoneNoneNone
opnfv/functest:cvp.0.2.0NoneNoneNone
opnfv/testapi:cvp.0.3.0
  1. Add Linux Foundation ID as account provider
  2. Upload multi results in one time
  3. Submit results to CVP to review
  4. Results are grouped by test id
  5. Add details for each test
NoneNone

 

1.4 cvp.0.4.0 (2017/8/4)

1.4.1 Docker images used:

opnfv/dovetail:cvp.0.4.0 (commit ID a8d7665ca783684bc10d7c3dbbc606e30f083bc7)

opnfv/yardstick:danube.3.1

opnfv/functest:cvp.0.2.0 (commit ID 3d03bbcfc45d00d4ce995d8aabff5808accb0687)

opnfv/testapi:cvp.0.4.0

opnfv/bottlenecks:cvp.0.4.0 (commit ID 3592f9016821a603930ebda7f4112abf732e6750)

1.4.2 Updates and problems found and fixed in this version

ImageUpdates and problems found and fixedJIRAGerrit
opnfv/dovetail:cvp.0.4.0
  1. Bugfix: wrong command in function 'check_https_enabled'
  2. Use opnfv/functest:cvp.0.5.0 and do some adaptions in dovetail tool
  3. Create a tar file for all logs
  4. Add tempest test cases into proposed_tests
  5. Fix testapi base_url for public_url
  6. Add user guide for results operations on web
  7. Add nginx to host static files
  1. DOVETAIL-482
  2. DOVETAIL-480
  3. DOVETAIL-479
  4. DOVETAIL-478
  5. None
  6. DOVETAIL-471
  7. DOVETAIL-485
  1. https://gerrit.opnfv.org/gerrit/#/c/39393/
  2. https://gerrit.opnfv.org/gerrit/#/c/39221/
  3. https://gerrit.opnfv.org/gerrit/#/c/38891/
  4. https://gerrit.opnfv.org/gerrit/#/c/38835/
  5. https://gerrit.opnfv.org/gerrit/#/c/38823/
  6. https://gerrit.opnfv.org/gerrit/#/c/38065/
  7. https://gerrit.opnfv.org/gerrit/#/c/39489/
opnfv/yardstick:danube.3.1Bugfix: cannot run task if without yardstick.conf in danubeYARDSTICK-776https://gerrit.opnfv.org/gerrit/#/c/38955/
opnfv/functest:cvp.0.2.0
  1. Remove functest repo and only python module have been installed
  2. Functest Docker image will not host any OS images
  3. Add a dedicated file to configure extra params in tempest.conf

  1. FUNCTEST-836

  2. None

  3. FUNCTEST-860

  1. https://gerrit.opnfv.org/gerrit/#/c/

...

  1. 35813/
    https://gerrit.opnfv.org/gerrit/#/c/

...

  1. 36163/
  2. https://gerrit.opnfv.org/gerrit/#/c/

...

  1. 37373/
  2. https://gerrit.opnfv.org/gerrit/#

...

  1. /c/39019/
opnfv/testapi:cvp.0.4.0
  1. log browsing in directory style
  
opnfv/

...

bottlenecks:

...

cvp.0.

...

4.0

...

  1. Bugfix: Monitor command in tc019 may not show the real nova-api service status
  2. Bugfix: "monitor_multi" type monitor in HA test case cannot get "max_recover_time"
  3. Improvement: Terminate openstack service's process using "kill" command
  4. Improvement: "monitor_process" type monitor pass criteria
  5. Bugfix: test.dbf file not deleted after execution of Disk I/O Block High Availability test

...

  1. YARDSTICK-655
  2. YARDSTICK-657
  3. YARDSTICK-659
  4. YARDSTICK-660
  5. YARDSTICK-696

...

  1. https://gerrit.opnfv.org/gerrit/#/c/34817/
  2. https://gerrit.opnfv.org/gerrit/#/c/35251/
  3. https://gerrit.opnfv.org/gerrit/#/c/36215/
  4. https://gerrit.opnfv.org/gerrit/#/c/35329/
  5. https://gerrit.opnfv.org/gerrit/#/c/36759/

...

https://gerrit.opnfv.org/gerrit/#/c/35051/2

https://gerrit.opnfv.org/gerrit/#/c/36777/3

...

1.2.1 Docker images used:

opnfv/dovetail:cvp.0.2.0 (commit ID 5ddf932dc28bcc47169c3091267d57ba5a99f9b2)

opnfv/yardstick:danube.3.0

opnfv/functest:cvp.0.2.0 (commit ID 3d03bbcfc45d00d4ce995d8aabff5808accb0687)

opnfv/testapi:cvp.0.2.0 (commit ID a7f82c093fab2ad19c7ffb0a81ec5756e91e73ae)

1.2.2 Updates and problems found and fixed in this version

...

  1. Update IPv6 tc018-tc025: give more details and modify some misunderstanding about these test cases
  2. Delete the ERROR log when it is not https
  3. Adapt Yardstick to copy yardstick.conf to /etc/yardstick
  4. Fix the docker version to be 17.03.0-ce rather than the latest one

...

  1. DOVETAIL-461
  2. DOVETAIL-462
  3. DOVETAIL-466
  4. DOVETAIL-467
   

 

1.5 cvp.0.5.0 (2017/8/18)

1.5.1 Docker images used:

opnfv/dovetail:cvp.0.5.0 (commit ID 474795c11c556dcd325559853fe7ed436a69418c)

opnfv/yardstick:danube.3.2 (commit ID e9338f29a70a38a5838498adf79ec91f92d085d1)

opnfv/functest:cvp.0.5.0  (commit ID 13067de652d6ccdcfc85f0f5e65841b644d9a6e9)

opnfv/bottlenecks:cvp.0.4.0 (commit ID 3592f9016821a603930ebda7f4112abf732e6750)

opnfv/testapi:cvp.0.5.0 

opnfv/dovetail:nginx.cvp.0.5.0

1.5.2 Updates and problems found and fixed in this version

ImageUpdates and problems found and fixedJIRAGerrit
opnfv/dovetail:cvp.0.5.0
  1. Merge addendum doc and workflow doc
  2. Update user guide docs
  3. Remake aarch64 patch for Dovetail container
  4. Remove some tempest test cases that are not too common
  5. Add additional Tempest API test cases into test case tempest.tc005
  6. persist db file in host for mongo container
  7. add nginx to host static files
  8. add a sample tempest_conf.yaml file into userconfig
  9. Add a patch to set ha SLA to be 30 seconds
  10. Modify logs
  11. Bugfix: the config files in useeconfig will cover user's config
  12. Bugfix: functest only keeps the last test case's log
  13. Bugfix: pre_condition cmds is just executed once
  14. Bugfix: No cirros-0.3.5-x86_64-disk.img
  15. Merge all test specification docs about test area tempest and defcore
  1. None
  2. DOVETAIL-493
  3. ARMBAND-286
  4. None
  5. DOVETAIL-495
  6. DOVETAIL-488
  7. DOVETAIL-485
  8. DOVETAIL-484
  9. DOVETAIL-496
  10. DOVETAIL-498
  11. DOVETAIL-492
  12. DOVETAIL-490
  13. DOVETAIL-489
  14. DOVETAIL-487
  15. DOVETAIL-431
    DOVETAIL-473

  1. https://gerrit.opnfv.org/gerrit/#/c/35161/
    https://gerrit.opnfv.org/gerrit/#/c/34643/
  2. https://gerrit.opnfv.org/gerrit/#/c/40369/
  3. https://gerrit.opnfv.org/gerrit/#/c/39073/
  4. https://gerrit.opnfv.org/gerrit/#/c/

...

  1. 40963/
  2. https://gerrit.opnfv.org/gerrit/#/c/

...

  1. 40745/
  2. https://gerrit.opnfv.org/gerrit/#/c/

...

  1. 39851/
  2. https://gerrit.opnfv.org/gerrit/#/c/

...

  1. 39489/

...

  1. Support ignoring server certificate
  2. Add debug log for recovering service failure
  3. Improve method to kill keystone process

...

  1. https://gerrit.opnfv.org/gerrit/#/c/39463/
  2. https://gerrit.opnfv.org/gerrit/#/c/

...

  1. 41251/

...

  1. role based access control and result upload
  2. testapi tox bugfix
  3. some bugfix
  1. https://gerrit.opnfv.org/gerrit/#/c/41285/
  2. https://gerrit.opnfv.org/gerrit/#/c/

...

  1. 40733/
  2. https://gerrit.opnfv.org/gerrit/#/c/

...

  1. 40051/
  2. https://gerrit.opnfv.org/gerrit/#/c/

...

  1. 39925/
  2. https://gerrit.opnfv.org/gerrit/#/c/

...

  1. 39719/
  2. https://gerrit.opnfv.org/gerrit/#/c/

...

  1. 34649/
    https://gerrit.opnfv.org/gerrit/#/c/

...

1.3 cvp.0.3.0 (2017/7/21)

1.3.1 Docker images used:

opnfv/dovetail:cvp.0.3.0 (commit ID 7ed1cf189676693d84c9d48bb964c178b16bae76)

opnfv/yardstick:danube.3.1

opnfv/functest:cvp.0.2.0 (commit ID 3d03bbcfc45d00d4ce995d8aabff5808accb0687)

opnfv/testapi:cvp.0.3.0 (commit ID 4e0b74b0141e3ae5d250006f3d75fcc880738cc9)

1.3.2 Updates and problems found and fixed in this version

ImageUpdates and problems found and fixedJIRAGerrit
opnfv/dovetail:cvp.0.3.0
  1. Add docs about vping test cases (test scope and test specification)
  2. Add 2 vping test cases (vping.tc001->vping_userdata, vping.tc002->vping_ssh) into proposed_tests
  3. Bugfix: Build docker image failed because of lacking of packages
  4. Add resiliency.tc001 into proposed_tests
  1. None
  2. DOVETAIL-446
  3. DOVETAIL-475
  4. DOVETAIL-474
  1. 39389/

 

  1. https://gerrit.opnfv.org/gerrit/#/c/37989/
    https://gerrit.opnfv.org/gerrit/#/c/39459/
    https://gerrit.opnfv.org/gerrit/#/c/39585/
    https://gerrit.opnfv.org/gerrit/#/c/39223/
    https://gerrit.opnfv.org/gerrit/#/c/3625339229/
    https://gerrit.opnfv.org/gerrit/#/c/3785339227/
    https://gerrit.opnfv.org/gerrit/#/c/3857338379/
    https://gerrit.opnfv.org/gerrit/#/c/3871339225/

 

 

opnfv/yardstick:danube.3.12NoneNoneNone
opnfv/functest:cvp.0.25.0NoneNoneNone
opnfv/testapi:cvp.0.35.0
  1. Add Linux Foundation ID as account provider
  2. Upload multi results in one time
  3. Submit results to CVP to review
  4. Results are grouped by test id
  5. Add details for each test
NoneNone   
opnfv/dovetail:nginx.cvp.0.5.0   

1.

4

6 cvp.0.

4

6.0 (2017/9/8

/4

)

1.

4

6.1 Docker images used:

opnfv/dovetail:cvp.0.46.0 0 (commit ID a8d7665ca783684bc10d7c3dbbc606e30f083bc70aacdceeeac62333651ec5440fb32af965c7689e)

opnfv/yardstick:danube.3.12 (commit ID e9338f29a70a38a5838498adf79ec91f92d085d1)

opnfv/functest:cvp.0.25.0  (commit ID 13067de652d6ccdcfc85f0f5e65841b644d9a6e9)

opnfv/bottlenecks:cvp.0.4.0 (commit ID 3d03bbcfc45d00d4ce995d8aabff5808accb0687ID 3592f9016821a603930ebda7f4112abf732e6750)

opnfv/testapi:cvp.0.4.5.0 

opnfv/dovetail:web.cvp.0.6.0

opnfv/bottlenecksdovetail:api.cvp.0.4.0 (commit ID 3592f9016821a603930ebda7f4112abf732e6750)6.0

1.

4

6.2 Updates and problems found and fixed in this version

opnfv/dovetail:nginx.
ImageUpdates and problems found and fixedJIRAGerrit
opnfv/dovetail:cvp.0.46.0
  1. Tag dovetail with cvp.0.7.0
  2. Disable get DB schema in Dockerfile
  3. Bugfix:
wrong command in function 'check_https_enabled'
  • Use opnfv/functest:cvp.0.5.0 and do some adaptions in dovetail tool
  • Create a tar file for all logs
  • Add tempest test cases into proposed_tests
  • Fix testapi base_url for public_url
  • Add user guide for results operations on web
  • Add nginx to host static files
  • DOVETAIL-482
  • DOVETAIL-480
  • DOVETAIL-479
  • DOVETAIL-478
  • None
  • DOVETAIL-471
    • DOVETAIL-485
    1. sdnvpn test cases failed due to wrong module
    2. Support to run mandatory or optional test cases separately
    3. add api&web services for cvp
    4. Don't generate tar file when run without pushing results to DB
    5. Make Yardstick log level can be set by CLI
    6. Update the script for building local DB and testapi
    7. Bugfix: tar results dir cannot be parsed by web portal
    8. Replacing references to "DefCore"
    9. Updating the test specification document with iterative inprovements
    10. Dovetail tool replaces defcore with osinterop
    11. Clearer structuring current and future test scope of CVP addendum
    12. Adding a stub document for a CVP reviewer guide
    13. Clarify criteria for testcase inclusion as running in Functest/CI And additional clarifications
    14. add more information for the Identity testcases in the refstack
    15. Doc: update user guide due to add new test cases and features
    1. DOVETAIL-516
    2. None
    3. DOVETAIL-510
    4. DOVETAIL-505
    5. DOVETAIL-512
    6. DOVETAIL-511
    7. DOVETAIL-506
    8. DOVETAIL-507
    9. DOVETAIL-508
    10. DOVETAIL-499
    11. DOVETAIL-352
    12. DOVETAIL-499
    13. None
    14. DOVETAIL-408
    15. DOVETAIL-477
    16. DOVETAIL-431
    17. DOVETAIL-493
    1. https://gerrit.opnfv.org/gerrit/#/c/3939343575/
    2. https://gerrit.opnfv.org/gerrit/#/c/3922143565/
    3. https://gerrit.opnfv.org/gerrit/#/c/3889143393/
    4. https://gerrit.opnfv.org/gerrit/#/c/3883543109/
    5. https://gerrit.opnfv.org/gerrit/#/c/3882343411/
    6. https://gerrit.opnfv.org/gerrit/#/c/3806543407/
    7. https://gerrit.opnfv.org/gerrit/#/c/3948942981/
    		opnfv/yardstick:danube.3.1Bugfix: cannot run task if without yardstick.conf in danubeYARDSTICK-776
    1. https://gerrit.opnfv.org/gerrit/#/c/43231/
    2. https://gerrit.opnfv.org/gerrit/#/c/43131/
    3. https://gerrit.opnfv.org/gerrit/#/c/
    38955
    1. 41601/
    		opnfv/functest:cvp.0.2.0
    1. Remove functest repo and only python module have been installed
    2. Functest Docker image will not host any OS images
    3. Add a dedicated file to configure extra params in tempest.conf

    1. FUNCTEST-836

    2. None

    3. FUNCTEST-860https://gerrit.opnfv.org/gerrit/#/c/32499/
    4. https://gerrit.opnfv.org/gerrit/#/c/3581342205/
    5. https://gerrit.opnfv.org/gerrit/#/c/3616341241/
    6. https://gerrit.opnfv.org/gerrit/#/c/3737333851/
    7. https://gerrit.opnfv.org/gerrit/#/c/3901938763/
    opnfv/testapi:cvp.0.4.0
    1. log browsing in directory style
    		  
    opnfv/bottlenecks:cvp.0.4.0   

     

    1.5 cvp.0.5.0 (2017/8/18)

    1.5.1 Docker images used:

    opnfv/dovetail:cvp.0.5.0 (commit ID 474795c11c556dcd325559853fe7ed436a69418c)

    opnfv/yardstick:danube.3.2 (commit ID e9338f29a70a38a5838498adf79ec91f92d085d1)

    opnfv/functest:cvp.0.5.0  (commit ID 13067de652d6ccdcfc85f0f5e65841b644d9a6e9)

    opnfv/bottlenecks:cvp.0.4.0 (commit ID 3592f9016821a603930ebda7f4112abf732e6750)

    opnfv/testapi:cvp.0.5.0 

    1. https://gerrit.opnfv.org/gerrit/#/c/34649/
    2. https://gerrit.opnfv.org/gerrit/#/c/40369/
    opnfv/yardstick:danube.3.2NoneNoneNone
    ImageUpdates and problems found and fixedJIRAGerrit
    opnfv/functest:cvp.0.5.0

    1.5.2 Updates and problems found and fixed in this version

    opnfv/dovetailNoneNoneNone
    opnfv/testapi:cvp.0.5.0
    1. Merge addendum doc and workflow doc
    2. Update user guide docs
    3. Remake aarch64 patch for Dovetail container
    4. Remove some tempest test cases that are not too common
    5. Add additional Tempest API test cases into test case tempest.tc005
    6. persist db file in host for mongo container
    7. add nginx to host static files
    8. add a sample tempest_conf.yaml file into userconfig
    9. Add a patch to set ha SLA to be 30 seconds
    10. Modify logs
    11. Bugfix: the config files in useeconfig will cover user's config
    12. Bugfix: functest only keeps the last test case's log
    13. Bugfix: pre_condition cmds is just executed once
    14. Bugfix: No cirros-0.3.5-x86_64-disk.img
    15. Merge all test specification docs about test area tempest and defcore
    1. None
    2. DOVETAIL-493
    3. ARMBAND-286
    4. None
    5. DOVETAIL-495
    6. DOVETAIL-488
    7. DOVETAIL-485
    8. DOVETAIL-484
    9. DOVETAIL-496
    10. DOVETAIL-498
    11. DOVETAIL-492
    12. DOVETAIL-490
    13. DOVETAIL-489
    14. DOVETAIL-487
      15. DOVETAIL-431
      DOVETAIL-473
    15. https://gerrit.opnfv.org/gerrit/#/c/35161/
      https://gerrit.opnfv.org/gerrit/#/c/34643/
    16. https://gerrit.opnfv.org/gerrit/#/c/40369/
    17. https://gerrit.opnfv.org/gerrit/#/c/39073/
    18. https://gerrit.opnfv.org/gerrit/#/c/40963/
    19. https://gerrit.opnfv.org/gerrit/#/c/40745/
    20. https://gerrit.opnfv.org/gerrit/#/c/39851/
    21. https://gerrit.opnfv.org/gerrit/#/c/39489/
    22. https://gerrit.opnfv.org/gerrit/#/c/39463/
    23. https://gerrit.opnfv.org/gerrit/#/c/41251/
    24. https://gerrit.opnfv.org/gerrit/#/c/41285/
    25. https://gerrit.opnfv.org/gerrit/#/c/40733/
    26. https://gerrit.opnfv.org/gerrit/#/c/40051/
    27. https://gerrit.opnfv.org/gerrit/#/c/39925/[cvp-web]correct dockerfile
    28. [cvp-web]remove useless docker-compose yaml
    29. [cvp-web]remove workflow on homepage
    1. DOVETAIL-515
    2. DOVETAIL-514
    3. DOVETAIL-513
    1. https://gerrit.opnfv.org/gerrit/#/c/43555/
    2. https://gerrit.opnfv.org/gerrit/#/c/43529/
    3. https://gerrit.opnfv.org/gerrit/#/c/43527/
    opnfv/dovetail:web.cvp.0.6.0   
    opnfv/dovetail:api.cvp.0.6.0   

    1.7 cvp.0.7.0 (2017/9/29)

    1.7.1 Docker images used:

    opnfv/dovetail:cvp.0.7.0 (commit ID 043ef2dad58c3aef0ea91cf3063dc9deb6f40bd9)

    opnfv/yardstick:danube.3.2 (commit ID e9338f29a70a38a5838498adf79ec91f92d085d1)

    opnfv/functest:cvp.0.5.0  (commit ID 13067de652d6ccdcfc85f0f5e65841b644d9a6e9)

    opnfv/bottlenecks:cvp.0.4.0 (commit ID 3592f9016821a603930ebda7f4112abf732e6750)

    opnfv/testapi:cvp.0.5.0 

    opnfv/dovetail:web.cvp.0.7.0

    opnfv/dovetail:api.cvp.0.7.0 

    1.7.2 Updates and problems found and fixed in this version

     

    ImageUpdates and problems found and fixedJIRAGerrit
    opnfv/dovetail:cvp.0.7.0
    1. Tag dovetail with cvp.0.8.0
    2. Fixing references and layout in the docs.
    3. Updating test case requirements document
    4. Term refstack is mentioned in dovetail Testing User Guide
    5. Add dovetail tool CLI document
    6. Cvp application form
    7. Use functest clean module to do the sdnvpn resources cleanup
    8. SDNVPN test cases resources left may effect the next test cases
    9. Add testsuite cvp.0.8.0 for Beta tests
    10. Bugfix: the sdnvpn DB initialization information lacks two test cases
    11. Bugfix: build docker image failed
    1. DOVETAIL-536
    2. DOVETAIL-521
    3. None
    4. DOVETAIL-519
    5. DOVETAIL-509
    6. None
    7. DOVETAIL-524
    8. DOVETAIL-524
    9. DOVETAIL-528
    10. DOVETAIL-535
    11. DOVETAIL-534
    1. https://gerrit.opnfv.org/gerrit/#/c/3971946155/
    2. https://gerrit.opnfv.org/gerrit/#/c/3464944497/
    3. https://gerrit.opnfv.org/gerrit/#/c/3938942291/
    4. https://gerrit.opnfv.org/gerrit/#/c/3798944859/
    5. https://gerrit.opnfv.org/gerrit/#/c/3945944919/
    6. https://gerrit.opnfv.org/gerrit/#/c/3958542455/
    7. https://gerrit.opnfv.org/gerrit/#/c/3922345319/
    8. https://gerrit.opnfv.org/gerrit/#/c/3922945283/
    9. https://gerrit.opnfv.org/gerrit/#/c/3922745483/
    10. https://gerrit.opnfv.org/gerrit/#/c/3837946103/
    11. https://gerrit.opnfv.org/gerrit/#/c/3922546101/

     

     
    opnfv/yardstick:danube.3.2NoneNoneNone
    opnfv/functest:cvp.0.5.0NoneNoneNone
    opnfv/testapi:cvp.0.5.0   
    opnfv/dovetailfunctest:nginx.cvp.0.5.0 None None 
    1.6
    None
    opnfv/testapi:cvp.0.
    6
    5.0
    (2017/9/8)1.6.1 Docker images used:
    None
    None
    None
    opnfv/dovetail:web.cvp.0.
    6.0 (commit ID 0aacdceeeac62333651ec5440fb32af965c7689e)

    opnfv/yardstick:danube.3.2 (commit ID e9338f29a70a38a5838498adf79ec91f92d085d1)

    opnfv/functest:cvp.0.5.0  (commit ID 13067de652d6ccdcfc85f0f5e65841b644d9a6e9)

    opnfv/bottlenecks:cvp.0.4.0 (commit ID 3592f9016821a603930ebda7f4112abf732e6750)

    opnfv/testapi:cvp.0.5.0 

    opnfv/dovetail:web.cvp.0.6.0

    opnfv/dovetail:api.cvp.0.6.0

    1.6.2 Updates and problems found and fixed in this version

    ImageUpdates and problems found and fixedJIRAGerrit
    opnfv/dovetail:cvp.0.6.0
    1. Tag dovetail with cvp.0.7.0
    2. Disable get DB schema in Dockerfile
    3. Bugfix: sdnvpn test cases failed due to wrong module
    4. Support to run mandatory or optional test cases separately
    5. add api&web services for cvp
    6. Don't generate tar file when run without pushing results to DB
    7. Make Yardstick log level can be set by CLI
    8. Update the script for building local DB and testapi
    9. Bugfix: tar results dir cannot be parsed by web portal
    10. Replacing references to "DefCore"
    11. Updating the test specification document with iterative inprovements
    12. Dovetail tool replaces defcore with osinterop
    13. Clearer structuring current and future test scope of CVP addendum
    14. Adding a stub document for a CVP reviewer guide
    15. Clarify criteria for testcase inclusion as running in Functest/CI And additional clarifications
    16. add more information for the Identity testcases in the refstack
    17. Doc: update user guide due to add new test cases and features
    1. DOVETAIL-516
    2. None
    3. DOVETAIL-510
    4. DOVETAIL-505
    5. DOVETAIL-512
    6. DOVETAIL-511
    7. DOVETAIL-506
    8. DOVETAIL-507
    9. DOVETAIL-508
    10. DOVETAIL-499
    11. DOVETAIL-352
    12. DOVETAIL-499
    13. None
    14. DOVETAIL-408
    15. DOVETAIL-477
    16. DOVETAIL-431
    17. DOVETAIL-493
  • https://gerrit.opnfv.org/gerrit/#/c/43575/
  • https://gerrit.opnfv.org/gerrit/#/c/43565/
  • https://gerrit.opnfv.org/gerrit/#/c/43393/
  • https://gerrit.opnfv.org/gerrit/#/c/43109/
  • https://gerrit.opnfv.org/gerrit/#/c/43411/
  • https://gerrit.opnfv.org/gerrit/#/c/43407/
  • https://gerrit.opnfv.org/gerrit/#/c/42981/
    • https://gerrit.opnfv.org/gerrit/#/c/43231/7.0
    1. [web-cvp]Direct drill-down for failed tests
    2. [web-cvp]Additional columns in ‘My Results’
    3. [web-cvp]share result by email
    4. [web-cvp]failed to build docker image due to setuptools version
    5. [web-cvp]add a api to fecth logs of backend api
    6. [web-cvp]fix api&web service in dockerfiles
    7. [web-cvp]missing files in web sites
    8. [web-cvp]osinteros is counted as optional test case
    9. [web-cvp]failed to expand test results
    1. DOVETAIL-531
    2. DOVETAIL-532
    3. DOVETAIL-529
    4. DOVETAIL-526
    5. DOVETAIL-525
    6. DOVETAIL-523
    7. DOVETAIL-522
    8. DOVETAIL-518
    9. DOVETAIL-517
    1. https://gerrit.opnfv.org/gerrit/#/c/45757/
    2. https://gerrit.opnfv.org/gerrit/#/c/45943/
    3. https://gerrit.opnfv.org/gerrit/#/c/45709/
    4. https://gerrit.opnfv.org/gerrit/#/c/44913/
    5. https://gerrit.opnfv.org/gerrit/#/c/44907/
    6. https://gerrit.opnfv.org/gerrit/#/c/44555/
    7. https://gerrit.opnfv.org/gerrit/#/c/44545/
    8. https://gerrit.opnfv.org/gerrit/#/c/43713/
    9. https://gerrit.opnfv.org/gerrit/#/c/43709/
    opnfv/dovetail:api.cvp.0.7.0

     

    1.8 cvp.0.8.0 (2017/10/25)

    1.8.1 Docker images used:

    opnfv/dovetail:cvp.0.8.0 (commit ID 73f332fc513f184513be483db6a108bd3c7b7d9b)

    opnfv/yardstick:danube.3.2 (commit ID e9338f29a70a38a5838498adf79ec91f92d085d1)

    opnfv/functest:cvp.0.5.0  (commit ID 13067de652d6ccdcfc85f0f5e65841b644d9a6e9) 

    opnfv/bottlenecks:cvp.0.4.0 (commit ID 3592f9016821a603930ebda7f4112abf732e6750)

    opnfv/testapi:cvp.0.5.0 

    opnfv/dovetail:web.cvp.0.8.0 (commit ID 3617b6958c4c44678d64c05e669b50e427fc45f8)

    opnfv/dovetail:api.cvp.0.8.0 (commit ID 3617b6958c4c44678d64c05e669b50e427fc45f8)

    1.8.2 Updates and problems found and fixed in this version

     

    -web]correct dockerfileremove useless docker-compose yamlremove workflow on homepage515514DOVETAIL-513435554352943527
    ImageUpdates and problems found and fixedJIRAGerrit
    opnfv/dovetail:cvp.0.8.0
    1. Remove obsolete developer documents
    2. Add test case name into tempest testspecification docs
    3. Update the user guide environment diagrams
    4. Modify reference to online mode
    5. Get hardware info of SUT
    6. Get SUT's endpoint info
    7. Minor docs change to test docs integration
    8. Bugfix: Failed to build local DB on Centos
    9. Tag dovetail cvp.0.9.0
    1. DOVETAIL-539
    2. DOVETAIL-530
    3. DOVETAIL-538
    4. DOVETAIL-533
    5. DOVETAIL-540
    6. DOVETAIL-546
    7. None
    8. DOVETAIL-548
    9. DOVETAIL-550
    1. https://gerrit.opnfv.org/gerrit/#/c/4313146215/
    2. https://gerrit.opnfv.org/gerrit/#/c/4160146197/
    3. https://gerrit.opnfv.org/gerrit/#/c/3249946185/
    4. https://gerrit.opnfv.org/gerrit/#/c/4220546051/
    5. https://gerrit.opnfv.org/gerrit/#/c/4124146409/
    6. https://gerrit.opnfv.org/gerrit/#/c/3385146501/
    7. https://gerrit.opnfv.org/gerrit/#/c/3876346701/
    8. https://gerrit.opnfv.org/gerrit/#/c/3464946547/
    9. https://gerrit.opnfv.org/gerrit/#/c/4036946829/
    opnfv/yardstick:danube.3.2NoneNoneNone
    opnfv/functest:cvp.0.5.0NoneNoneNone
    opnfv/testapi:cvp.0.5.0NoneNoneNone
    opnfv/dovetail:web.cvp.0.8.0
    1. [web-cvp
    1. ] Trigger an email when admin submit an application form
    2. [cvp-web]
    1. Show some SUT hardware info in 'My Results'
    2. [cvp-web]
    1. Show SUT endpoint in web page
    2. Remove approve / no approve options for reviewers
    3. Distinguish results shared or submitted from a tester
    4. Bugfix: remove redundancy line
    1. DOVETAIL-542
    2. DOVETAIL-541
    3. DOVETAIL-547
    4. DOVETAIL-
    1. 544
    2. DOVETAIL-
    1. 545
    2. None
    1. https://gerrit.opnfv.org/gerrit/#/c/46357/
    2. https://gerrit.opnfv.org/gerrit/#/c/46405/
    3. https://gerrit.opnfv.org/gerrit/#/c/
    1. 46405/
    2. https://gerrit.opnfv.org/gerrit/#/c/
    1. 46511/
    2. https://gerrit.opnfv.org/gerrit/#/c/
    1. 46511/
    opnfv/dovetail:web.cvp.0.6.0   
    1. https://gerrit.opnfv.org/gerrit/#/c/46551/
    opnfv/dovetail:api.cvp.0.68.0  

     

    1.

    7

    9 cvp.0.

    7

    9.0 (2017/

    9

    11/

    29

    8)

    1.

    7

    9.1 Docker images used:

    opnfv/dovetail:cvp.0.79.0 (commit ID 043ef2dad58c3aef0ea91cf3063dc9deb6f40bd984d413ec0d980d4d0b80bf77bc35fffb063cf686)

    opnfv/yardstick:danube.3.2 (commit ID e9338f29a70a38a5838498adf79ec91f92d085d1)

    opnfv/functest:cvp.0.5.0  (commit ID 13067de652d6ccdcfc85f0f5e65841b644d9a6e9) 

    opnfv/bottlenecks:cvp.0.4.0 (commit ID 3592f9016821a603930ebda7f4112abf732e6750)

    opnfv/testapi:cvp.0.5.0 

    opnfv/dovetail:web.cvp.0.7.0

    opnfv/dovetail:api.cvp.0.7.0 

    1.7.2 Updates and problems found and fixed in this version

     

    ImageUpdates and problems found and fixedJIRAGerritopnfv/dovetail

    3592f9016821a603930ebda7f4112abf732e6750)

    opnfv/testapi:cvp.0.

    7

    5.

    0

    0 

    Tag dovetail with

    opnfv/dovetail:web.cvp.0.8.0

  • Fixing references and layout in the docs.
  • Updating test case requirements document
  • Term refstack is mentioned in dovetail Testing User Guide
  • Add dovetail tool CLI document
  • Cvp application form
  • Use functest clean module to do the sdnvpn resources cleanup
  • SDNVPN test cases resources left may effect the next test cases
  • Add testsuite cvp.0.8.0 for Beta tests
  • Bugfix: the sdnvpn DB initialization information lacks two test cases
  • Bugfix: build docker image failed
    1. DOVETAIL-536
    2. DOVETAIL-521
    3. None
    4. DOVETAIL-519
    5. DOVETAIL-509
    6. None
    7. DOVETAIL-524
    8. DOVETAIL-524
    9. DOVETAIL-528
    10. DOVETAIL-535
    11. DOVETAIL-534
    1. https://gerrit.opnfv.org/gerrit/#/c/46155/
    2. https://gerrit.opnfv.org/gerrit/#/c/44497/
    3. https://gerrit.opnfv.org/gerrit/#/c/42291/
    4. https://gerrit.opnfv.org/gerrit/#/c/44859/
    5. https://gerrit.opnfv.org/gerrit/#/c/44919/
    6. https://gerrit.opnfv.org/gerrit/#/c/42455/
    7. https://gerrit.opnfv.org/gerrit/#/c/45319/
    8. https://gerrit.opnfv.org/gerrit/#/c/45283/
    9. https://gerrit.opnfv.org/gerrit/#/c/45483/
    10. https://gerrit.opnfv.org/gerrit/#/c/46103/
    11. https://gerrit.opnfv.org/gerrit/#/c/46101/
    opnfv/yardstick:danube.3.2NoneNoneNoneopnfv/functest:cvp.0.5.0NoneNoneNoneopnfv/testapi:cvp.0.5.0
    None
    None
    None
    opnfv/dovetail:web.cvp.0.7.0
    1. [web-cvp]Direct drill-down for failed tests
    2. [web-cvp]Additional columns in ‘My Results’
    3. [web-cvp]share result by email
    4. [web-cvp]failed to build docker image due to setuptools version
    5. [web-cvp]add a api to fecth logs of backend api
    6. [web-cvp]fix api&web service in dockerfiles
    7. [web-cvp]missing files in web sites
    8. [web-cvp]osinteros is counted as optional test case
    9. [web-cvp]failed to expand test results
    1. DOVETAIL-531
    2. DOVETAIL-532
    3. DOVETAIL-529
    4. DOVETAIL-526
    5. DOVETAIL-525
    6. DOVETAIL-523
    7. DOVETAIL-522
    8. DOVETAIL-518
    9. DOVETAIL-517
    1. https://gerrit.opnfv.org/gerrit/#/c/45757/
    2. https://gerrit.opnfv.org/gerrit/#/c/45943/
    3. https://gerrit.opnfv.org/gerrit/#/c/45709/
    4. https://gerrit.opnfv.org/gerrit/#/c/44913/
    5. https://gerrit.opnfv.org/gerrit/#/c/44907/
    6. https://gerrit.opnfv.org/gerrit/#/c/44555/
    7. https://gerrit.opnfv.org/gerrit/#/c/44545/
    8. https://gerrit.opnfv.org/gerrit/#/c/43713/
    9. https://gerrit.opnfv.org/gerrit/#/c/43709/
    opnfv/dovetail:api.cvp.0.7.0

     

    1.8 cvp.0.8.0 (2017/10/25)

    1.8.1 Docker images used:

    opnfv/dovetail:cvp.0.8.0 (commit ID 73f332fc513f184513be483db6a108bd3c7b7d9b)

    opnfv/yardstick:danube.3.2 (commit ID e9338f29a70a38a5838498adf79ec91f92d085d1)

    opnfv/functest:cvp.0.5.0  (commit ID 13067de652d6ccdcfc85f0f5e65841b644d9a6e9) 

    opnfv/bottlenecks:cvp.0.4.0 (commit ID 3592f9016821a603930ebda7f4112abf732e6750)

    opnfv/testapi:cvp.0.5.0 

    opnfv/dovetail:web.cvp.0.8.0 (commit ID 3617b6958c4c44678d64c05e669b50e427fc45f8)

    opnfv/dovetail:api.cvp.0.8.0 (commit ID 3617b6958c4c44678d64c05e669b50e427fc45f8)

     

    2. Running by CI

    in order to make the dovetail tool  more stable, the dovetail has been integreated in the CI for daily running.

    ...

    (commit ID c70193d65811a8a2a3b50fdad5463bca3643e219)

    opnfv/dovetail:api.cvp.0.8.0 (commit ID c70193d65811a8a2a3b50fdad5463bca3643e219)


    2. Running by CI

    in order to make the dovetail tool  more stable, the dovetail has been integreated in the CI for daily running.

    for more detil, please refer to link: https://build.opnfv.org/ci/view/dovetail/

     

     

    3. Running against scenarios by owners

     

    scenariosinstallerownerscvp.0.1.0cvp.0.2.0cvp.0.3.0cvp.0.4.0cvp.0.5.0cvp.0.6.0cvp.0.7.0cvp.0.8.0cvp.0.9.0
    Apex Scenarios Currently Test Against
    os-nosdn-nofeature-ha   Apex     

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-apex-huawei-pod4-proposed_tests-danube/4/console

    PASS/total

    defcore: 204/205

    HA: 7/8

    vping: 2/2

    ipv6: 25/25

    tempest: 4/6

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-apex-huawei-pod4-proposed_tests-danube/199/consoleFull

    PASS/total

    defcore: 205/205

    HA: 7/8

    vping: 2/2

    ipv6: 25/25

    tempest: 5/5

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-apex-huawei-pod4-proposed_tests-danube/215/consoleFull

    PASS/total

    osinterop: 205/205

    HA: 7/8

    vping: 2/2

    ipv6: 25/25

    tempest: 5/5

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-apex-huawei-pod4-proposed_tests-danube/228/consoleFull

    PASS/total

    osinterop: 205/205

    HA: 7/8

    vping: 2/2

    ipv6: 25/25

    tempest: 5/5

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-apex-huawei-pod4-proposed_tests-danube/223/consoleFull

    PASS/total

    osinterop: 205/205

    HA: 7/8

    vping: 2/2

    ipv6: 25/25

    tempest: 5/5

    SDNVPN: 0/4 no SDN feature

    os-odl-bgpvpn-haApex       

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-apex-huawei-pod4-proposed_tests-danube/208/consoleFull

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-apex-baremetal-proposed_tests-danube/21/consoleFull

    PASS/total

    defcore: 205/205

    HA: 7/8

    vping: 2/2

    ipv6: 17/25

    tempest: 4/5

    SDNVPN: 4/4

    		  
    Compass Scenarios Currently Test Against
    os-nosdn-nofeature-ha Compass 

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtual-proposed_tests-danube/176/consoleFull

    PASS/total

    defcore: 204/205

    HA: 7/8 tc007, SLA not passed

    ipv6: 25/25

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/

    ...

     

     

    3. Running against scenarios by owners

     

    scenariosinstallerownerscvp.0.1.0cvp.0.2.0cvp.0.3.0cvp.0.4.0cvp.0.5.0cvp.0.6.0cvp.0.7.0
    Apex Scenarios Currently Test Against
    os-nosdn-nofeature-ha   Apex     

    job/dovetail-compass-baremetal-proposed_tests-danube/157/consoleFull

    PASS/total

    defcore: 204/205

    HA: 8/8

    ipv6: 25/25

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-apexcompass-huawei-pod4baremetal-proposed_tests-danube/4196/consoleconsoleFull

    PASS/total

    defcore: 204/205

    HA: 78/8vping: 2/2

    ipv6: 25/25

    tempest: 4/6

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-apexcompass-huawei-pod4virtual-proposed_tests-danube/199335/consoleFull

    PASS/total

    defcore: 205204/205

    HA: 78/8

    vping: 2/2

    ipv6: 25/25

    tempest: 5/5

    SDNVPN: 0/4 no SDN feature

    resiliency: 1/1

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-apexcompass-huaweibaremetal-pod4-proposed_tests-danube/215288/consoleFull

    PASS/total

    defcore: 205/205

    HA: 78/8

    vping: 2/2

    ipv6: 25/25

    tempest: 53/56

    SDNVPN: 0/4 no SDN feature

    		os-odl-bgpvpn-haApex       

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-apexcompass-huawei-pod4baremetal-proposed_tests-danube/208/consoleFull331/consoleFull

    PASS/total

    defcore: 205/205

    HA: 8/8

    vping: 2/2

    ipv6: 25/25

    tempest: 5/5

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-apexhuawei-baremetalpod7-proposed_tests-danube/2118/consoleFull

    PASS/total

    defcoreosinterop: 205/205

    HA: 78/8

    vping: 2/2

    ipv6: 1725/25

    tempest: 45/5

    SDNVPN: 4/4

    Compass Scenarios Currently Test Against
    os-nosdn-nofeature-ha Compass 

    0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-huawei-virtualpod7-proposed_tests-danube/17638/consoleFull

    PASS/total

    defcoreosinterop: 204205/205

    HA: 78/8 tc007, SLA not passed

    vping: 2/2

    ipv6: 25/25

    tempest: 5/5

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-huawei-baremetalpod7-proposed_tests-danube/15731/consoleFull

    PASS/total

    defcoreosinterop: 204205/205

    HA: 8/8

    vping: 2/2

    ipv6: 25/25

    tempest: 5/5

    SDNVPN: 0/4 no SDN feature

    os-odl_l2-nofeature-ha Compass  

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/196163/consoleFull

    PASS/total

    defcore: 204/205

    HA: 8/8

    ipv6: 25/25

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtualbaremetal-proposed_tests-danube/335191/consoleFull

    PASS/total

    defcore: 204/205

    HA: 8/8

    vping: 2/2

    ipv6: 2517/25

    SDNVPN: 0/4 no SDN featureresiliency: 1/1/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetalvirtual-proposed_tests-danube/288309/consoleFull

    PASS/total

    defcore: 205204/205

    HA: 87/8

    vping: 2/2

    ipv6: 25/25tempest: 3/6

    SDNVPN: 0/4 no SDN feature

    resiliency: 1/1

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/331304/consoleFull

    PASS/total

    defcore: 205/205

    HA: 8/8

    vping: 2/2

    ipv6: 2524/25

    tempest: 52/56

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-huaweibaremetal-pod7-proposed_tests-danube/18335/consoleFull

    PASS/total

    defcore: 205/205

    HA: 8/8

    vping: 2/2

    ipv6: 25/25

    tempest: 54/5 (Failed on testing port's admin state)

    SDNVPN: 0/4 no SDN feature

    		os-odl_l2-nofeature-ha Compass  

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetalhuawei-pod7-proposed_tests-danube/16319/consoleFull

    PASS/total

    defcoreosinterop: 204205/205

    HA: 8/8

    vping: 2/2

    ipv6: 25/25

    tempest: 4/5 (Failed on testing port's admin state)

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-huawei-baremetalpod7-proposed_tests-danube/19140/consoleFull

    PASS/total

    defcoreosinterop: 204205/205

    HA: 8/8

    vping: 2/2

    ipv6: 1725/25

    tempest: 4/5 (Failed on testing port's admin state)

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-huawei-virtualpod7-proposed_tests-danube/30932/consoleFull

    PASS/total

    defcoreosinterop: 204205/205

    HA: 78/8

    vping: 2/2

    ipv6: 25/25

    tempest: 4/5 (Failed on testing port's admin state)

    SDNVPN: 0/4 no SDN featureresiliency: 1/1

    os-odl_l3-nofeature-ha Compass 

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetalvirtual-proposed_tests-danube/304178/consoleFull

    PASS/total

    defcore: 205204/205

    HA: 87/8

    vping: 2/2

    tc007, SLA not passed

    ipv6: 2417/25

    tempest: 2/6

    tc018-tc025 fails

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/335159/consoleFull

    PASS/total

    defcore: 205204/205

    HA: 8/8vping: 2/2

    ipv6: 2517/25

    tempest: 4/5 (Failed on testing port's admin state)

    tc018-tc025 fails

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-huawei-pod7baremetal-proposed_tests-danube/19198/consoleFull

    PASS/total

    defcore: 205204/205

    HA: 8/8

    vping: 2/2

    ipv6: 25/25tempest: 4/5 (Failed on testing port's admin state)17/25 tc018-tc025 fails

    SDNVPN: 0/4 no SDN feature

    os-odl_l3-nofeature-ha 

    		Compass 

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtualbaremetal-proposed_tests-danube/178243/consoleFull

    PASS/total

    defcore: 204/205

    HA: 78/8 tc007, SLA not passed

    vping: 2/2

    ipv6: 17/25 tc018-tc025 fails

    SDNVPN: 0/4 no SDN feature

    resiliency: 0/1

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/159311/consoleFull

    PASS/total

    defcore: 204205/205

    HA: 8/8

    vping: 2/2

    ipv6: 17/25 tc018-tc025 fails

    tempest: 2/6

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/198326/consoleFull

    PASS/total

    defcore: 204205/205

    HA: 8/8

    vping: 2/2

    ipv6: 17/25 tc018-tc025 fails

    tempest: 3/5

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetalhuawei-pod7-proposed_tests-danube/24321/consoleFullconsole

    PASS/total

    defcoreosinterop: 204205/205

    HA: 8/8

    vping: 2/2

    ipv6: 17/25 tc018-tc025 fails

    tempest: 3/5

    SDNVPN: 0/4 no SDN featureresiliency: 0/1

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetalhuawei-pod7-proposed_tests-danube/31134/consoleFull

    PASS/total

    defcoreosinterop: 205/205

    HA: 8/8

    vping: 2/2

    ipv6: 17/25 tc018-tc025 fails

    tempest: 23/65

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-huawei-baremetalpod7-proposed_tests-danube/32633/consoleFull

    PASS/total

    defcoreosinterop: 205/205

    HA: 8/8

    vping: 2/2

    ipv6: 17/25 tc018-tc025 fails

    tempest: 3/5

    SDNVPN: 0/4 no SDN feature

     

    Fuel Scenarios Currently Test Against
    os-nosdn-nofeature-ha

    Fuel

    		      

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-fuel-zte-pod1-proposed_tests-danube/2/consoleFull

    PASS/total

    defcore: 205/205

    HA: 8/8

    vping: 2/2

    ipv6: 25/25

    tempest: 4/5

    SDNVPN: 0/4 no SDN feature

    		   
    os-odl_l2-bgpvpn-ha Fuel 

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-fuel-baremetal-proposed_tests-danube/45/consoleFull

    PASS/total

    defcore: 205/205

    HA: 8/8

    ipv6: 17/25 tc018-tc025 fails

    SDNVPN: 3/4 sdnvpn.tc001 "

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-fuel-baremetal-proposed_tests-danube/52/console

    PASS/total

    defcore: 205/205

    HA: 8/8

    ipv6: 17/25 tc018-tc025 fails

    SDNVPN: 4/4

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-fuel-baremetal-proposed_tests-danube/71/consoleFull

    PASS/total

    defcore: 205/205

    HA: 8/8

    ipv6: 17/25 tc018-tc025 fails

    SDNVPN: 4/4

    		      
    Other Scenarios

    os-onos-nofeature-ha 

    Don't test against this scenario since Compass will not release this scenario anymore.

    		Compass 

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtual-proposed_tests-danube/177/consoleFull

    PASS/total

    defcore: 150/171

    HA: 7/8 tc007, SLA not passed

    ipv6: 17/25 tc018-tc025 fails

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/160/consoleFull

    PASS/total

    defcore: 198/205

    HA: 8/8

    ipv6: 17/25 tc018-tc025 fails

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/195/consoleFull

    PASS/total

    defcore: 198/205

    HA: 8/8

    ipv6: 17/25 tc018-tc025 fails

    SDNVPN: 0/4 no SDN feature

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtual-proposed_tests-danube/307/consoleFull

    PASS/total

    defcore: 198/205

    HA: 7/8

    vping: 2/2

    ipv6: 17/25 tc018-tc025 fails

    SDNVPN: 0/4 no SDN feature

    resiliency: 0/1

    https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/312/consoleFull

    PASS/total

    defcore: 198/205

    HA: 5/8

    vping: 2/2

    ipv6: 17/25 tc018-tc025 fails

    tempest: 1/6

    SDNVPN: 0/4 no SDN feature

    		    

     

    cvp.0.5.0 Tempest Test Cases Results on Compass Danube

    ...

     

    Issue description

    Test case dovetail.ha.tc008 failed due to the HA settings on Apex.

    Reason

    This test case will fail on Apex danube SUT due to haproxy processes outage time is larger than 30s or the openstack service outage time is larger than 5s after killing all haproxy processes in one controller node.

    The failure is be caused by the HA setting of the SUT. It uses pacemaker to realize HA, and there are two settings that affect this test case.

    1. The haproxy. The default monitor interval time of it is 60 seconds, which means it will check haproxy every 60 seconds. haproxy processes will recover after it detects the failure. This interval time may cause the recover time to be more than 30 seconds sometimes.

    2. The float of virtual ip. The default monitor interval time of it is 10 seconds and the monitor timeout of it is 20 seconds. This means it will check the virtual ip every 10 seconds and float it to another controller node after 20 seconds when it detects the failure.

    3. All this make the openstack services recover time always larger than 5 seconds.

    Affect

    All SUTs use pacemaker and the settings of the recover time is too long.

    Status

    Suggest configure SUT' uses enlarge test case’s process outage time and service outage time to 60s.

    References

    1. Test case specification http://artifacts.opnfv.org/dovetail/docs/testing_user_testspecification_highavailability/index.html#test-case-8-controller-load-balance-as-a-service-high-availability


     

    Issue description

    Test case failed when using ODL as the network management provider.

    tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_cross_tenant_traffic

    Reason

    Refer to [1], this test case will fail on test assertion 1.

    Conntrack isn't enabled with odl legacy netvirt by default. With enabling conntracking, this test case passes.

    This is a bug of neworking-odl and the bug report link is [2].

    But this bug won’t fix, because conntrack requires relatively new version of odl and linux kernel. It is unstable with odl legacy netvirt.

    Affect

    All SUTs use odl and disable conntrack.

    Status

    Suggest testers enable conntrack.

    References

    1. test case specification http://artifacts.opnfv.org/dovetail/docs/testing_user_testspecification_securitygroup/index.html#test-case-2-test-security-group-cross-tenant-traffic

    2. bug report https://bugs.launchpad.net/networking-odl/+bug/1643033


     

    Issue description

    Test cases failed when using ODL as the network management provider.

    tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_update_instance_port_admin_state

    tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_update_router_admin_state

    Reason

    Refer to [1], these two test cases will fail on test assertion 3 and test assertion 2 respectively.

    The ping operation will still be successful after setting port’s/router’s admin_state_up attribute to be false.

    This is a bug of ODL Boron release and fixed with Carbon release.

    Affect

    All SUTs use odl Boron or before.

    Status

    None

    References

    1. test case specification http://artifacts.opnfv.org/dovetail/docs/testing_user_testspecification_dynamicnetwork/index.html#dynamic-network-runtime-operations-test-specification

    2. bug report

            https://bugs.opendaylight.org/show_bug.cgi?id=6718

            https://bugs.opendaylight.org/show_bug.cgi?id=4543


     

    Issue description

    Test cases failed when using ODL as the network management provider.

    ipv6.tc018 to ipv6.tc025

    Reason

    1. All these test cases try to ping6 to the router internal interface.
    2. In ODL Boron release, it supports IPv6 IPAM (IPAddress Assignment), IPv6 Security Groups and IPv6 east-west routing.
    3. Support for IPv6 north-south communication and ping6 support for neutron router-interface is currently under progress.

    Affect

    All SUTs use odl Boron or before.

    Status

    None

    References

    1. test case specification http://artifacts.opnfv.org/dovetail/docs/testing_user_testspecification_ipv6/index.html


     

    Issue description

    Test case is unstable on Fuel

    tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_port_security_disable_security_group

    Reason

    During out test, this test case sometimes passes but sometimes fails on Fuel.

    Affect

    unknonw

    Status

    None

    References

    1. test case specification http://artifacts.opnfv.org/dovetail/docs/testing_user_testspecification_securitygroup/index.html#test-case-5-test-port-security-disable-security-group



    Issue description

    SDNVPN Test cases won't clean environment when it throws an exception during the test

    sdnvpn.tc001, sdnvpn.tc002, sdnvpn.tc004 and sdnvpn.tc008

    Reason

    When running an sdnvpn test case on an non-sdnvpn SUT, it will throw an exception and exit without cleanning resources it created.

    Affect

    All SUTs with no sdnvpn feature but still run sdnvpn test cases.

    Status

    Will be fixed in SDNVPN E release. Dovetail uses Functest clean module to clean temporarily.

    References

    1. JIRA ticket https://jira.opnfv.org/browse/SDNVPN-184

     

    ...