this wiki page is used to record the running histroy for the dovetil tool.this may cover the runnning in the plugfest, CI and vendors lab.
1. Version change logs
For each dovetail release version, the docker images used are introduced first, then problems found and fixed are listed as well as the updates.
1.1 cvp.0.1.0 (2017/06/23)
1.1.1 Docker images used:
opnfv/dovetail:cvp.0.1.0 (commit ID 041a51d6d3ba5f2d9eabcf3501abb9af30ebc394)
opnfv/yardstick:danube.2.0
opnfv/functest:danube.2.0
opnfv/testapi:cvp.0.1.0 (Commit ID 056b7ccefe38b13e4b4eb11ff7e259e254e35c32)
1.1.2 Updates and problems found and fixed in this version
Image | Updates and problems found and fixed | JIRA | Gerrit |
---|---|---|---|
opnfv/dovetail:cvp.0.1.0 |
| ||
opnfv/yardstick:danube.2.0 |
| ||
opnfv/functest:danube.2.0 | SDNVPN test cases are not enable in Functest | None | |
opnfv/testapi:cvp.0.1.0 | user management and some bugfix | None | None |
1.2 cvp.0.2.0 (2017/07/07)
1.2.1 Docker images used:
opnfv/dovetail:cvp.0.2.0 (commit ID 5ddf932dc28bcc47169c3091267d57ba5a99f9b2)
opnfv/yardstick:danube.3.0
opnfv/functest:cvp.0.2.0 (commit ID 3d03bbcfc45d00d4ce995d8aabff5808accb0687)
opnfv/testapi:cvp.0.2.0 (commit ID a7f82c093fab2ad19c7ffb0a81ec5756e91e73ae)
1.2.2 Updates and problems found and fixed in this version
Image | Updates and problems found and fixed | JIRA | Gerrit |
---|---|---|---|
opnfv/dovetail:cvp.0.2.0 |
| ||
opnfv/yardstick:danube.3.0 |
| None | |
opnfv/functest:cvp.0.2.0 | None | None | None |
opnfv/testapi:cvp.0.2.0 |
|
|
1.3 cvp.0.3.0 (2017/7/21)
1.3.1 Docker images used:
opnfv/dovetail:cvp.0.3.0 (commit ID 7ed1cf189676693d84c9d48bb964c178b16bae76)
opnfv/yardstick:danube.3.1
opnfv/functest:cvp.0.2.0 (commit ID 3d03bbcfc45d00d4ce995d8aabff5808accb0687)
opnfv/testapi:cvp.0.3.0 (commit ID 4e0b74b0141e3ae5d250006f3d75fcc880738cc9)
1.3.2 Updates and problems found and fixed in this version
Image | Updates and problems found and fixed | JIRA | Gerrit |
---|---|---|---|
opnfv/dovetail:cvp.0.3.0 |
| ||
opnfv/yardstick:danube.3.1 | None | None | None |
opnfv/functest:cvp.0.2.0 | None | None | None |
opnfv/testapi:cvp.0.3.0 |
| None | None |
1.4 cvp.0.4.0 (2017/8/4)
1.4.1 Docker images used:
opnfv/dovetail:cvp.0.4.0 (commit ID a8d7665ca783684bc10d7c3dbbc606e30f083bc7)
opnfv/yardstick:danube.3.1
opnfv/functest:cvp.0.2.0 (commit ID 3d03bbcfc45d00d4ce995d8aabff5808accb0687)
opnfv/testapi:cvp.0.4.0
opnfv/bottlenecks:cvp.0.4.0 (commit ID 3592f9016821a603930ebda7f4112abf732e6750)
1.4.2 Updates and problems found and fixed in this version
Image | Updates and problems found and fixed | JIRA | Gerrit |
---|---|---|---|
opnfv/dovetail:cvp.0.4.0 |
| ||
opnfv/yardstick:danube.3.1 | Bugfix: cannot run task if without yardstick.conf in danube | YARDSTICK-776 | https://gerrit.opnfv.org/gerrit/#/c/38955/ |
opnfv/functest:cvp.0.2.0 |
| ||
opnfv/testapi:cvp.0.4.0 |
| ||
opnfv/bottlenecks:cvp.0.4.0 |
1.5 cvp.0.5.0 (2017/8/18)
1.5.1 Docker images used:
opnfv/dovetail:cvp.0.5.0 (commit ID 474795c11c556dcd325559853fe7ed436a69418c)
opnfv/yardstick:danube.3.2 (commit ID e9338f29a70a38a5838498adf79ec91f92d085d1)
opnfv/functest:cvp.0.5.0 (commit ID 13067de652d6ccdcfc85f0f5e65841b644d9a6e9)
opnfv/bottlenecks:cvp.0.4.0 (commit ID 3592f9016821a603930ebda7f4112abf732e6750)
opnfv/testapi:cvp.0.5.0
opnfv/dovetail:nginx.cvp.0.5.0
1.5.2 Updates and problems found and fixed in this version
1.6 cvp.0.6.0 (2017/9/8)
1.6.1 Docker images used:
opnfv/dovetail:cvp.0.6.0 (commit ID 0aacdceeeac62333651ec5440fb32af965c7689e)
opnfv/yardstick:danube.3.2 (commit ID e9338f29a70a38a5838498adf79ec91f92d085d1)
opnfv/functest:cvp.0.5.0 (commit ID 13067de652d6ccdcfc85f0f5e65841b644d9a6e9)
opnfv/bottlenecks:cvp.0.4.0 (commit ID 3592f9016821a603930ebda7f4112abf732e6750)
opnfv/testapi:cvp.0.5.0
opnfv/dovetail:web.cvp.0.6.0
opnfv/dovetail:api.cvp.0.6.0
1.6.2 Updates and problems found and fixed in this version
Image | Updates and problems found and fixed | JIRA | Gerrit |
---|---|---|---|
opnfv/dovetail:cvp.0.6.0 | Disable get DB schema in Dockerfile |
| |
opnfv/yardstick:danube.3.2 | None | None | None |
opnfv/functest:cvp.0.5.0 | None | None | None |
opnfv/testapi:cvp.0.5.0 | |||
opnfv/dovetail:nginx.cvp.0.6.0 |
1.7 cvp.0.7.0 (2017/9/29)
1.7.1 Docker images used:
opnfv/dovetail:cvp.0.7.0 (commit ID 043ef2dad58c3aef0ea91cf3063dc9deb6f40bd9)
opnfv/yardstick:danube.3.2 (commit ID e9338f29a70a38a5838498adf79ec91f92d085d1)
opnfv/functest:cvp.0.5.0 (commit ID 13067de652d6ccdcfc85f0f5e65841b644d9a6e9)
opnfv/bottlenecks:cvp.0.4.0 (commit ID 3592f9016821a603930ebda7f4112abf732e6750)
opnfv/testapi:cvp.0.5.0
opnfv/dovetail:web.cvp.0.7.0
opnfv/dovetail:api.cvp.0.7.0
in order to make the dovetail tool more stable, the dovetail has been integreated in the CI for daily running.
for more detil, please refer to link: https://build.opnfv.org/ci/view/dovetail/
3. Running against scenarios by owners
scenarios | installer | owners | cvp.0.1.0 | cvp.0.2.0 | cvp.0.3.0 | cvp.0.4.0 | cvp.0.5.0 | cvp.0.6.0 |
---|---|---|---|---|---|---|---|---|
Apex Scenarios Currently Test Against | ||||||||
os-nosdn-nofeature-ha | Apex | PASS/total defcore: 204/205 HA: 7/8 vping: 2/2 ipv6: 25/25 tempest: 4/6 SDNVPN: 0/4 no SDN feature | ||||||
Compass Scenarios Currently Test Against | ||||||||
os-nosdn-nofeature-ha | Compass | PASS/total defcore: 204/205 HA: 7/8 tc007, SLA not passed ipv6: 25/25 SDNVPN: 0/4 no SDN feature | PASS/total defcore: 204/205 HA: 8/8 ipv6: 25/25 SDNVPN: 0/4 no SDN feature | PASS/total defcore: 204/205 HA: 8/8 ipv6: 25/25 SDNVPN: 0/4 no SDN feature | PASS/total defcore: 204/205 HA: 8/8 vping: 2/2 ipv6: 25/25 SDNVPN: 0/4 no SDN feature resiliency: 1/1 | PASS/total defcore: 205/205 HA: 8/8 vping: 2/2 ipv6: 25/25 tempest: 3/6 SDNVPN: 0/4 no SDN feature | PASS/total defcore: 205/205 HA: 8/8 vping: 2/2 ipv6: 25/25 tempest: 5/5 SDNVPN: 0/4 no SDN feature | |
os-odl_l2-nofeature-ha | Compass | PASS/total defcore: 204/205 HA: 8/8 ipv6: 25/25 SDNVPN: 0/4 no SDN feature | PASS/total defcore: 204/205 HA: 8/8 ipv6: 17/25 SDNVPN: 0/4 no SDN feature | PASS/total defcore: 204/205 HA: 7/8 vping: 2/2 ipv6: 25/25 SDNVPN: 0/4 no SDN feature resiliency: 1/1 | PASS/total defcore: 205/205 HA: 8/8 vping: 2/2 ipv6: 24/25 tempest: 2/6 SDNVPN: 0/4 no SDN feature | PASS/total defcore: 205/205 HA: 8/8 vping: 2/2 ipv6: 25/25 tempest: 4/5 (Failed on testing port's admin state) SDNVPN: 0/4 no SDN feature | ||
os-odl_l3-nofeature-ha | Compass | PASS/total defcore: 204/205 HA: 7/8 tc007, SLA not passed ipv6: 17/25 tc018-tc025 fails SDNVPN: 0/4 no SDN feature | PASS/total defcore: 204/205 HA: 8/8 ipv6: 17/25 tc018-tc025 fails SDNVPN: 0/4 no SDN feature | PASS/total defcore: 204/205 HA: 8/8 ipv6: 17/25 tc018-tc025 fails SDNVPN: 0/4 no SDN feature | PASS/total defcore: 204/205 HA: 8/8 vping: 2/2 ipv6: 17/25 tc018-tc025 fails SDNVPN: 0/4 no SDN feature resiliency: 0/1 | PASS/total defcore: 205/205 HA: 8/8 vping: 2/2 ipv6: 17/25 tc018-tc025 fails tempest: 2/6 SDNVPN: 0/4 no SDN feature | ||
Fuel Scenarios Currently Test Against | ||||||||
os-nosdn-nofeature-ha | Fuel (zte-pod1) | PASS/total defcore: 205/205 HA: 8/8 vping: 2/2 ipv6: 25/25 tempest: 4/5 SDNVPN: 0/4 no SDN feature | ||||||
os-odl_l2-bgpvpn-ha | Fuel | PASS/total defcore: 205/205 HA: 8/8 ipv6: 17/25 tc018-tc025 fails SDNVPN: 3/4 sdnvpn.tc001 " | PASS/total defcore: 205/205 HA: 8/8 ipv6: 17/25 tc018-tc025 fails SDNVPN: 4/4 | PASS/total defcore: 205/205 HA: 8/8 ipv6: 17/25 tc018-tc025 fails SDNVPN: 4/4 | ||||
Other Scenarios | ||||||||
os-nosdn-kvm-ha | Apex | |||||||
os-odl_l3-fdio-ha | Apex | |||||||
os-onos-nofeature-ha Don't test against this scenario since Compass will not release this scenario anymore. | Compass |
|
|
|
|
| ||
os-odl_l2-nofeature-ha | Fuel(ZTE-POD1) | |||||||
os-nosdn-ovs-ha | Fuel | |||||||
os-odl_l3-nofeature-ha | Fuel | |||||||
os-nosdn-kvm_ovs_dpdk-ha | Fuel |
cvp.0.5.0 Tempest Test Cases Results on Compass Danube
Dovetail Test Cases Run Results for Apex, Compass and Fuel
4. try by vendors
in order to make the dovetail tool and douments more stable, we are trying more enviroments inluding no OPNFV enviroments
company | cloudOS | feedbacks |
---|---|---|
ZTE | ZTE TECS | |
Wind River | Wind River Titanium 3 based on Mitaka with AVS | |
Huawei | FS | |
Redhat | ||
BII | OPNFV Distr |
5. Running in the OPNFV plugfest(orange)
hardware | installer | scenarios | dovetail test cases | results | owner |
---|---|---|---|---|---|
Nokia hardware | apex | os-nosdn-nofeature-ha | defcore and IPV6 | volume_device_name = vdb or vdc or others | Rautakumpu, Mika |
Huawei hardware | compass | os-nosdn-nofeature-ha | defcore and IPV6 | all pass | hongbo tian |
lenovo | Joid | os-nosdn-nofeature-ha | defcore and IPV6 | all pass | Narinder Gupta |
lenovo | Wind River Titanium 3 based on Mitaka with AVS | defcore and IPV6 | all pass | Raineri, Eddy |