Status of proposed tests

2018-06-25

Mandatory Tests

Installer	Scenario	vping_userdata	vping_ssh	stress.ping	tempest.osinterop	tempest.compute	tempest.identity_v3	tempest.image	tempest.network_api	tempest.volume	tempest.trunk_ports	tempest.ipv6_api	security.patrole	ha.nova_api	ha.neutron_server	ha.keystone	ha.glance_api	ha.cinder_api	ha.cpu_load	ha.disk_load	ha.haproxy	ha.rabbitmq	ha.database

Installer

Scenario

vping_userdata

vping_ssh

stress.ping

tempest.osinterop

tempest.compute

tempest.identity_v3

tempest.image

tempest.network_api

tempest.volume

tempest.trunk_ports

tempest.ipv6_api

security.patrole

ha.nova_api

ha.neutron_server

ha.keystone

ha.glance_api

ha.cinder_api

ha.cpu_load

ha.disk_load

ha.haproxy

ha.rabbitmq

ha.database

Apex

nosdn-nofeature

fraser

PASS

FAIL

https://lf-anuket.atlassian.net/browse/DOVETAIL-662

PASS

FAIL

HA proxy processes don't recover

PASS

Compass

nosdn-nofeature

fraser

PASS

FAIL

cannot connect to 192.168.11.xxx

PASS

total: 200

PASS

FAIL

https://lf-anuket.atlassian.net/browse/DOVETAIL-738

PASS

Fuel

nosdn-nofeature

master

PASS

FAIL

SLA failure:
10.725534 > 5.000000

UNSTABLE

most of the time the test detects 11 keystone processes of which only 9 recover

Is the keystone process query correct?

PASS

FAIL

SLA failure:
12.149971 > 5.000000

PASS

FAIL

SLA failure:
12.336285 > 5.000000

rabbitmq-server not found
in the host environment

FAIL

SLA failure: 13.879296 > 5.000000

2018-06-25

Installer	Scenario	stress.ping	tempest.compute	tempest.identity_v2	tempest.identity_v3	tempest.image	tempest.network_api	tempest.volume	tempest.trunk_ports	security.patrole	ha.controller_restart	ha.rabbitmq	ha.database	ha.neutron_l3_agent	tempest.network_scenario	vnf.vEPC	vnf.vIMS	tempest.bgpvpn	snaps.smoke

Installer	Scenario	stress.ping	tempest.compute	tempest.identity_v2	tempest.identity_v3	tempest.image	tempest.network_api	tempest.volume	tempest.trunk_ports	security.patrole	ha.controller_restart	ha.rabbitmq	ha.database	ha.neutron_l3_agent	tempest.network_scenario	vnf.vEPC	vnf.vIMS	tempest.bgpvpn	snaps.smoke
Apex	nosdn-nofeature stable/fraser	FAIL	PASS	FAIL - only v3 used skip mechanism needed or just part of reviewer instructions (does not need to be automated) https://lf-anuket.atlassian.net/browse/DOVETAIL-656	PASS	PASS	PASS	PASS	PASS	PASS	FAIL IPMI info missing	PASS	PASS	PASS	PASS	FAIL passes on Apex daily loop	PASS	not supported	FAIL `test_volume_type_with_encryption` `test_volume_type_with_qos_and_encryption` passes on Apex daily CI loop
	odl stable/fraser	FAIL	FAIL	FAIL - only v3 used skip mechanism needed or just part of reviewer instructions (does not need to be automated)			FAIL	FAIL	FAIL	FAIL	FAIL IPMI info missing	PASS	PASS	not supported: l3 agent not installed	FAIL	FAIL	FAIL	FAIL	FAIL
	nosdn-nofeature master	FAIL	PASS	not supported	PASS	PASS	PASS	PASS	PASS	PASS	FAIL IPMI info missing	PASS	PASS	PASS	PASS	PASS	FAIL VM in error state: "no valid host found" passes on Apex daily CI loop	not supported	FAIL
Compass	nosdn-nofeature stable/fraser	PASS create stack rate 5/5	PASS	SKIP	PASS	PASS	PASS	PASS	PASS skip: 9 pass: 43	PASS skip: 3 (dvr is not enabled) pass: 132 `test_create_distributed_router ... skip` `test_show_distributed_router ... skip` `test_update_distributed_router ... skip`	PASS	PASS	PASS	FAIL `SLA failure: 9.023325 > 5.000000`	FAIL fail: 5 `No IPv4 addresses found`	PASS	PASS	not supported	FAIL fail: 1 pass: 120 `test_ssh_client_fip_after_reboot ... fail`
Compass	odl stable/fraser	PASS create stack rate 5/5	PASS	SKIP	PASS	PASS	PASS	PASS	PASS skip: 9 pass: 43	PASS skip: 5 (l3_agent_scheduler extension not enabled) pass: 130 `test_create_router_on_l3_agent ... skip` `test_delete_router_from_l3_agent ... skip` `test_list_routers_on_l3_agent ... skip` `test_create_high_availability_router ... skip` `test_update_high_availability_router ... skip`	PASS	PASS	PASS	not supported: l3 agent not installed	FAIL fail: 2 (False is not true : ip address 192.168.11.114 is reachable) test_update_instance_port_admin_state test_update_router_admin_state	PASS	PASS	not supported	FAIL fail: 2 pass: 119 `test_create_network_router_admin_user_to_new_project ... fail` `test_create_network_router_new_user_to_admin_project ... fail`
Fuel (x86)	nosdn-nofeature master	PASS	PASS	FAIL - only v3 used skip mechanism needed or just part of reviewer instructions (does not need to be automated) https://lf-anuket.atlassian.net/browse/DOVETAIL-656	PASS	PASS	PASS	PASS	PASS skip: 9 pass: 43	PASS skip: 3 (dvr is not enabled) pass: 132 `test_create_distributed_router ... skip` `test_show_distributed_router ... skip` `test_update_distributed_router ... skip`	PASS	FAIL rabbitmq located in msg0x VMs https://docs.opnfv.org/en/stable-fraser/submodules/fuel/docs/release/installation/installation.instruction.html#automatic-installation-of-a-baremetal-pod	FAIL database located in dbs0x VMs https://docs.opnfv.org/en/stable-fraser/submodules/fuel/docs/release/installation/installation.instruction.html#automatic-installation-of-a-baremetal-pod	FAIL l3-agent in gtw0x VMs https://gerrit.opnfv.org/gerrit/#/c/58333/	FAIL `- Success: 3 - Skipped: 0 - Expected failures: 0 - Unexpected success: 0 - Failures: 2` `test_update_instance_port_admin_state ... fail` `test_update_router_admin_state ... fail`	FAIL auth problem passes on FUEL daily CI loop	FAIL auth problem `deployment clearwater-opnfv timed out`	not supported	PASS

Installer

Scenario

vping_ssh

vping_userdata

tempest.osinterop

tempest.ipv6

tempest.multi_node_scheduling

tempest.security

tempest.vm_lifecycle

ha.nova_api

ha.neutron_server

ha.keystone

ha.glance_api

ha.cinder_api

ha.cpu_load

ha.disk_load

ha.haproxy

sdnvpn.subnet_connectivity

sdnvpn.tenant_separation

sdnvpn.router_assoc

sdnvpn.router_assoc_fip

Apex

nosdn-nofeature

stable/fraser

PASS

FAIL

passes on Apex daily loop

FAIL