Versions Compared

Version Old Version 28 New Version Current
Changes made by

Georg Kunz

Dan Xu

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


2018

...

-06-25

Mandatory Tests

InstallerScenariovping_userdatavping_sshstress.pingtempest.osinteroptempest.computetempest.identity_v3tempest.imagetempest.network_apitempest.volumetempest.trunk_portstempest.ipv6_apisecurity.patroleha.nova_apiha.neutron_serverha.keystoneha.glance_apiha.cinder_apiha.cpu_loadha.disk_loadha.haproxyha.rabbitmqha.database
Apex

nosdn-nofeature

fraser

PASSPASS
PASS

FAIL

Jira Legacy
serverSystem Jira
serverId1afe526e-48e5-33b1-8ed7-4f559eac1ef8
keyDOVETAIL-662

PASS
PASS

PASS

PASS

PASS

PASS

PASS

PASS

PASS
PASS
PASSPASS
PASS
PASS
PASS

FAIL

HA proxy processes don't recover

PASS
PASS
Compass

nosdn-nofeature

fraser

PASS

FAIL

cannot connect to 192.168.11.xxx

PASS

PASS

total: 200

PASSPASSPASSPASSPASSPASSPASSPASSPASS

PASS

PASSPASSPASS

PASS

PASSPASS

FAIL

Jira Legacy
serverSystem Jira
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId1afe526e-48e5-33b1-8ed7-4f559eac1ef8
keyDOVETAIL-738

PASS

Fuel

nosdn-nofeature

master

PASSPASSPASSPASSPASSPASSPASSPASSPASSPASS

PASS

PASSPASS

FAIL

SLA failure:
10.725534 > 5.000000

UNSTABLE

most of the time the test detects 11 keystone processes of which only 9 recover

Is the keystone process query correct?

PASSPASS

FAIL

SLA failure:
12.149971 > 5.000000
PASSPASS

FAIL

SLA failure:
12.336285 > 5.000000
rabbitmq-server not found
in the host environment

FAIL

SLA failure: 13.879296 > 5.000000



2018-06-25

FAIL

passes on Apex daily loop

FAIL

IPMI info missingodl

stable/fraser

FAIL
PASS

IPMI info missing

CompassPASS

failed to create stacks (Time out)

create stack rate 0/20

FAIL

fail: 4

pass: 15

test_hotplug_nic ... fail
test_network_basic_ops ... fail
test_subnet_details ... fail
test_update_instance_port_admin_state ... failnot supported. skip

FAIL

FAIL

database located in dbs0x VMs

https://docs.opnfv.org/en/stable-fraser/submodules/fuel/docs/release/installation/installation.instruction.html#automatic-installation-of-a-baremetal-pod
InstallerScenariostress.pingtempest.computetempest.identity_v2tempest.identity_v3tempest.imagetempest.network_apitempest.volumetempest.trunk_portssecurity.patroleha.controller_restartha.rabbitmqha.databaseha.neutron_l3_agenttempest.network_scenariovnf.vEPCvnf.vIMStempest.bgpvpnsnaps.smoke
Apex

nosdn-nofeature

stable/fraser

FAIL

PASS

FAIL - only v3 used

skip mechanism needed or just part of reviewer instructions (does not need to be automated)

Jira Legacy
serverSystem Jira
serverId1afe526e-48e5-33b1-8ed7-4f559eac1ef8
keyDOVETAIL-656

PASSPASSPASSPASSPASSPASS

FAIL

IPMI info missing

PASSPASSPASSPASS

FAIL


passes on Apex daily loop

PASS

not supported

FAIL

test_volume_type_with_encryption 
test_volume_type_with_qos_and_encryption

passes on Apex daily CI loop

odl

stable/fraser

FAILFAIL

FAIL - only v3 used

skip mechanism needed or just part of reviewer instructions (does not need to be automated)



FAILFAILFAILFAIL

FAIL

IPMI info missing

PASSPASS

not supported:

l3 agent not installed

FAILFAILFAIL

FAIL

FAIL

nosdn-nofeature

master

FAILPASSnot supportedPASSPASSPASSPASSPASSPASS

FAIL

IPMI info missing

PASSPASSPASSPASSPASS

FAIL

VM in error state: "no valid host found"

passes on Apex daily CI loop

not supported

FAIL
Compass

nosdn-nofeature

stable/fraser

PASS

create stack rate 5/5

PASS

SKIP

PASSPASS

PASS

FAIL

test_volume_type_with_encryption 
test_volume_type_with_qos_and_encryption

FAIL

PASS

FAIL - only v3 used

skip mechanism needed or just part of reviewer instructions (does not need to be automated)

Jira Legacy
serverSystem Jira
serverId1afe526e-48e5-33b1-8ed7-4f559eac1ef8
keyDOVETAIL-656

PASSPASSPASS

PASS

PASS

skip: 9

pass: 43

PASS

skip: 3 (dvr is not enabled)

pass: 132

test_create_distributed_router ... skip
test_show_distributed_router ... skip
test_update_distributed_router ... skip


PASSPASSPASS

FAIL

SLA failure: 9.023325 > 5.000000

FAIL

fail: 5

No IPv4 addresses found
PASSPASSnot supported

FAIL

fail: 1

pass: 120

test_ssh_client_fip_after_reboot ... fail

odl

stable/fraser

PASS

create stack rate 5/5


PASS

SKIPPASSPASS

PASS

PASSFAILFAILFAILFAILFAIL

FAIL - only v3 used

skip mechanism needed or just part of reviewer instructions (does not need to be automated)

FAILFAILFAIL

FAIL

FAIL

FAIL

IPMI info missing

PASS

not supported:

l3 agent not installed

PASS

nosdn-nofeature

master

FAIL

VM in error state: "no valid host found"

passes on Apex daily CI loop

PASSFAILPASSnot supportedPASSPASSPASS

PASS

skip: 9

pass: 43

PASS

skip: 5 (l3_agent_scheduler extension not enabled)

pass: 130

test_create_router_on_l3_agent ... skip
test_delete_router_from_l3_agent ... skip
test_list_routers_on_l3_agent ... skip
test_create_high_availability_router ... skip
test_update_high_availability_router ... skip



PASSPASSPASS

not supported:

l3 agent not installed

FAIL

fail: 2

(False is not true : ip address 192.168.11.114 is reachable)

test_update_instance_port_admin_state

test_update_router_admin_state



PASSPASS

not supported

FAIL

PASSPASSPASS

fail: 2

pass: 119

test_create_network_router_admin_user_to_new_project ... fail
test_create_network_router_new_user_to_admin_project ... fail


Fuel (x86)

nosdn-nofeature

stable/fraser

master

PASSPASS

FAIL

PASS

SKIP

PASSPASS

- only v3 used

skip mechanism needed or just part of reviewer instructions (does not need to be automated)

Jira Legacy
serverSystem Jira
serverId1afe526e-48e5-33b1-8ed7-4f559eac1ef8
keyDOVETAIL-656

PASSPASSPASSPASS

PASS

skip: 9

pass: 43

PASS

skip: 3 (dvr is not enabled)

pass: 132

test_create_distributed_router ... skip
test_show_distributed_router ... skip
test_update_distributed_router ... skip

PASS

FAIL

rabbitmq located in msg0x VMs

https://docs.opnfv.org/en/stable-fraser/submodules/fuel/docs/release/installation/installation.

PASSPASS

FAIL

SLA failure: 9.023325 > 5.000000
PASS

odl

stable/fraser

PASSPASS

FAIL

fail: 2

pass: 117

FAIL

failed to create stacks

PASS

SKIPPASSPASS

FAIL

fail: 3

pass: 17

PASS

PASS

skip: 9

pass: 43

not supported

PASS

skip: 5

pass: 130

PASSPASS

not supported:

l3 agent not installed

PASS
Fuel (x86)

nosdn-nofeature

master

instruction.html#automatic-installation-of-a-baremetal-pod

FAIL

database located in dbs0x VMs

https://docs.opnfv.org/en/stable-fraser/submodules/fuel/docs/release/installation/installation.instruction.html#automatic-installation-of-a-baremetal-pod

FAIL

l3-agent in gtw0x VMs

https://gerrit.opnfv.org/gerrit/#/c/58333/

FAIL

 - Success: 3
 - Skipped: 0
 - Expected failures: 0
 - Unexpected success: 0
 - Failures: 2
test_update_instance_port_admin_state ... fail
test_update_router_admin_state ... fail

FAIL

auth problem

passes on FUEL daily CI loop

FAIL

auth problem

deployment clearwater-opnfv timed out


not supported

FAIL

passes on FUEL daily CI loop

PASSFAILFAIL

FAIL - only v3 used

skip mechanism needed or just part of reviewer instructions (does not need to be automated)

Jira Legacy
serverSystem Jira
serverId1afe526e-48e5-33b1-8ed7-4f559eac1ef8
keyDOVETAIL-656

PASSPASS

FAIL

 - Success: 18
 - Skipped: 0
 - Expected failures: 0
 - Unexpected success: 0
 - Failures: 1 
test_hotplug_nic ... fail
PASS

PASS

skip: 9

pass: 43

not supported

PASS

skip: 3 (dvr is not enabled)

pass: 132

test_create_distributed_router ... skip
test_show_distributed_router ... skip
test_update_distributed_router ... skip

PASS

FAIL

rabbitmq located in msg0x VMs

https://docs.opnfv.org/en/stable-fraser/submodules/fuel/docs/release/installation/installation.instruction.html#automatic-installation-of-a-baremetal-pod

FAIL

l3-agent in gtw0x VMs

https://gerrit.opnfv.org/gerrit/#/c/58333/

PASS


InstallerScenariovping_sshvping_userdatatempest.osinteroptempest.ipv6tempest.multi_node_schedulingtempest.securitytempest.vm_lifecycleha.nova_apiha.neutron_serverha.keystoneha.glance_apiha.cinder_apiha.cpu_loadha.disk_loadha.haproxysdnvpn.subnet_connectivitysdnvpn.tenant_separationsdnvpn.router_assocsdnvpn.router_assoc_fip
Apex

nosdn-nofeature

stable/fraser

PASS

FAIL

passes on Apex daily loop

FAIL

passes on Apex daily CI loop

FAIL

FAIL

FAIL

FAILPASSFAILPASSPASS

FAIL

PASS

FAIL

IPMI info missing

PASSnot supportednot supportednot supportednot supported
Fuelnosdn-nofeaturePASSPASSFAILFAILFAILFAILFAILPASSPASSFAILPASSFAILPASSPASSPASSnot supportednot supportednot supportednot supported







Historical results below

2018-05-23 - limited results due to failed CI jobs

...