Page Comparison

2018

...

stable/fraser

...

FAIL

ERROR - abot-epc-basic
workload differs from active

passes on Apex daily CI loop

...

FAIL

https://gerrit.opnfv.org/gerrit/#/c/58481/

...

FAIL - only v3 used

skip mechanism needed or just part of reviewer instructions (does not need to be automated)

Jira Legacy

server	System Jira
serverId	1afe526e-48e5-33b1-8ed7-4f559eac1ef8
key	DOVETAIL-656

...

FAIL

 - Success: 19
 - Skipped: 0
 - Expected failures: 0
 - Unexpected success: 0
 - Failures: 1

test_mtu_sized_frames ... fail

=> remove test

...

not supported

...

FAIL

IPMI info missing

...

FAIL

wrong process name

https://gerrit.opnfv.org/gerrit/#/c/58317/

...

FAIL

stack create failed

https://gerrit.opnfv.org/gerrit/#/c/58333/

flavor m1.tiny does not exist on Apex deployments by default

https://gerrit.opnfv.org/gerrit/#/c/58601/

...

odl

nstable/fraser

FAIL

one test is broken due to a bug in ODL driver

...

-06-25

Mandatory Tests

Installer

Scenario

vping_userdata

vping_ssh

stress.ping

tempest.osinterop

tempest.compute

tempest.identity_v3

tempest.image

tempest.network_api

tempest.volume

tempest.trunk_ports

tempest.ipv6_api

security.patrole

ha.nova_api

ha.neutron_server

ha.keystone

ha.glance_api

ha.cinder_api

ha.cpu_load

ha.disk_load

ha.haproxy

ha.rabbitmq

ha.database

Apex

nosdn-nofeature

fraser

PASS

FAIL

Jira Legacy

server	System Jira
serverId	1afe526e-48e5-33b1-8ed7-4f559eac1ef8
key	DOVETAIL-662

PASS

FAIL

HA proxy processes don't recover

PASS

Compass

nosdn-nofeature

fraser

PASS

FAIL

cannot connect to 192.168.11.xxx

PASS

total: 200

PASS

FAIL

Jira Legacy

server	System Jira
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	1afe526e-48e5-33b1-8ed7-4f559eac1ef8
key	DOVETAIL-738

PASS

Fuel

nosdn-nofeature

master

PASS

FAIL

SLA failure:
10.725534 > 5.000000

UNSTABLE

most of the time the test detects 11 keystone processes of which only 9 recover

Is the keystone process query correct?

PASS

FAIL

SLA failure:
12.149971 > 5.000000

PASS

FAIL

SLA failure:
12.336285 > 5.000000

rabbitmq-server not found
in the host environment

FAIL

SLA failure: 13.879296 > 5.000000

2018-06-25

PASS

failed to create stacks (Time out)

create stack rate 0/20

FAIL

fail: 4

pass: 15

test_hotplug_nic ... fail

test_network_basic_ops ... fail

test_subnet_details ... fail

test_update_instance_port_admin_state ... failnot supportedFAIL

FAIL

database located in dbs0x VMs

https://docs.opnfv.org/en/stable-fraser/submodules/fuel/docs/release/installation/installation.instruction.html#automatic-installation-of-a-baremetal-pod

Installer

Scenario

stress.ping

tempest.compute

tempest.identity_v2

tempest.identity_v3

tempest.image

tempest.network_api

tempest.volume

tempest.trunk_ports

security.patrole

ha.controller_restart

ha.rabbitmq

ha.database

ha.neutron_l3_agent

tempest.network_scenario

vnf.vEPC

vnf.vIMS

tempest.bgpvpn

snaps.smoke

Apex

nosdn-nofeature

stable/fraser

FAIL

PASS

FAIL - only v3 used

skip mechanism needed or just part of reviewer instructions (does not need to be automated)

Jira Legacy

server	System Jira
serverId	1afe526e-48e5-33b1-8ed7-4f559eac1ef8
key	DOVETAIL-656

PASS

FAIL

IPMI info missing

PASS

FAIL

passes on Apex daily loop

PASS

not supported

FAIL

test_volume_type_with_encryption

test_volume_type_with_qos_and_encryption

passes on Apex daily CI loop

odl

stable/fraser

FAIL

FAIL - only v3 used

skip mechanism needed or just part of reviewer instructions (does not need to be automated)

FAIL

IPMI info missing

PASS

not supported:

l3 agent not installed

FAIL

nosdn-nofeature

master

FAIL

PASS

not supported

PASS

FAIL

IPMI info missing

PASS

FAIL

VM in error state: "no valid host found"

passes on Apex daily CI loop

not supported

FAIL

Compass

nosdn-nofeature

stable/fraser

PASS

create stack rate 5/5

PASS

SKIP

PASS

skip: 9

pass: 43

PASS

skip: 3 (dvr is not enabled)

pass: 132

test_create_distributed_router ... skip

test_show_distributed_router ... skip

test_update_distributed_router ... skip

PASS

FAIL

SLA failure: 9.023325 > 5.000000

FAIL

fail: 5

No IPv4 addresses found

PASS

not supported

FAIL

fail: 1

pass: 120

test_ssh_client_fip_after_reboot ... fail

odl

stable/fraser

PASS

create stack rate 5/5

PASS

SKIP

PASS

skip: 9

pass: 43

PASS

skip: 5 (l3_agent_scheduler extension not enabled)

pass: 130

test_create_router_on_l3_agent ... skip

test_delete_router_from_l3_agent ... skip

test_list_routers_on_l3_agent ... skip

test_create_high_availability_router ... skip

test_update_high_availability_router ... skip

PASS

not supported:

l3 agent not installed

PASS

nosdn-nofeature

master

FAIL

VM in error state: "no valid host found"

passes on Apex daily CI loop

FAIL

VM in error state: "no valid host found"

passes on Apex daily CI loop

FAIL

test_volume_type_with_encryption

test_volume_type_with_qos_and_encryption

FAIL

PASS

not supported

PASS

not supported

PASS

FAIL

IPMI info missing

PASS

Compass

nosdn-nofeature

stable/fraser

FAIL

fail: 2

(False is not true : ip address 192.168.11.114 is reachable)

test_update_instance_port_admin_state

test_update_router_admin_state

PASS

not supported

FAIL

fail: 2

pass: 119

test_create_network_router_admin_user_to_new_project ... fail

test_create_network_router_new_user_to_admin_project ... fail

Fuel (x86)

nosdn-nofeature

master

PASS

FAIL

PASS

SKIP

PASS

- only v3 used

skip mechanism needed or just part of reviewer instructions (does not need to be automated)

Jira Legacy

server	System Jira
serverId	1afe526e-48e5-33b1-8ed7-4f559eac1ef8
key	DOVETAIL-656

PASS

skip: 9

pass: 43

PASS

skip: 3 (dvr is not enabled)

pass: 132

test_create_distributed_router ... skip

test_show_distributed_router ... skip

test_update_distributed_router ... skip

PASS

FAIL

SLA failure: 9.023325 > 5.000000

PASS

odl

stable/fraser

FAIL

Unable to create a number

PASS

FAIL

fail: 2

pass: 117

FAIL

failed to create stacks

PASS

SKIP

PASS

FAIL

fail: 3

pass: 17

PASS

skip: 9

pass: 43

not supported

PASS

skip: 5

pass: 130

PASS

FAIL

change attack process name failed

https://gerrit.opnfv.org/gerrit/#/c/58693/

FAIL

No neutron-l3-agent process

PASS

Fuel (x86)

nosdn-nofeature

master

FAIL

rabbitmq located in msg0x VMs

https://docs.opnfv.org/en/stable-fraser/submodules/fuel/docs/release/installation/installation.instruction.html#automatic-installation-of-a-baremetal-pod

FAIL

database located in dbs0x VMs

https://docs.opnfv.org/en/stable-fraser/submodules/fuel/docs/release/installation/installation.instruction.html#automatic-installation-of-a-baremetal-pod

FAIL

l3-agent in gtw0x VMs

https://gerrit.opnfv.org/gerrit/#/c/58333/

FAIL

 - Success: 3
 - Skipped: 0
 - Expected failures: 0
 - Unexpected success: 0
 - Failures: 2

test_update_instance_port_admin_state ... fail

test_update_router_admin_state ... fail

FAIL

auth problem

passes on FUEL daily CI loop

FAIL

auth problem

deployment clearwater-opnfv timed out

FAIL

passes on FUEL daily CI loop

FAIL

passed on normal Fuel CI job

PASS

or

FAIL

create stack rate 20/20 and 16/20

PASS

FAIL - only v3 used

skip mechanism needed or just part of reviewer instructions (does not need to be automated)

Jira Legacy

server	System Jira
serverId	1afe526e-48e5-33b1-8ed7-4f559eac1ef8
key	DOVETAIL-656

PASS

FAIL

 - Success: 18
 - Skipped: 0
 - Expected failures: 0
 - Unexpected success: 0
 - Failures: 1

test_hotplug_nic ... fail

PASS

skip: 9

pass: 43

not supported

PASS

skip: 3 (dvr is not enabled)

pass: 132

test_create_distributed_router ... skip

test_show_distributed_router ... skip

test_update_distributed_router ... skip

PASS

FAIL

rabbitmq located in msg0x VMs

https://docs.opnfv.org/en/stable-fraser/submodules/fuel/docs/release/installation/installation.instruction.html#automatic-installation-of-a-baremetal-pod

FAIL

l3-agent in gtw0x VMs

https://gerrit.opnfv.org/gerrit/#/c/58333/

not supported

PASS

Installer

Scenario

vping_ssh

vping_userdata

tempest.osinterop

tempest.ipv6

tempest.multi_node_scheduling

tempest.security

tempest.vm_lifecycle

ha.nova_api

ha.neutron_server

ha.keystone

ha.glance_api

ha.cinder_api

ha.cpu_load

ha.disk_load

ha.haproxy

sdnvpn.subnet_connectivity

sdnvpn.tenant_separation

sdnvpn.router_assoc

sdnvpn.router_assoc_fip

Apex

nosdn-nofeature

stable/fraser

PASS

FAIL

passes on Apex daily loop

FAIL

passes on Apex daily CI loop

FAIL

PASS

FAIL

PASS

FAIL

PASS

FAIL

IPMI info missing

PASS

not supported

Fuel

nosdn-nofeature

PASS

FAIL

PASS

FAIL

PASS

FAIL

PASS

not supported

Historical results below

2018-05-23 - limited results due to failed CI jobs

...

Version	Old Version 27	New Version Current
Changes made by	Dan Xu	Dan Xu
Saved on	Jun 22, 2018	Sept 04, 2018

Versions Compared

Key

Historical results below