WARNING: This page is not currently maintained. Test areas will be defined in the test strategy document (being developed in this Gerrit proposal and tracked in this JIRA ticket - see current draft document), and specific test cases will be discussed and included in the Dovetail JIRA and Gerrit repository

This is the list of candidate test areas which have been proposed by community members for addition to the Dovetail test suite. These test areas will be evaluated under the Dovetail test case requirements, proposed for inclusion as patches in Gerrit against Dovetail, and reviewed by the Dovetail maintainers. Once approved, they will become part of the Dovetail test suite.

1 Brainstorming
2 Details on Test Areas
- 2.1 VIM Operations on Compute
- 2.2 VIM Operations on Network
- 2.3 VIM Operations on Volume
- 2.4 VIM Operations on Image
- 2.5 VIM Operations on Identity
- 2.6 NFVI
- 2.7 High Availability (HA)
- 2.8 IPv6
- 2.9 VPN
- 2.10 Fault Management(Doctor)
- 2.11 KVM
- 2.12 Parser
- 2.13 virtual Traffic Classifier
- 2.14 Copper
- 2.15 Promise
- 2.16 Multisite
- 2.17 SDN controller(ODL)
- 2.18 SDN controller(ONOS)
- 2.19 Open source VNF running on NFVI

Brainstorming

This section is intended to help us get to quick consensus on the test priorities for Danube. The rest of the content of this page will be updated to be aligned with this section.

This is intended to help us discuss and collaboratively edit a set of priorities for the dovetail program in OPNFV.

If we need a more long-winded/threaded discussion on the topic, please add an item to https://techdiscuss.opnfv.org/.

This is itself a strawman proposal for how we can discuss this important topic. Alternatives are welcome, although the basic goal is that we can come to collaborative consensus as quickly as possible, with minimal barriers to participation in the discussion.

Also see:

https://jira.opnfv.org/browse/DOVETAIL-345

High

Component verification: leverage all available tests to validate functional/API/etc capabilities of component distros

NFVI+VIM: start with well-established aspects of NFVI+VIM

OpenStack: Refstack (https://refstack.openstack.org)

ODL: https://github.com/opendaylight/integration-test

OpenContrail: https://github.com/Juniper/contrail-test-ci/wiki

ONOS: https://wiki.onosproject.org/display/ONOS14/How+to+Contribute+to+System+Test

Reference VNF portability: demonstrate portability and consistent operation for an ever-expanding suite of reference VNFs, which have been verified for most/all OPNFV distros

vIMS

Cross-controller dependent OPNFV features: verify operation of features that depend upon coordinated use of multiple components

SFC

Broadly supported OPNFV features: focus on features that are support in many/most distros and hardware platforms

Medium

Reliability: operation/performance under stress e.g. load and chaos

Performance: benchmarking

Security: static scans (vulnerabilities, licenses) and response to threats in operation

Infra lifecycle: deployment and day-2 operations

MANO: MANO-stack interop

Low

Narrowly-supported features: any features that are distro or hardware platform specific

Application (e.g. VNF) compliance with standards: while we may leverage reference VNFs as typical use cases their use should focus entirely upon verifying that the NFVI etc implementations result in a consistent user experience. The "user" in this case is the service provider, and consistency does not extend to whether the VNF itself or how it is operated adheres to any standards-based expectation on functionality.

Details on Test Areas

VIM Operations on Compute

The Gerrit reference for this test area: https://gerrit.opnfv.org/gerrit/#/c/27493

ID	Type	Test Case	Description	Pre-condition & requirements	Status	Gerrit References

ID	Type	Test Case	Description	Pre-condition & requirements
1	VIM operations on Compute	Create and show root certificates	Create and show root certificates	VIM: Openstack Components: Keystone, Nova
2	VIM operations on Compute	Flavors CRD ops	Create, delete, list and show flavors	VIM: Openstack Components: Keystone, Nova
3	VIM operations on Compute	Floating IPs CRUD ops	Allocate, delete, associate, disassociate, show and list floating ips	VIM: Openstack Components: Keystone, Nova, Neutron
4	VIM operations on Compute	Image metadata CRUD ops	List, set, update, show, delete image metadatas	VIM: Openstack Components: Keystone, Nova
5	VIM operations on Compute	Images CRUD ops	Delete, list, show and create images	VIM: Openstack Components: Keystone, Nova, Glance
6	VIM operations on Compute	Keypairs CRUD ops	Create, list, delete and show keypairs	VIM: Openstack Components: Keystone, Nova
7	VIM operations on Compute	Show absolute limits	Show absolute limits	VIM: Openstack Components: Keystone, Nova
8	VIM operations on Compute	Security group rules CRUD ops	Create, delete and list security group rules	VIM: Openstack Components: Keystone, Nova
9	VIM operations on Compute	Security group CRUD ops	Create, delete, show, update and list security groups	VIM: Openstack Components: Keystone, Nova
10	VIM operations on Compute	Interfaces CRUD ops	Create, delete, show and list interfaces; attach and deattach ports to servers	VIM: Openstack Components: Keystone, Nova, Neutron
11	VIM operations on Compute	List availability zones	List of availability zones and available services	VIM: Openstack Components: Keystone, Nova
12	VIM operations on Compute	Servers CRD ops	Create, delete, list and get servers	VIM: Openstack Components: Keystone, Nova, Neutron
13	VIM operations on Compute	Rebuild, resize and update server	Rebuild, resize and update server	VIM: Openstack Components: Keystone, Nova
14	VIM operations on Compute	List and show instance actions	List and show instance actions	VIM: Openstack Components: Keystone, Nova
15	VIM operations on Compute	Multiple create servers	Verify whether reservation id is in the response body of creating servers	VIM: Openstack Components: Keystone, Nova
16	VIM operations on Compute	Server actions	Change server passwords; reboot, rebuild, resize, pause, unpause, suspend, resume, shelve, unshelve, stop, start, lock, unlock server; create backup; get console oupt; show vnc console;	VIM: Openstack Components: Keystone, Nova, Glance
17	VIM operations on Compute	List server addresses	List server addresses	VIM: Openstack Components: Keystone, Nova, Neutron
18	VIM operations on Compute	Server group	Create/delete server-groups with different policies. policies = affinity/anti-affinity It also adds the tests for list and get details of server-groups	VIM: Openstack Components: Keystone, Nova
19	VIM operations on Compute	Server metadata CRUD ops	List, set, update, show, delete server metadata	VIM: Openstack Components: Keystone, Nova
20	VIM operations on Compute	Delete and get server password	Delete and get server password	VIM: Openstack Components: Keystone, Nova
21	VIM operations on Compute	Server personality ops	Create and rebuild servers with personality	VIM: Openstack Components: Keystone, Nova
22	VIM operations on Compute	Rescue and unrescue servers	Rescue and unrescue servers	VIM: Openstack Components: Keystone, Nova
23	VIM operations on Compute	Create server with special metadata	Create server with admin password; create server with existing server name; create specify keypair; update server name; create server with IPv6 only	VIM: Openstack Components: Keystone, Nova
24	VIM operations on Compute	List virtual interfaces	List virtual interfaces	VIM: Openstack Components: Keystone, Nova, Neutron
25	VIM operations on Compute	Attach and deattach volume	Attach and deattach volume; list and show volume attachments; delete attached volume	VIM: Openstack Components: Keystone, Nova, Cinder
26	VIM operations on Compute	Volume snapshot CRD operations	Create, show, list and delete volume snapshots	VIM: Openstack Components: Keystone, Nova, Cinder
27	VIM operations on Compute	Volumes CRUD ops	Create, show, list, update and delete volumes	VIM: Openstack Components: Keystone, Nova, Cinder
28	VIM operations on Compute	Show and list extensions	Show and list extensions	VIM: Openstack Components: Keystone, Nova
29	VIM operations on Compute	Live block migration	Migrating to an invalid host should not change the status	VIM: Openstack Components: Keystone, Nova
30	VIM operations on Compute	List networks	List networks	VIM: Openstack Components: Keystone, Nova, Neutron
31	VIM operations on Compute	Quotas CRUD ops	Show, create, update and delete quotas	VIM: Openstack Components: Keystone, Nova
32	VIM operations on Compute	List and show tenant networks	List and show tenant networks	VIM: Openstack Components: Keystone, Nova, Neutron
33	VIM operations on Compute	List and show API versions	List and show API versions	VIM: Openstack Components: Keystone, Nova
34	VIM operations on Compute	Agents CRUD	Create, update, delete and list agents	VIM: Openstack Components: Keystone, Nova
35	VIM operations on Compute	Aggregate CRUD	Create, update, delete, show and list agents	VIM: Openstack Components: Keystone, Nova

Dovetail Test Areas and Test Cases

Brainstorming

Details on Test Areas

VIM Operations on Compute