Anuket Project

Dovetail Future Release Priority Inputs

Owned by Former user (Deleted)
Last updated: Mar 14, 2018 by Georg Kunz

The following Etherpad will be used to gather inputs from upstream test projects and the Dovetail community: 

https://etherpad.opnfv.org/p/Dovetail_Future_Release_Priority_Inputs

Mapping of Requirements and Status

In the Nov 28, 2017 TSC meeting, we discussed ways of planning for future releases of the OPNFV Verified Program.. During the Danube cycle, we adopted a pragmatic approach of working from bottom-up (i.e. what the community had  already done), but noted its limitations as well as top-down analysis of overall program requirements. The details are documented in this Addendum. Specifically, the Addendum called out areas where either features or test cases fall short in OPNFV release (Danube), and future test area considerations based inputs from the community and EUAG. The following is an attempt to organize the information in a table format with an intent to facilitate community wide discussions, which, hopefully lead to actions to identify where the priorities are, and efforts from all projects to fill those gaps. We invite the whole community to provide feedback, ideas/comments.

Test Area in OPNFV VerifiedDanubeEuphratesFraserGambia
Basic Cloud Capabilities (defined a single VM, networked, providing simple network functions). See here.
  • API testing (2016-08)
  • API backward compatibility - yes, same as Openstack, but not yet clearly defined in other areas
  • tempest scenario (i.e. small nuggets of use patterns in system level) - included but optional.
    • Port security and security groups
    • VM life-cycle events
    • VM networking
    • VM resource scheduling
    • Forwarding packets in the data path
  • VIM: Openstack only
  • SDN: transparent mode only (e.g. ODL can be used as a Neutron plugin)
  • Datapath: L3 level readability only - includes vPing, and ping tests on all networking datapath validations.
  • VNF: VM for testing/ping.
  • A subset of the tempest scenario tests are commonly used/supported/adopted and we should consider promote them to mandatory.
  • Additional test cases


Candidates

  • Tempest smoke tests: OpenStack API tests DOVETAIL-599 - Getting issue details... STATUS
  • SNAPS smoke tests: OpenStack API tests  DOVETAIL-630 - Getting issue details... STATUS

 

  • VIM: k8s test suite (see kubernetes certified.)
  • VIM: define how to accommodate VIM choices in dovetail test framework
  • Functest's proposed work on k8s


Candidates

  • Patrole: testing OpenStack RBAC configuration using Tempest

 
NFV Specific Functional Requirements
  • SDNVPN testing included in optional
  • IPv6 testing included in optional
  • SFC is not included due to upstream lack of maturity, or adoption
  • SDN explicit north bound API not included - opnfv has not taken on this level of testing in large scale, upstream status unclear.

 - IPv6 extensions?

Candidates

  • SDNVPN Tempest API tests DOVETAIL-598 - Getting issue details... STATUS

 Candidates

  • Neutron trunk ports
 
HA and resiliency, (VM migration)
  • HA included as mandatory tests - supporting Openstack control service HA and resource overloads.
    • verify service continuity
    • verify control process recovery
  • HA extensions needed.

Candidates

  • OpenStack controller node HA  DOVETAIL-605 - Getting issue details... STATUS
  • OpenStack message bus HA DOVETAIL-605 - Getting issue details... STATUS
  • OpenStack Virtual Router HA  DOVETAIL-605 - Getting issue details... STATUS

Candidates

  • OpenStack database HA YARDSTICK-960 - Getting issue details... STATUS
  • OpenStack Nova conductor HA YARDSTICK-959 - Getting issue details... STATUS
  • OpenStack Nova scheduler HA  YARDSTICK-958 - Getting issue details... STATUS
  • OpenStack Heat-API HA YARDSTICK-961 - Getting issue details... STATUS
 
stress testing, platform-in-place-upgrade
  •  Considered Bottleneck project proposal with a basic set, but it needed more testing experience in the community.
  • More comprehensive set yet to be defined.
Candidates
  • Bottleneck stress test DOVETAIL-631 - Getting issue details... STATUS

 
  •  Is VM migration having a commonly accepted use pattern?
 
Security (including security features as well as tools such as vulnerability scanning)
  • Security: openstack features included in API and tempest scenario testing
    • virtual network isolation,
    • security groups,
    • port security
    • and role based access control 
 
  • Discussing during the plugfest
  • Domain expertise and contribution needed
  •  
 

Service Assurance, also include the tools for SA

  • platform operational insights, e.g. telemetry, logging
  • fault management
  •  Not yet
  • looking for contributions
  • Doctor has a long list of dependencies
  • looking for contributions
  
Use Case Testing
  •  Net yet
  • Some VNFs have been worked on in Functest, sampleVNF, discussion planned in plugfest how to proceed
  •  Discussing methods to incorporate use case testing
    • Functest vIMS (cloudify orchestrated) - dependency to be analyzed, stable/mature?
    • alternatively, vIMS with Heat
    • other use cases in sampleVNF
    • other use cases in ONAP (e.g. vCPE) or other upstream projects
  
Performance
  •  Not yet
  • There will be a info session to discuss during plugfest at Portland
 
  • Many good tools have been worked on in projects such as vsperf, and others, discussion planned in plugfest on how to address hardware dependency and methodology pitfalls, then decide if or how to proceed
 
MANO - ONAP - VNF testing (e.g. packaging, on-boarding etc.)
  •  Not yet
  • There will be an info session to start the discussion in plugfest to think how we can collaborate
 
  • ONAP just had the first release, related projects: VNFSDK, VVP
  • Consider Fraser as a possible integration point
 

Multi-Site Federation

  •  Not yet
   
Efficiency, e.g. hardware and energy footprint of the platform
  •  Net yet
  • Interesting area calling for contributions
   
Dovetail tool improvements (together with all testing projects) 
  • format result.json in a human readable and json-compliant format

  • provide test result overview after a test run

  • separation of test suite yaml spec file from the tool, so that the tool framework development/testing and the test suite spec can be cleanly decoupled

  • developer's guide documentation


  • harmonization of log files
  • tagging mechanism for test cases from all test projects
  • LaaS support - dovetail should become a part of LaaS toolset, can be easily run any time by everyone
 

 

Earlier Notes

Below are some previous inputs from the Design Summit, Beijing Summit and OPNFV Plugfests:

  1. Inputs collected during the Design Summit dovetail session on June 13

    1. Stress testing  

    2. Incorporate more OPNFV feature projects (e.g. SFC etc.)

    3. Include Models (the OPNFV project) test cases (launch a sample multi-VM VNF)

      1. We need to check these against the test case requirements

    4. Include Doctor (optional/mandatory is tbd)

  2. EUAG inputs June 15 during the Beijing Summit
  3. Earlier inputs from Plugfest April 24-28
    1. discussed the topic of how to potentially do performance testing through opnfv projects in this area
    2. any area of security testing that may be applicable, e.g. scanning for security vulnerability

 

 

( DOVETAIL-599 - Getting issue details... STATUS )