...
OPNFV Verified Program (
...
OVP)
The CVP OVP is a program initiated by the OPNFV Board of Directors with the following key objectives:
...
Dovetail is the project that delivers all the open source software toolchain, documentation, and test suite for CVPOVP. Dovetail project meets CVP OVP testing requirements by leveraging many OPNFV projects and upstream projects. The Certification and Compliance committee (C&C committee) serves as the program's administrator on behalf the OPNFV Board.
Products that pass the CVP OVP compliance verification suite may be labeled as “OPNFVReady”“OPNFV Verified”. The OPNFV C&C Committee on behalf of the Board of Directors can award a product “OPNFV Ready” Verified” status.
More details of CVP OVP can be found in the CVP OVP document.
System Under Test (SUT)
The System Under Test (SUT) are products from vendors, not OPNFV release artifacts themselves. OPNFV development tooling, such as CI/CD, is also not part of the SUT. The vendors can bring up the SUT to a pre-test state in any way they choose. Dovetail will provide documentation to help the testers to prepare and configure the SUT.
In the current CVP OVP version (Danube release time of Dovetail), the SUT consists of NFVI software, VIM software, and necessary hardware in one System Under Test. The hardware should follow Pharos guidelines (but the guideline is generalized in the Dovetail documentation to remove specifics tied to OPNFV CI/CD environment). Vendors can use their own hardware, or third party or white box hardware to be tested as a whole. And if the combined whole passes the Test Suite, the CVP OVP label applies to the software.
In the Danube cycle, CVP OVP does not plan to test hardware-only systems.
Test Suite
A Test Suite is a collection of test cases grouped together for a specific purpose, for example, CVPOVP. A Test Suite can be named and versioned for easy and accurate reference. The Dovetail test tool primarily supports the Compliance Test Suite for CVPOVP, but can in general support multiple test suites.
Compliance Test Suite
A Compliance Test Suite is the Test Suite used in the OPNFV CVPOVP. By default, this is the Test Suite we are referring to. The Compliance Test Suite is to be versioned with MAJOR.MINOR.PATCH together with the Dovetail tool.
Test Area
A Test Area is a collection of test cases grouped together by functional areas or upstream projects. For example, Openstack Refstack/Defcore, IPv6, HA, SDNVPN.
Test Case
A Test Case is an instance of test unit within a Test Area. In some implementation of the test cases, we may see additional sub-test-cases as smaller units under a test case.
...
OVP (or Dovetail) Testing
The following diagram help illustrates the main components of the testing system for CVPOVP.
Tester
The tester is a knowledgeable engineer of the vendor that has admin right to the SUT within its lab or administrative domain. We use the term in abstract sense, not necessarily meaning one individual. The tester must have sufficient admin privilege to make all hardware and software configurations according to the Dovetail Tester's Guide.
The tester must also have login/access rights established with the Linux Foundation (OPNFV) to access the CVP OVP server to review test results.
Reviewer
The reviewer is a knowledgeable engineer to review test results submitted by vendors to determine if CVP OVP requirements have been met and OPNFV-Ready OPNFV Verified label should be granted. Again we use the term in abstract sense, not necessarily meaning one individual. The C&C committee administers review process.
The reviewer must have login/access ID established with the Linux Foundation (OPNFV) to access the CVP OVP server to review test results.
...
OVP Server
The CVP OVP server is the backend system (currently consists of a web frontend and a database backend) hosted by the Linux Foundation on behalf of OPNFV. When a test suite is executed, test results are submitted and saved in the CVP OVP Server's database. The CVP OVP Server can be accessed via a Test API using Linux Foundation credentials (to be finalized).
...
OVP Client (Dovetail client or command-line interface)
The CVP OVP Client, or Dovetail client or Dovetail CLI, is the software that runs in a vendor's host to drive the automated CVP OVP test suite. the Tester who executes the test suite must have the valid credential credentials of the CVP OVP server in order to store and review test results.
The CVP OVP Client comes with two forms: either as a Dorker container, or as a software install (based on Python). Dovetail Tester's Guide contains detailed guides on how to execute the test suite using the Dovetail client CLI.
...