This page is intended as a working draft of how we will look to leverage and develop test cases for the dovetail project.
...
- Test cases should implement a published standard interface for validation
- Where no standard is available provide API support references
- If a standard exists is not followed an exemption method needs to be established
- Test cases be documented
- Use case specification
- Test preconditions
- Basic test flow execution descriptor
- Post conditions and pass fail criteria
- Parameter border test cases descriptions
- Fault/Error test case descriptions (this feels optional at this time)
- Test cases must pass on OPNFV reference deployments
- Tests must pass with deployments with at least two installers
- Tests must pass with at least two deployment scenarios involving difference different SDN controllers
- (hongbo: this needs to be discussed further. there are several SDN controllers. some test cases from dedicated SDN controller can not be used for other controller.)
- Test documentation/implementation file and directory structure (per supported framework)
- Test result storage, structure and test result information management (these should be able to be run publically or privately)
...
hongbo: the same as those defined in the phase 1 and phase 2
Out of scope (I left this for now, but it does not belong on this page)
Dovetail is not a testing problem with a goal of identifying gaps in the platform. Its goal is to validate the conformance of OPNFV "stacks" to a minimum set of capabilities common to all OPNFV implementations. As such, it is not a goal of the Dovetal project to write failing tests, or to engage in the development of APIs to fill perceived gaps in the platform. While formulating the Dovetail test suite, it is possible that we will encounter limits of the commonalities of OPNFV implementations in areas which we would like to test conformance. In this case, efforts to create alignment, or to implement missing features, should happen in the context of other OPNFV projects. Dovetail will always be primarily concerned with documenting and verifying common platform capabilities after they exist.
...