Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Update test case requirements to clarify intentions around upstreaming of patches

...

  • Test cases should favour implementation of a published standard interface for validation
    • Where no standard is available provide API support references
    • If a standard exists and is not followed, an exemption method needs to be established
  • Test cases be documented
    • Use case specification
    • Test preconditions
    • Basic test flow execution descriptor
    • Post conditions and pass fail criteria
    • Parameter border test cases descriptions
    • Fault/Error test case descriptions  (this feels optional at this time)
  • Test cases must pass on OPNFV reference deploymentsTest result storage, structure and test result information management (these should be able to be run publicly or privately)
    • Tests must pass with deployments with at least two installers
    • Tests must pass with at least two deployment scenarios involving different SDN controllers
    • Tests must not depend on code which has not been accepted into the relevant upstream projects
    • (hongbo: this needs to be discussed further. there are several SDN controllers. some test cases from dedicated SDN controller can not be used for other controller.)
  • Test documentation/implementation file and directory structure (per supported framework)
    • not require a specific NFVi platform composition or installation tool
    • Tests must not require unmerged patches to the relevant upstream projects in order to pass when run against OPNFV reference deployments
  • The certification test suite must not require access to OPNFV infrastructure or resources in order to pass

Dovetail Test Suite Structure

...