The Dovetail test results database has certain unique requirements due to the commercial nature of the OPNFV CVP. These requirements and related information will be used by the OPNFV Test Community to develop Test API, Database and Dashboard features for use by Dovetail. The OPNFV Test community has a goal to reuse and normalize across test projects as much as possible in order to reduce development effort, help with on-boarding and increase test tool efficiency.
Test results are written into the common database by the Dovetail Test Tool and shall use the Common Test API for this purpose.
Database location
- All data shall be stored in LF hosted lab
Writing Test Results
- Requires an authentication mechanism to ensure that only authorized and identifiable individuals (companies?) who are part of the compliance program and executing valid CVP tests can write test data into the database.
- Each set of test data must be identifiable by 1) date and time of test run as well as 2) a unique identifier for each test run)
Reading Test Results
- Only authorized individuals (CVP reviewers) shall be able to access individual test run data and identify the data with a particular test run.
- Whoever writes data into the database shall also be able to read their test data from the database
Privacy Requirements
- CVP test data shall be logically isolated from Community test data
- There is no requirement to encrypt CVP test data
Aggregated Results ... the Test community is working with Bitergia consultants to develop dash boards for test projects ... Dovetail could add requirements
- Is there a requirement to provide aggregated (anonymized) statistics of results? For example ...
- How many test runs are stored
- How many test cases executed
- % of test cases passed on average
- % of test cases failed on average
- Apply date ranges ... e.g. how many test runs stored in the past year
- Are test cases passed improving over time