This wiki page is based on an email sent to the Anuket TSC on Sept 6, 2021.

Introduction

During the Lakelse release, some issues with RC became apparent.  

An important objective of the release process is to make the development status of deliverables visible to the TSC, and the community.  RC lacks transparency, despite steps in the release process intended to provide transparency.  

RC Mapping to RA

RC is frequently described as being dependent on RA.  However, during the Lakelse release, it became apparent that RC is, at best, loosely coupled to RA.  More accurately, RC seems to be based primarily on OpenStack releases. The apparent assumption is that this will result in sufficient, although unknown, coverage for RA.  There doesn't appear to have been any attempt to understand or even read the RA requirements during the development of RC.

1. M2 requires a description of the coverage of RA.  I was provided a link to RC documentation that does not provide specific mapping to RA requirements.  In fact, there aren't even any references to RA requirement numbers.
   1. This means that we do not have an accurate, or even approximate, measurement of RA coverage.
   2. We also cannot describe how coverage has changed from one release to the next.
   3. This is a contradiction of the RC documentation, itself.  For example:  "The conformance specifications must provide the mapping between tests and requirements to demonstrate traceability and coverage."
2. The RC documentation includes lists of test case names.  However, there are no links to detailed information about the test cases.  
   1. For example, a requirement author, or other stakeholder, might want to understand how a requirement is being tested.
   2. The engineering team conducts a review of RA documentation for each release.  Sometimes they want to see how a requirement is being tested.

RC Validation Testing

We discussed validation testing at the Sept 6 Technical Discussion meeting.  At the meeting I learned that "validation" is performed by running the test suites against a known-good environment and verifying that the test cases pass.

...