January 16, 2012

Test Review Board - PA-16

Rule on the validity of the tests

A. Scope

A Test Review Board (TRB) is a formal meeting that determines the validity of a test or sequence of tests. It is aiming at providing recommendation concerning the induced acceptance or delivery of a product following qualification or acceptance tests, and recommendations concerning the start of subsequent tests. Holding a Test Review Board meeting is systematic after any equipment or system test breakdown).

The specific rules for a software testing TRB are set out in the section "Software Quality (see PA-7)".

This document is based on the CNES Standards Reference (RNC) document "RNC-CNES-Q-ST-20-101 Commission de Revue des Essays (CRE)".

B. Fundamentals

The TRB is a milestone which comes after the test and consists in checking that the test, or sequence of tests was carried out correctly and in full. The TRB then makes a decision as to its validity, ensuring the following points:

  • the actions recorded at the Test Readiness Review (TRR) (see PA-15) have been carried out in the required time ;
  • all test results are available and correctly recorded or noted, as are the results or records relating to the test conditions ;
  • the various sequences were carried out in accordance with the plans and procedures, without any unauthorised stopping or resuming of sequences ;
  • all significant events were correctly traced and recorded.

 

In the event that the test validation or product delivery cannot be pronounced, a decision is made concerning the follow-up (resume the tests, perform additional tests, etc.).

Minutes are mandatory to record the TRB issues and conclusions. They must agree that the dismantling of the test configuration by its instigators is authorised, as a requisite for dismantling.

C. Organising and implementing the TRB

1. Notification

The party responsible for the supply suggests a date for convening the TRB, giving sufficient notice for those at the next level up (CNES, scientific partners, etc.) to make sure they are available.

2. Location

The TRB is, in principle, held on the premises of the space laboratory or manufacturer responsible for the supply.

3. Participants

Attendees whose presence is recommended include:

On behalf of the party responsible for the supply (lab or manufacturer):

  • the Project Manager and/or product technical manager and/or test manager ;
  • the Product Assurance engineer (or Integration Quality Assurance supervisor).

 

On behalf of the party in charge at the Customer's (CNES, lab or manufacturer):

  • the Project Manager and/or product technical manager and/or test manager ;
  • the Product Assurance engineer (or Quality) ;
  • in cases where he/she is mandated, the Product Assurance customer representative (Delegation of task, DGA-SQ support, etc.).

 

4. Equipment and available documentation and providing

a) The product and the implementation and testing means forming the subject of the Test Review Board must be available for presentation on request from the Customer during the Test Review Board.

b) All the documentation relative to the following must be able for examination:

  • the reference configurations (As designed) and applicable (As built) configurations:
    • complete production file (Configuration Item Data List),
    • list of applicable modifications (accepted modification proposal).
  • the actual configuration:
    • list of the elements constituting the supply along with all the related control documents,
    • list of the other elements and/or sub-assemblies of different individual responsibility along with their control documents,
    • list of major and minor deviations, waivers and repairs,
    • requests for deviation, waivers and repairs,
    • list of non-conformance reports or open technical events.
  • the tests: test plans and procedures ;
  • the electrical and mechanical ground support equipments and the environmental control means: validation and calibration procedures ;
  • AIV operations during the tests and the test proceedings: handling, storage and cleaning procedures, 'red lined' leading procedure, attesting to the monitoring of all the stages in the procedure, etc.

 

c) The End-Item Data Package (EIDP) and the Log Book (LB).

d) All the documentation relative to the manufacturing history, as defined by the project's Product Assurance Specification or the approved corresponding plan, must be available for consultation, including:

  • Inspection report and/or certificate of conformance for the constituents,
  • EIDP (see PA10) and LB (see PA14) (hardware and software) from lower levels.

 

5. Minutes and decisions

Minutes of the TRB drafted in session must be issued at the end of the meeting and signed by the relevant parties with responsibilities.

The minutes must clearly indicate the decision(s) taken from amongst the following:

  • full acceptance of the test results (Successful DRB) ,
  • acceptance of the test results (pending) closure of identified actions,
  • authorisation to continue the tests (in the case of intermediate TRBs),
  • authorisation to deliver the equipment with a list of opened actions,
  • unreserved authorisation to deliver the equipment,
  • refusal of authorisation to deliver the equipment. (Rejected DRB)

 

6. Conducting TRBs after equipment tests

The following is checked during a TRB for equipment tested:

  • the values obtained conform to the values required through pass criteria, as defined in the procedure and the values outside tolerance are covered by a non-conformance report (see PAF-14 ;
  • the unexpected technical events that arose are covered by a non-conformance report ;
  • the test data recorded conform to the expected objectives and their subsequent use will enable validation of the conformity of the results in relation to the expected performance ;
  • the non-conformance reports have been correctly indexed and processed ;
  • all non-conformances have been closed or the actions underway identified.

 

In the particular case of product acceptance or delivery, the following is checked:

  • the supply is complete (accessories, products delivered separately, threaded fasteners, etc.);
  • the supply shows no signs of visible damage, has been correctly stored and is clean (i.e. in compliance with the procedures for storage or transportation configuration procedures) ;
  • the requisite documentation (End-Item Data Package (EIDP), the Log Book (LB)) is in existence and is complete.

 

7. Proposed TRB agenda

The "recommended" agenda is presented in chronological order:

  • review of the actions from the TRR ;
  • review of test proceedings (chronology, material difficulties encountered, configuration modifications, etc.) ;
  • analysis of the results ;
  • examination of the non-conformance reports and deviations ;
  • examination of the records associated with the test means ;
  • examination of the documentation (CIDL, EIDP, LB, test reports, etc.) ;
  • visual examination of the product ;
  • conclusions.

 

Risks

RisksInvalid or incomplete tests
 RecommandationsHOLD A FORMAL MEETING AFTER A TEST OR A SEQUENCE OF TESTS
Test Review Board (TRB)

 

Activities / documentation

Published in: