January 16, 2012

Organisation and conduct of reviews - M-13

Organise the end-of-phase reviews

A. Scope

This document briefly recapitulates the principles described in the CNES Standards Reference (RNC) document "RNC-ECSS-M-ST-10-01 Organisation and conduct of reviews". It also features a typical plan for an organisation memo (see section C) and for the minutes of a Review Group (see section D).

B. Review process recap

1.1 Organising a review

1.1.1 Schedule

Reviews are generally planned in accordance with satellite-customer development constraints and the schedule is set by the customer, in conjunction with the laboratory (-ies).

It is included in the Project Development Plan (see M-4).

1.1.2 The parties involved

1.1.2.1 The Project Group

Responsible for designing and developing the product subjected to review. The Project Group prepares the review file and presents it to the Review Group, answers any questions and implements the decisions made by the Steering Committee.

1.1.2.2 The Review Group

A consultative body, independent of the project. It is responsible for providing its expertise through studying the review file and making recommendations to be put to the Steering Committee.

It is usually made up of the following members:

  • a president: he/she is familiar with the discipline in question, possesses the necessary qualities for running the Review Group effectively and is free from any other commitments within the project;
  • a scientist: he/she is skilled in the discipline concerned, works in a scientific laboratory but is not directly involved in developing the project;
  • a customer Product Assurance engineer;
  • a Programme Management customer representative;
  • a satellite-customer representative;
  • technical experts: they are chosen in accordance with the specific nature of the project and are not involved in developing the project.

 

For effective monitoring, it is advisable to make sure that the group can be sustained for the entire project.

1.1.2.3 The Steering Committee

A decision-making body. It studies the recommendations of the Review Group and issues decisions concerning the remainder of the project.

In its most limited form, it is made up of at least:

  • one of the customer's managers;
  • the director(s) from the laboratory (-ies) concerned;
  • the Review Group President;
  • the customer's subject matter specialist;
  • the PI;
  • the customer Project Manager.

 

1.2 Review proceedings

1.2.1 Setting up the Review Group

The manager for the customer organisation in question designates the Review Group president. He/she sets up the Review Group together with the customer Project Manager. He/she validates and signs off the organisation memo drawn up by the Project Manager, which defines the review's objectives. For CNES reviews, the organisation memo is approved by the review board.

1.2.2 Sending the documents subjected to review to the Review Group

It is preferable for the Review Group members to be able to study the documentation subjected to review two weeks before the presentation session.

1.2.3 Presenting the project to the Review Group

The customer project team and laboratory teams present the project's actual state of progress at a special meeting and in accordance with the objectives defined in the organisation memo.

The teams must present the critical points and proposed solutions in a precise manner.

1.2.4 Review Group questions

After the Review Group has studied the documentation and seen the presentation, its members identify the potential problems via Review Item Discrepancy forms (RID) (see form MF-9). The RIDs are then sent to the Project Group. They must be real questions about the perceived malfunctions, rather than concerning additional information on established correct functioning.

The Review Group will ensure that lessons learned (LL) are taken into account when drafting its questions.

1.2.5 Sending the questions to the Project Group

The president of the Review Group centralises the RIDs and tries to limit their number by focusing on potential risks and eliminating duplicates in order to facilitate the Project Group's task. He/she also groups RIDs together into themes and sends them to the project department.

1.2.6 Project Group responses - Processing RIDs

After the Project Group has analysed the RIDs, it gives its responses to the Review Group at a meeting during which the problems highlighted are debated.

For each RID, depending on the nature of the response, there are several possible scenarios:

  • the Project Group provides an immediate and acceptable response to the problem highlighted: the RID is closed ;
  • the problem cannot be immediately resolved but the Project Group can deal with it: it then opens an action ;
  • the Project Group cannot deal with the problem highlighted or does not have the authority to do so: recommendations are made to the Steering Committee.

 

1.2.7 Drafting the Review minutes

The president of the Review Group drafts the minutes after the RIDs have been processed: he/she summarises the review by going over each of the objectives assigned and sets out the recommendations submitted to the Steering Committee. The RIDs are appended.

As such, the review minutes constitute the working basis for the Steering Committee.

1.2.8 Convening the Steering Committee

The president of the Review Group presents the review summary to the members of the Steering Committee and in particular the resulting recommendations. The Steering Committee decides whether or not to accept the recommendations. Accepted recommendations become decisions and their method of application is specified.

1.2.9 Drafting the Steering Committee minutes

The Steering Committee minutes set out and justify every decision taken: whether a recommendation has been accepted or not and how to apply accepted recommendations.

This document is prepared by the Project Manager and approved by the president of the Steering Committee.

1.3 Documentation relating to the Review

  • Documents subjected to review

This has nothing to do with review-specific documentation. These are only those documents which were drawn up to take the project forward during the phase in question.

The list of documents subjected to review is included in the review organisation memo. There may also be a list of reference documents required to understand the project.

  • Organisation memo for a review

See section C of this document.

  • Review File

It is made up of the documentation subjected to review, the organisation memo and the slides created by the CNES project teams and laboratories for the presentation to the Review Group.

  • List of RIDs

They are to be matched up with the corresponding actions and/or recommendations. RIDs must be drawn up in accordance with the MF-9 template.

  • Review minutes

The typical plan for the minutes is set out in section D of this document.

Recommendations must be drafted in accordance with the form MF-10.

  • Steering Committee minutes

The Steering Committee minutes go over the recommendations issued by the Review Group and specify whether each one has been accepted or not, reformulating them if necessary. Once they have been accepted, recommendations are referred to as decisions. Likewise, any actions still open are highlighted for the Project Group's attention.

C. Typical content of a review organisation memo

1. Designation of the review

This section designates the review concerned by the organisation memo. It specifies how the review fits into the global development framework, recapitulates reviews from higher and lower levels which have already taken place and restates their main conclusions if necessary.

2. Review objectives

This section specifies the scope covered and the review's general and specific objectives.

3. Review organisation

3.1 Review Group membership

The members of the Review Group are listed here.

3.2 Steering Committee membership

The members of the Steering Committee are listed here.

3.3 Schedule of events

It shows the dates of the main stages in the review process:

  • sending the documents subjected to review to the Review Group,
  • project presentation to the Review Group,
  • sending the questions to the Project Group,
  • question and answer session with the Project Group / Processing RIDs,
  • convening the Steering Committee (planned date).

 

3.4 Access to documentation

This section specifies the format and conditions in which the documentation will be made available to the Review Group: CD-ROM, hardcopy, network server, etc. Any applicable rules of confidentiality will also appear here.

3.5 Review secretariat

The name of the person responsible for the review secretariat appears here. This role involves: providing documentation, centralising presentations, RID tool configuration and management, circulation of useful information, printing out the RIDs for the Question/Answer session, etc.

3.6 RID processing

This section sets out how RIDs are processed, according to the tool used. The categories of RIDs are specified (e.g. MANagement (MAN), Product Assurance (PA), Mechanical and Thermal Architecture (MTA), etc. The numbering system to be used by the authors is also explained, as well as the Major/Minor classification if necessary.

3.7 Composition of the documentation

This section lists the documentation subjected to review, along with the list of reference documents of use to the members of the Review Group. These lists will be classified into themes to make it easier to share out the analysis work amongst the different experts in the Review Group.

3.8 Agenda for the presentation day(s)

At least the following points will be tackled:

  • Introduction of the Review Group and its mandate by the Review Group president;
  • general presentation of the project by the CNES and laboratory teams;
  • status of recommendations from the previous review;
  • presentation of the product by the scientific and technical managers (scientific objectives, technical specifications, definition, product assurance, etc. depending on the phase concerned);
  • overview of critical points.

 

3.9 Logistical information

This section provides all the logistical information that is necessary for the participants:

  • address and map,
  • possible transportation and timetables,
  • meal times and places,
  • possible hotels,
  • on-site contact.

 

D. Typical content of a review group minutes

1. Subject of the document

This section recaps the subject of the document, specifying the review concerned.

1.1 Applicable and reference documents

2. Review organisation and proceedings

2.1 Review objectives

This paragraph sets out the objectives identified in the organisation memo.

2.2 Review proceedings

This paragraph goes over the main events in the review (date of the presentation, date of the question/answer session, etc.).

2.3 Review Group membership

This paragraph sets out the members of the Review Group.

3. Assessment of the review objectives

3.1 Global assessment

This paragraph gives a global assessment of how far the review objectives have been met.

3.2 C.2 Objective 1

This paragraph gives an assessment of how far the first review objective has been met.

3.3 C.n Objective n

This paragraph gives an assessment of how far review objective n has been met.

3.4 C.n+1 Risk assessment

This paragraph gives an assessment of the risks identified by the Review Group for the project.

4. Statement of recommendations and major actions

The recommendations are to be presented on sheets (refer to MF-10).

5. Conclusion

This section specifies the Review Group's position on moving forward to the next phase of development.

6. List of actions taken to process RIDs

Risks

RisksCumbersome reviews
 RecommandationsSUGGEST AN APPROPRIATE REVIEW ORGANISATION
Review organisation memo

 

Forms

Activities / documentation

Published in: