Yes, quite a lot!

ECSS-E-ST-10C was produced by the European Cooperation for Space Standardization (ECSS) Secretariat, ESA-ESTEC, Requirements and Standards Division, Noordwijk, The Netherlands. The standard is one of a series of ECSS standards intended to be applied together for the management, engineering and product assurance in space projects and applications. The standard does not limit its statement of intended application to European space projects, although the copyright notice implies this intent.

ECSS-E-ST-10C represents a major departure from previous versions: ECSS-E-10A (1996) and ECSS-E-10 Part 1B (2004), in that ECSS-E-ST-10C contains only requirements, and defers to a new handbook, ECSS-E-HB-10, for informative material.

As a systems engineering standard for space systems, ECSS-E-ST-10C has much to commend it for that application. Applied to other systems, ECSS-E-ST-10C could be overkill, or insufficient, or both. All comments below relate to the application of ECSS-E-ST-10C to high value, complex space projects and applications. An absence of comment represents support for the content of the standard in relation to this application domain. Comments are also made below to highlight particularly good or important parts of the standard.

Broadly, the standard advocates sound practices in most respects. On a scale, that places ECSS-E-ST-10C well ahead of most other public domain systems engineering standards. There are, however, problem areas. Again broadly, these can be summarized as follows:

• an absence of distinction between requirements (per Oxford and Websters English dictionaries), Measures of Effectiveness and goals, leading to a lack of objective criteria against which to eliminate infeasible solution alternatives, and against which to evaluate and select feasible solution alternatives;
• a muddling of problem and solution domains, in places;
• widespread use of vague and undefined terms (e.g. system analysis)
• weak reflection of the principles and practices of concurrent engineering;
• very little coverage of validation;
• the standard gets into trouble by failing to distinguish between systems engineering (the principles and methods of the engineering of systems) and system engineering (the act of engineering a specific system); and
• the incorrect use of some English words (although the standard of English is high overall).

The comments that follow are intended to assist users of the standard. Although some comments may be interpreted as being critical of ECSS-E-ST-10C, such comments should be interpreted in the context of the bigger picture of ECSS-E-ST-10C being overall sound, and seriously conducive to achievement of successful outcomes from space-related projects. Referenced standards have not been evaluated in most cases.

Detailed comments, referenced to clauses in ECSS-E-ST-10C, follow:

3.2.1 The definition of “agreement” defies the English language.

3.2.2 The definition of “critical” is poor, in that it lacks the concept of high importance. Something can deserve control and special attention but not be critical.

3.2.3 The definition of “design to cost” is very unusual. “Design to cost” normally means a situation in which design is to be conducted to maximize value delivery within a cost ceiling, i.e. cost-capability tradeoffs are not possible. The definition in ECSS-E-ST-10C makes every project that has pre-established objectives of cost and time a “design to cost” project. Also, the term is “design to cost” which the definition describes as a method of managing.

3.2.5 The definition of “mission statement” as a “document expressing the set of collected needs” is poor. A mission is an outcome to be pursued, e.g. establish a human colony on mars, move a satellite from one orbital slot to another, repair a defective satellite, etc. Using language inconsistent with community/dictionary meaning inevitably results in confusion.

3.2.6 The definition of “requirements traceability” does not distinguish between requirements traceability in requirements analysis and requirements traceability in design. The definition uses the term “higher level” in a vague way, not distinguishing between physical level versus level of of abstraction. The definition implies requirements traceability in design, but if this is the intent, it is not clear. If it is the intent, it leaves unaddressed requirements traceability in requirements analysis.

3.2.8 The definition of “system engineering” is very poor. The huge body of thought related to systems engineering, as reflected in contemporary standards, handbooks and text books, has systems engineering not as an approach to turning a requirement into a system solution, but an approach to turning stakeholder needs into an optimum solution which satisfies those needs, or a problem description (comprising requirements and other information) into an optimum solution to that problem. The reference to a very old draft IEEE standard is unfortunate. IEEE P1220 draft standard defines “systems engineering”, not “system engineering”.

3.2.10 The definition of “system engineering process” suffers from the issue raised against 3.2.8, viz. the failure to distinguish between “systems engineering” and “system engineering”.

3.2.11 The definition of “technical requirement” as the required technical capability of the product in terms of performances, interfaces and operations is both poor and unfortunate. Firstly, the word “technical” means relating to technique or technology. That is solution. Requirements, amongst other information, define problem. The definition excludes requirements for states, limits on consumption or use of externally provided resources, environmental requirements, required physical characteristics such as mass, required qualities such as reliability, safety, cost or maintainability, and valid solution direction within requirements.

4.1 In the first paragraph, a second definition of system engineering at least refers to requirements (plural), but ignores the other information which defines a problem, viz. Measures of Effectiveness, goals, value relationships, and verification requirements (requirements which define the quality of evidence that each system requirement has been satisfied).

4.1 The definition in the second paragraph is a definition of an engineered system, not a definition of a system in general.

4.1 The second paragraph implies that to be a system, elements of all of the types listed must be included. This is not so. A system may be constructed of any combination of elements of one or more of the types listed.

4.1 The fifth paragraph and related Figure 4.1 are very muddled, misleading and poorly conceived. Major issues are:

• the definition of the scope of requirements engineering (requirements engineering is most usefully defined to embrace requirements capture, requirements validation, requirements specification, the creation of requirements as being in design, and requirements traceability in its two forms: in requirements analysis and in design). Requirements analysis is most usefully and commonly defined to embrace requirements capture, requirements validation, and specification or re-specification as necessary of captured and validated requirements).
• the treatment of requirements engineering as an activity mutually exclusive of analysis and design. There is in fact major overlap – requirements engineering is widely regarded as a discipline, but rarely defined as an engineering process element.
  requirements engineering is stated to incorporate requirements validation and requirements analysis, as if they are mutually exclusive, however, in reality, requirements analysis (analysis of the problem domain) is the single most effective means of performing requirements capture and requirements validation.
• requirements allocation is listed as an activity of requirements engineering and an activity of analysis (analysis of what is not stated). However, requirements allocation is, without exception, an act of solution decision making, i.e. an act of design design.
• the purposes of analysis are stated, but what is to be analyzed (disected into pieces) is mostly left unstated. In reality, analysis is used in almost every human intellectual endeavour, including requirements capture and validation, and design.
• verification is limited to verification of deliverables. Requirements (etc.) verification and design verification are thereby excluded. This is tragic!
• validation (does the work product meet the need for the work product?) is excluded. With development of the “wrong thing” being historically the single biggest cause of failure and loss in engineering, this omission is of profound significance.
• Figure 4-2 uses the term “System Analysis”, a term with dozens of meanings, without definition
• Figure 4-2 shows a box “Functional Analysis and Allocation” separate to a “Requirement Engineering” box. However, functional analysis is a major tool in requirements analysis (requirements capture and requirements validation).
• Figure 4-2 shows a box “Functional Analysis and Allocation” separate to a “Design and Configuration” box. However, functional analysis is a major tool in design, producing functional design. Functional design, for a given solution concept, is an aid to physical design.

4.2 In the fourth paragraph, it is stated that with reference to some phases that “the systems engineering organization derives design-oriented technical solutions”. In fact, the systems engineering organization does not do this. The systems engineering organization in these phases develops one or more designs (descriptions of possible/intended solution).

5.2.1 a. Requirements analysis must determine whether the requirements from the customer are already of a standard such that if these requirements are satisfied, the customer’s and supplier’s needs will be satisfied. If the requirements are not already to this standard, requirements analysis must identify specific issues with the requirements, facilitate the resolution of these issues with the customer, and restate the requirements set to the necessary standard.

5.2.1 b. The paragraph could be stated more clearly and completely by listing the potential types of requirements on lower level elements, viz: state/mode, functional, performance, external interface, environmental, resource, physical (except for software elements), other qualities and design requirements. A clear distinction should be made between sub-system requirements and corresponding sub-system verification requirements.

5.2.3.3 Requirements-related risk (risk having its origins in defective or missing requirements) and solution-related risk (risk having its origins in technology or in the development system – availability of skills, availability of tools) both need to be considered.

5.2.3.4 a. The heading is misleading. The methods referred to are not requirements verification methods (requirements verification is a different issue), they are system verification methods. The system is verified as complying with its requirements.

5.2.3.4 a. This paragraph focuses on verification methods, leaving open the real concern, which is requirements defining the quality of evidence that each system requirement has been met. This paragraph perpetuates bad practice which commonly leads to inadequate system verification, excessive system verification, and/or dispute.

5.2.3.4 b. This paragraph need elaboration to make it clear that the verification methods to be incorporated in the verification matrix (along with system requirements and system verification requirements) must be the product of verification design to satisfy verification requirements.

5.2.3.5 The paragraph implies a bad practice. Verification requirements on lower level system elements should be developed based on first principles, for the element, and verification methods be determined by design as to how best satisfy each verification requirement.

5.2.3.7 This appears to be the first use of “validation” in ECSS-E-ST-10C. The term needs to be defined. Validation against expressed need is usually inadequate, because needs are usually expressed incompletely and often inaccurately.

5.3.1 b. Note that the functional design which is the product of this activity corresponds to a particular physical concept. Functional design is used in developing each concept to foster correctness and completeness of design, to assist in design verification, and to aid in explaining the design to others.

5.3.3 Although the paragraph is titled “Trade-off analysis”, the content below the heading includes design activities related to feasibility (satisfaction of requirements). There is no concept of infeasible solutions being involved in trade-off analysis. Trade-off analysis is concerned with achieving the best solution amongst feasible alternatives. This section suffers enormously from the absence of distinction between requirements, measures of effectiveness, and goals.

5.3.3 a. 4.The term “design requirements” is undefined and appears to be used loosely. “examine alternative technologies to satisfy requirements”.

5.3.3 a. 7.The words “establish the system” need explanation.

5.3.3 a. 10. The word “qualification” need definition.

5.3.4 d. NOTE 2 should be either omitted or explained. How else can exchange take place than via interfaces? What does “direct” mean?

5.3.4 e. NOTE 2 should be either omitted or explained. How else can exchange take place than via interfaces? What does “direct” mean?

5.4 Much of the content of this paragraph duplicates content in 5.2 and 5.3, because of conceptual problems with the content.

5.4.1.1 a.This paragraph implies that functional architecture should precede physical architecture. That is not possible. Even if it were possible, it still would be counterproductive. The sequence must be physical concept, functional design with expected iteration to refine and develop physical, physical design to formalize implementation of functional design. This sequence applies at architectural and detailed levels of abstraction of design.

5.4.1.1 a.Analyses of what?

5.5.2 e. In a paragraph titled “Verification”, the requirement specifies system validation! System validation needs to be a separate section, and the term needs to be defined.

5.6.1 c. The concept intended here is sound but the words are quite misleading. Engineering for producibility is a product engineering activity, not a production activity, or a production system engineering activity. Engineering for operability is a product engineering activity, not an ILS (Integrated Logistic Support) activity. Engineering for maintainability is also a product engineering activity. The concepts and practice of integrated product and process development (IPPD) and concurrent engineering (the concurrent, collaborative and balanced development of a system of interest and one or more enabling systems – e.g. production system) are missing. This is a serious omission.

5.6.4 c. The term “verification requirements” is used for the first time. The term needs definition. An appropriate definition is “verification requirement – a requirement which defines the quality of evidence required that a requirement has been satisfied.”

5..6.9 b. The assessment needs to be an assessment of impact of the nonconformance.

6.2 d. The reference to critical elements needs to be to “critical system elements”, and a relationship established to satisfaction of critical system requirements.

6.4 a. It must be the customer who finalizes the expression of the customer needs, even if only by sanctioning a needs statement prepared by the system engineering organization. To do otherwise is for the customer to loose control of the outcome. In a contractual situation, this can be fatal.

6.7 a. The term “qualification” needs definition. The word is normally used in the sense of “deeming something to be approved for a particular purpose”, often based on verification evidence. Successful verification and validation of the system are more usual criteria for marking completion of system development.

D.2.1 3.1 a. 3. This clause requires that major aspects of the system engineering have been done before the planning of the system engineering has been recorded and communicated. Plan the work then do the work!

D.2.1 3.1 a. 8. The list should be limited to applicable national and international regulations.

D.2.1 3.1 a. 9. The capacity for verification and validation seems not to be a meaningful concept. Capacity could be arbitrarily high depending on how much money is expended.

D.2.1 3.4 This clause requires that major aspects of the system engineering have been done before the planning of the system engineering has been recorded and communicated. Plan the work then do the work!

D.2.1 4.3 a. 2 The reference to system engineers and disciplines engineers reflects a common but rather inefficient way of implementing systems engineering. The most effective implementations in my experience regard systems engineering as a skill set to be possessed and used by all engineers. Splitting the necessary skills between technologists and process specialists, however, is much better than not applying the process skills at all!

D.2.1 5.2 b. 4. The Operations plans listed are (or should be) products of the system engineering – they are descriptions of aspects of solution. This needs to be clearly stated in the SEP, and the plans for developing these plans incorporated in the SEP.

D.2.1 5.3 a. The SEP is looking forward. As such, there is no concept of “uses” (verb), only “is to use”.

E.2.1 4.5 Interfaces “to what?” is not clear. Interfaces of the activities described in the TP to other activities which are the subject of other plans should be described in the TP.

F.2.1 3 This clause muddles up requirements and design. The technology matrix should identify the solution level functions that could/would be performed by each element in the list of technologies or technological elements relating to the subject requirements level function.

G1.2 The second paragraph is unclear as to the scope of information to be included. Is it to include the design of the identification solution, manufacturing solution, operational support solution, maintenance solution, configuration management solution, and removal from service solution? These designs need to exist, need to be recorded, and need to be developed concurrently and collaboratively with the design of “the system”.

G.2.1 3 c. It needs to be made clear to what level of physical decomposition these requirements specifications are to be included. Given the preamble to the standard, one would conclude just one level down. This is perfectly satisfactory, provided the same discipline is applied to the engineering of system elements that are themselves developmental.

G.2.1 7.4 This clause is about Constraints on Maintenance. The title should reflect the content.

H This Annex on Function tree is lacking substance and reflects serious misconceptions.

H.1.2 The function tree is not the starting point for establishment of the Product Tree. The physical concept of solution is the starting point for establishment of the Product Tree. The function tree is a basic representation of the logic of the physical concept. Functions in the function tree, at the levels which correspond to one level of physical decomposition, map to elements in the Product Tree at this level.

H.2.2 The function tree must be consistent with other functional descriptions.

J.2.2 The specification tree must be consistent with the product tree and the business agreement structure.