A client this week asked me a question along the following lines (I’ve edited the question a little to protect the innocent):
- Imagine my top level system is an existing civil aircraft
- Now the engine manufacturer introduces a new build standard (design) of the engine.
- Note that the engine interacts with a number of primary aircraft subsystems including the flight control system, airframe, cabin environmental control system, and electrical power system.
- The client needs to assess whether any changes to the engine build standard could plausibly have affected each of these other aircraft subsystems. The client is aware that his ability to do this will be greatly improved if they have applied sufficient diligence to things in the original aircraft development, such as technology skills, the three types of traceability (requirements, design and verification), standard of requirements definition, standard of solution definition, interface definition and management, configuration management, specialty engineering integration and the use of Integrated Product Teams (IPTs) or similar.
- The client wants to undertake an analysis to determine how much of the previous V&V evidence still stands, and which areas of V&V need to be re-visited. A typical question would be “could the change implemented to the engine plausibly impact the behaviour of the cabin environmental control system?”, for example. Occasionally we will get this wrong, but then hopefully the learning organisation can identify why we got it wrong and strengthen the process for next time round.
He shared an example in a past life of where this went wrong. The lighting system for a car was upgraded in response to reliability problems and less than ideal behavior. The company put the car into production with the lighting system changes, only to discover that the stopping distance had increased. It had not been considered plausible that a change to the lighting system could have impacted the braking system.
What guidance can you give as to how this sort of unexpected interaction might be avoided?
My reply was:
Taking your example of a change in build standard (design) for the engine, the first questions are:
- A. is there purposeful change with respect to engine Problem Definition Baseline (requirements, etc.)? If so, what?
- B. will there be, or could there be, unintended departure from compliance with current engine Problem Definition Baseline?. If so, why and what?
- C. will there be, or could there be, unintended change of wanted and unwanted engine behaviour whilst maintaining compliance with current engine Problem Definition Baseline?. If so, why and what?
Any one or more of these questions could return a “yes”.
The reason for asking/answering these questions is that the impact analysis will be different for A, B and C, as may be any related remedies.
A is the easy one. Return to the original aircraft design that created the engine requirements etc. that are changing. Revisit the design work to establish whether or not the original aircraft requirement(s) will still be met with the changed engine requirements, etc. We cannot generalise as to how this establishment can be performed – it is totally dependent on the design techniques used and applicable to the technologies involved in that aspect of aircraft design.
B necessitates predicting the change in “black box” characteristics of the engine, before proceeding as for A if the engine Problem Definition Baseline is violated. We cannot generalise as to how the prediction of effect on engine characteristics can be performed – it is dependent on the design techniques used and applicable to the technologies involved in that aspect of engine design. But there are techniques that may be useful (see C below).
C is the most problematic, because it gets to the very heart of adequacy of the original aircraft design. C also necessitates predicting the change in “black box” characteristics of the engine and the same comments as for B apply. But now, the challenge is to predict unwanted effects on the behaviour or other characteristics of the aircraft as a result of inadequacies in the design of the aircraft. One cannot generalise as to how this prediction can be performed, as it is heavily dependent on the specifics of the change in engine characteristics and the specific design techniques used and applicable to the technologies involved in relevant aspects of overall aircraft design. But there are general techniques that often can help:
-
- C1: if there is no overall aircraft functional design, formalize the functional design of the aircraft in areas to meet aircraft requirements where possible impact can be conceived. Build the changed engine behaviour into that functional design (control flow and item flow). In doing so, look for detrimental impacts.
- C2: if there is an overall aircraft functional design, build the changed engine behaviour into that functional design (control flow and item flow). In doing so, look for detrimental impacts. Possibly use Control Flow Analysis for stronger aircraft design verification.
- C3: execute the functional design from C2 (or functional design fragments from C1), looking for execution crashes and unwanted behaviour, if C1 or C2 has been done in an executable modeling language. This approach, although not a universal solution and not always possible, is very powerful.
- C4: do a Performance Thread Analysis pairwise between aircraft performance parameters and changed or added “black box” engine functions. Do this with the aircraft design team that is responsible for the first physical level of aircraft design, asking for each pair “is there any mechanism we can identify whereby the act of performing this engine function can affect the achievement of this aircraft performance parameter?”. This technique harnesses to a high degree the knowledge resident in the heads of a team of designers, because of its structured, pairwise approach.
- C5: do C1/C3, but using a constructed simulation rather than an executable design form of simulation. In doing so, look for detrimental impacts.
The emphasis in the above is in predicting the impacts (if any) and potentially dealing with the impacts by redesign, before they become problems or disasters. Designing in quality rather than testing in quality. Model-Based Design plays a large role. Some of the above could be construed as design re-verification. But, with or without redesign, that still leaves the issue of aircraft re-verification after the change is made. The basis for the decision on whether or not to reverify, aircraft requirement by aircraft requirement, is the difference between the predicted risk reduction benefit versus the predicted cost of achieving that benefit, unless re-verification is mandated by airworthiness regulations.
I have intentionally simplified the above, because the degree of risk and cost of the work are ever-present influences on whether we do any of the above at all, and if we do, to what degree.
So, I see the main way to avoid unintended interactions between elements of a system solution, initially and after modification, is to apply a high level of technology knowledge. This knowledge can be leveraged by many aspects of design process, especially tool-enabled MBSE. Do you see it the same way? Are there other angles or approaches, in your experience?
