Home » Resources » SyEN Newsjournal » PPI SyEN 53

PPI SyEN 53

Read monthly Project Performance International's Systems Engineering Newsjournal, named "PPI SyEN". PPI SyEN presents for the engineering professional 30-60 pages of valuable technical articles on topical subjects, shorter technical pieces, in-depth coverage of the month's news in systems engineering and directly related fields, pointers to useful resources and relevant industry events, plus limited information on PPI's activities.

Welcome to PPI SyEN 53

In This Edition

Quotations to Open On

Feature Article

The Role of V&V in the Early Stages of the Lifecycle by Steven H. Dam

Articles

What do the Terms Verification and Validation Really Mean? by Lou Wheatcraft

Systems Engineering Applied to Urban Transportation by J. Edward Anderson

Systems Engineering News

How Can We Share Solutions to Complex Systems Problems Across Domains and Application Areas?
Emergence – The Interaction of the Parts of a System
The Authentic Systems Engineer is More Effective
INCOSE Netherlands’ Report on its “SE Experience” Event
China’s Amazing Engineering Marvels
System Integration Case Study Wanted
2017 System Dynamics Summer School at MIT – Application for Scholarship Support

Featured Organizations

Society of Engineering Science

Conferences and Meetings

Some Systems Engineering-Relevant Websites

Systems Engineering Publications

Design in Tech Report, by John Maedea
From Modularity to Emergence: A Primer on the Design and Science of Complex Systems, by Chih-Chun Chen and Nathan Crilly
A Primer on User Experience Design, by Joseph Dickerson
Societal Trends in Technology, by Bruce Sterling
16 Rules of Engineering Design, by J. Edward Anderson
Systems Engineering with SysML/UML: Modeling, Analysis, Design, by Tim Weikiens

Systems Engineering Tools News

Basics of The Systems Modelling Language (SysML)
MagicDraw

Education and Academia

Changes to be Brought in Engineering Syllabi to Meet Industry Requirements
The Optimum Place and Time to Introduce Systems Thinking is at Middle-School Level
Future Engineering Careers will Depend on Present Education in Cyber-Physical Systems and IoT Integration
The Mastership Program for Engineers in Training
Systems Architecting and Engineering at the University of Southern California (USC)

Standards and Guides

Systems Engineering Standards: A Summary

Definition to Close On

Systems Engineering

PPI and CTI News

PPI Upcoming Participation in Professional Conferences

PPI and CTI Events

Quotations to Open On

The following quotations are attributed to Lou Wheatcraft, author of an article in this issue of SyEN:

“Requirements are the common thread that ties all system development lifecycle activities and work products together.”

“Writing requirements is an exercise in engineering, not an exercise in writing.”

“Requirements should not, and must not, be viewed in isolation, but rather viewed as how they relate to the whole.”

“We are 17 years into the 21^st century, yet many are still trapped in 20^th century thinking when it comes to requirements and systems engineering.”

“Technology is evolving at a rapid rate, especially information technology, not only resulting in more complex systems, but also enabling the documentation, management, and integration of large datasets that represent the many work products and underlying data and information generated as part of the SE lifecycle process activities.”

Feature Article

The Role of V&V in the Early Stages of the Lifecycle

by Steven H. Dam, Ph.D., ESEP

Abstract

System Verification and Validation (V&V) occur in the later parts of the system lifecycle to ensure that the requirements developed in the early phases of the lifecycle have been met. This paper discusses how to use the techniques of V&V, including simulation, early in the lifecycle to enhance the probability of success of the program by identifying errors early in the development and preparing for the V&V activities later in the lifecycle.

Email: steven.dam@specinnovations.com

Introduction

Having just watched the movie “Deepwater Horizon” (Achenbach, 2016) I observed the results when proper testing is skipped. If you have not seen the movie, several routine and critical tests were skipped and another test was performed with mixed results. The mixed results test had the result of people picking the optimistic conclusion, which turned out catastrophic.

We have seen these problems in many other instances. Just look at NASA Mishap Reports (Larson, 2009). They contain example after example of failures all through the lifecycle, often due to skipped tests. Such defects can be identified earlier in the lifecycle if modern modeling and simulation techniques are used. Unfortunately, most systems engineering and architecture work is done using the equivalent of Microsoft Office, which is fantastic for word processing, spreadsheets, and drawing tools, but wholly insufficient for engineering modeling and simulation.

The Lifecycle Model

If you are a systems engineer, you are likely familiar with the “V” lifecycle model. (Rook, 1986) Our version of the V is shown in Figure 1 below. This diagram shows the phases of the lifecycle as we decompose the system to a level from concept to where we can specify and build the components of the system. Then it shows the integration of the components to build up the system, while verifying and validating that the system meets the requirements.

Figure 1. The “V” Lifecycle Model

The “V” shape reminds us of the parallel V&V activities that are affected as we decompose the system. For example, during the architecture development, it is important to derive the acceptance criteria as the basis for the operational test and evaluation (OT&E) and transition activities. A draft test and evaluation master plan (TEMP) can and should be produced during this architecture development phase. Unfortunately, this process often is not done at all or it is done in a way that renders it effectively useless.

In addition to this lack of test planning, often the architecture development takes the form of creating a series of drawings based on a limited language, such as Systems Modeling Language (SysML). The nine SysML drawings are necessary, but not sufficient, to capture all the information about the system, which includes the risks, decisions, costs, and other parameters needed to fully describe the system. The work also usually focuses on operations and ignores maintenance and various potential failure modes of the system. In addition to these problems, the drawings are usually just treated as static depictions of processes and components. Those drawings can contain significant errors in logic that do not become obvious until the equivalent phase coming up the “V.” As is well known, finding errors later in the lifecycle is very expensive and may even lead to program cancellation. (Albrecht, 2017).

Deriving Requirements from Scenarios

Often in the architecture development phase, requirements, particularly functional requirements, have not been developed to a sufficient level that we can derive the verification requirements needed to begin test planning. In view of this fact, scenario analysis is often used to develop models of the operations (and maintenance) of the system, from which the functional requirements for the system can be identified. The challenge in using scenario analysis is evolving a set of scenarios that will ensure that the breadth and depth of the functionality will be developed. A number of years ago, I developed an approach that applied the idea of a test matrix to the development of scenarios for architecture analysis. The basic strategy was to develop a list of scenarios and a set of characteristics of these scenarios (Dam, 2007). These two pieces of information would then form a matrix, similar to a test matrix where you have the test scenarios juxtaposed with the parameters you want to measure. An example is shown in Figure 2. The example shown is a fairly simplistic example, but this can be expanded and used for much larger problems.

Figure 2. An Example of a Scenario Matrix

Once we have identified these scenarios, then we can build models of each. Figure 3 shows an Action Diagram of the first scenario for the table above (Dam, 2014).

Figure 3. Example of Scenario 1 Action Diagram Model

These models can then be used to derive the functional requirements for the system. Several tools automate this process by reading the functional elements of the models and their relationships to physical entities, such as the system, and then produce requirements and/or documents that can be used as the basis for the system specification.

But, how do we know this model works? Could we be building errors in the logic in these diagrams? The answer to these questions comes from further analysis using simulation techniques.

Application of Simulation

We are very familiar with simulation as one of the methods for V&V, along with analysis, inspection, demonstration, and test. We can apply simulation in the early phases of the lifecycle by applying various simulation techniques to the models developed in the previous section. Discrete Event Simulation (DES) uses “a mathematical/logical model of a physical system that portrays state changes at precise points in simulated time” [Albrecht]. Unfortunately, many of the modeling tools do not have an embedded DES capability, so models developed using the drawing technique of choice often have to be redeveloped in the simulation tool. The question becomes “Is this the same model?” In general, the answer is probably no. Just as when we had a diagram to a programmer and receive code back, the simulation modeler will make changes that often are not documented to fix problems with the original model. Fortunately, several tools [Tools] have both the modeling capability and a full DES. An example of the execution of the model is shown below in Figure 4.

Recently, I have discovered that given the uncertainty in the timing of each step of the process, the probabilities of failure paths, the variability of resources, and many other factors, DES is insufficient to estimate the potential range of capabilities of the systems being modelled. However, if we apply another simulation technique in conjunction with DES, we can easily include these uncertainties in the modeling. This technique is called Monte Carlo simulation. Monte Carlo simulation, the name given by John van Neumann and Stanislaw M. Ulam to reflect its gambling similarity, utilizes models of uncertainty where representation of time is unnecessary. The term originally attributed to a situation in which a difficult non-probabilistic problem is solved through the invention of a stochastic process that satisfies the relations of the deterministic problem. A more recent characterization is that Monte Carlo is the method of repetitive trials. This latter definition fits our interest in that what we want to do is to repeat the simulation produced by the DES, but vary the points within each distribution to see the impacts of the uncertainties in the timing, resources, branching, and other probabilistic elements of the simulation. Figure 5 shows an example of the Monte Carlo results of executing the same scenario shown in Figure 3.

Figure 4. Example of Discrete Event Simulation Output from Scenario 1

Figure 5. Example of Monte Carlo Simulation Output from Scenario 1

Figure 5 is a Time Tree Map, which provides the mean and standard deviation of each step of the process. The Time Bar Chart shows the distribution in time of the number of runs that occur within the specified time bins. This example was only executed in the model 100 times. To get a more accurate estimate (higher confidence interval), you can run the simulation many more times (Driels, 2004).

Test Planning

Another V&V activity that can at least begin early in the lifecycle is the test planning. As you can imagine, many final operational tests must be conducted at specialized ranges that provide the necessary test equipment and telemetry capabilities needed for final test. These ranges often must be scheduled years in advance. In addition, you may have to develop specialized test equipment and/or arrange for experts/users to participate in this kind of testing. Such tests are also expensive and must maximize the value. These reasons alone drive the need for early test planning.

Figure 6 shows an example of how to capture test expectations and results in a modeling tool.

Figure 6. Example of Capturing Test Plans and Results in a Modeling Tool

Future V&V

Having the requirements, combined with the models and V&V test plans, enables the potential for automating the V&V process. We can run a simulation against the test case using well defined performance parameters, including time, resource usage, and cost. Using the Monte Carlo simulation provides a confidence interval that can be improved over time as better data become available through the lifecycle process. This capability can then track the progress in improvement of the performance parameters from the initial estimation into final test results. The software industry has had the capability to conduct automated testing for some time due to their use of integrated development environments (IDEs). What is needed is an IDE for systems engineering. The capabilities discussed above come very close to this desired future state.

Summary

Using V&V techniques and effective planning for V&V early in the lifecycle provide many benefits to any project. Although it costs a little more up front, by discovering errors early we save problems later in the lifecycle and even save lives once the system is deployed. Applying these techniques and planning requires understanding the needs of the V&V personnel and thus should encourage the involvement of people with these skills early and throughout the lifecycle.

References

Achenbach, J. “’Deepwater Horizon’ movie gets the facts mostly right, but simplifies the blame.” The Washington Post. September 29, 2016. Accessed August 2, 2017. https://www.washingtonpost.com/news/achenblog/wp/2016/09/29/deepwater-horizon-movie-gets-the-facts-mostly-right-but-simplifies-the-blame/?utm_term=.34ca084b11e5.

Larson, W. J. et al. (2009). Applied Space Systems Engineering. editors, McGraw-Hill Education.

Forsberg, K. and Mooz, H. (1991). The Relationship of Systems Engineering to the Project Cycle. First Annual Symposium of the National Council on Systems Engineering (NCOSE).

Albrecht, M. C. (2010). Introduction to Discrete Event Simulation. Retrieved from http://www.albrechts.com/mike/DES/Introduction%20to%20DES.pdf.

Dam, S. H. (2007). Intelligent Operational Scenarios – a Strategy for Cost-Saving Scenario Selection. 17^th Annual INCOSE International Symposium.

Dam, S. H. (2014). DoD Architecture Framework 2.0. SPEC Innovations, 144.

[Tools] Two such tools are Vitech’s CORE and SPEC Innovations’ Innoslate.

Driels, M.R. (2004). Determining the number of iterations for Monte Carlo simulations of weapon effectiveness.

Article

What do the terms Verification and Validation Really Mean?

by Louis S. Wheatcraft

Introduction

A question I hear often is “What is difference between verification and validation?”

Note: In the following discussion, there is an underlying assumption that stakeholder needs and expectations are baselined in a scope review or similar gate review prior to developing the design-to requirements. There is a lot of work and analysis that needs to be done before requirements can be written. Part of the baselining process includes 1) ensuring the problem is well understood that the system under development is to address, 2) ensuring the Need, goals, and objectives are clearly defined and agreed to by the relevant stakeholders, and 3) defining a feasible concept that will result in the agreed-to Need, goals, and objectives being met. To be feasible, it must be proven that the set of resulting stakeholder needs and expectations for the concept can be realized within defined drivers and constraints (e.g., cost, schedule, technical, legal, ethical, regulatory, and interaction with existing systems) with acceptable risk.

Website: www.reqexperts.com

Email: louw@reqexperts.com

Why the Terms Verification and Validation Are Frequently Misunderstood

In general, verification refers to the basics (structure) of the item (requirements, design, system) being verified, making sure it meets requirements that drive the creation of the item, whether it be rules on writing well- formed requirements, standards and best practices (external and internal) on the design, or requirements on the coding or manufacturing of the system. Validation goes beyond the basics (structure) to how well the item communicates or addresses stakeholder needs and expectations while operating in the intended operational environment.

While these terms are commonly used, the true meaning of the concepts represented in each are often misunderstood and the terms are often used interchangeably without making clear the context in which they are used – resulting in ambiguity. This is true particularly when the terms are combined into the generic terms verification and validation (V&V); independent V&V (IV&V); or integration, verification and validation (IV&V).

Using the terms interchangeably with the assumed or implied meaning and context leads to confusion and misunderstanding. When referring to IV&V, the ambiguity in the general use of the terms verification and validation creates even greater difficulties, especially if an organization is being contracted to perform “IV&V”. First, the ‘I’ in IV&V is often used to mean ‘independent’ V&V in which an outside organization is called in to undertake ‘V&V’ as an activity. However, what is the assumption concerning what each term do the “Vs” refers to: “validation and verification” or “verification and validation”? Verification of what? Validation of what? Without a qualifying adjective, context is assumed and it is therefore not clear what ‘V&V’ refers to: requirements, design, or the designed and built system.

Sometimes, however, the ‘I’ means ‘integration’ when referring to the right-hand side of the systems engineering ‘Vee’ model that depicts the processes of integration, verification, and validation (see the discussion concerning the Vee Model below for explanation). By not being precise in the use of the terms and not indicating the context intended, confusion can result.

To avoid this ambiguity, each term needs to be preceded by a modifier (i.e., the subject) which denotes the proper context in which the term is being used, specifically requirement verification or requirement validation; design verification or design validation; system verification or system validation as shown in Figure 1. The concepts of verification and validation are very different depending on the modifier. When using these terms, it should be clear as to which concept is intended.

Context example: As shown in Figure 1, a requirement set results from a formal transformation of stakeholder needs and expectations. Correspondingly, design is a result of formal transformation of the requirement set in to an agree-to design, and a system is a formal transformation of the design into that system.

The process of creating a requirement set involves:

analyzing stakeholder needs and expectations to obtain the necessary elements to be included in the requirement set;
identifying the characteristics of the desired result against the organizational guidelines and rules by which the requirement statements and requirement set is to be written, and
transforming the stakeholder needs and expectations into a set of requirements that unambiguously communicates the stakeholder needs and expectations to the design organization (requirement validation) and conforms to the organizations guidelines for writing well-formed requirements and requirement sets (requirement verification).

Figure 1. Verification and validation are the processes of confirming that systems engineering artifacts generated during the transformation processes are acceptable.

Requirement Verification and Validation Defined

In this context, Requirement Verification confirms, by inspection, that the requirements contain the necessary elements and possess the characteristics of a well-formed requirement, and that the requirement set conforms to the rules set forth in the organization’s requirement development guidelines. Requirement Validation confirms, by inspection and analysis, that the resulting requirement set meets the intent of the stakeholder needs from which the requirements and requirement set was decomposed or derived. Thus, the requirement statements and the requirement set are confirmed by both verification and validation activities.

Based on this discussion, to help remove the ambiguity in the use of the terms “verification” and “validation”, the following definitions of these terms are included in terms of a product life cycle context:

Requirement Verification: the process of ensuring the requirement meets the rules and characteristics defined for writing a good requirement. The focus is on the wording and structure of the requirement. “Is the requirement worded or structured correctly in accordance with the organization’s standards, guidelines, rules, and checklists?”.
Requirement Validation: confirmation that the requirements and requirement set is an agreed-to transformation that clearly communicates the baselined stakeholder needs and expectations in a language understood by the developers. The focus is on the message the requirements and requirement set is communicating. “Does the requirements and requirements set clearly and correctly communicate the baselined stakeholder expectations and needs?” “Are we doing the right things?” or “Are we building the right thing [as defined by the requirement set]?”

Requirement verification and requirement validation activities should be done continuously as one develops the requirements at each level of the architecture and as part of baseline activities of the requirement set performed during the System Requirements Review (SRR) or similar type of gate review at each level.

Note: Most organizations do not make a distinction between requirement verification vs requirement validation. Rather they use only the phrase “requirement validation” to mean both. Using the phrase “requirement verification” often confuses the conversation in that many interpret “requirement verification” to have the same meaning as “system verification” as defined later.

Design Verification and Validation Defined

Figure 1 also carries these concepts forward to design and realization of the system under development.

Once the requirements set is baselined, the requirements are transformed into a design of the system. Most organizations have a set of “design guidelines” or “golden rules” that guide the design process. These represent best practices and lessons learned the design team is expected to follow. As part of the design process, the design team may develop prototypes or engineering units. They will use these to run tests to fine tune their design.

After the system design is complete, there is usually a gate review or series of reviews where the design is both verified and validated (e.g. System Design Review (SDR), Preliminary Design Review (PDR), Critical Design Review (CDR). In this context, design verification has two aspects: 1) Does the design clearly represent the requirement set that drove the design? and 2) Did the design team follow the organization’s guidelines for design? Also as part of the gate review, design validation is addressed to determine whether the resulting design for the system, when implemented, will result in the intended purpose being met in the operational environment and the baselined stakeholder’s expectations and needs being met.

Frequently, during the gate reviews, the design team “pushes back” on requirements that proved difficult to meet or were deemed not feasible as defined earlier. This results in proposed changes to the requirements, which are submitted to the configuration control authority for the system for approval. When this happens, not only do the requirements needed to be changed, but also the stakeholder expectations and needs from which the requirement were derived may need to be examined, which could result in a change to the baselined scope.

Design verification and design validation activities should be done as part of a continuous process during the design phase at each level of the architecture as well as during the baselining of the design in the gate review(s).

Based on this discussion, to help remove the ambiguity in the use of the terms “verification” and “validation”, the following definitions for design verification and design validation are included in terms of a product life cycle context.

Design Verification: the process of ensuring the design meets the rules and characteristics defined for the organization’s best practices associated with design. The focus is on the design process. “Did we follow our organizations guidelines for doing the design correctly?” The design process also includes ensuring the design reflects the design-to requirements. Thus, design verification is also a confirmation the design is an agreed-to transformation of the design-to requirements into a design that clearly implements those requirements correctly. “Does the design clearly and correctly represent the requirement set?” “Did we design the thing right?”
Design Validation: confirmation the design will result in a system that meets its intended purpose in its operational environment. Will the design result in a system that will meet the stakeholder needs and expectations that were baselined during the scope definition phase? The focus is on the message the design is communicating. “How well does the design meet the intent of the requirements?” “Do we have the right design?” “Are we doing the right things?” “Will this design result in the baselined stakeholder expectations and needs being met?”

System Verification and Validation Defined

Once the design is baselined, the design is transformed; via build, code, buy, or reuse; into the system of interest. Similar to the discussion for the design process, most organizations have a set of guidelines or “golden rules” that guide the build (manufacture or code) process. These include workmanship and quality control requirements for the organization.

After the system has been built or coded, there will be a gate review where the system is both verified and validated. At this stage of the system lifecycle, the concepts of system verification and system validation take on a more formal meaning. Thus, the Systems Engineering (SE) lifecycle processes include the processes of System Verification and System Validation. Each process represents a set of activities (test, demonstration, inspection, analysis) that cumulate with one or more gate reviews associated with the acceptance of the system by the customer. Thus, System Verification is a formal SE process and has a legal aspect where the developer is proving the system reflects the baselined requirements have been meet. From a contracting perspective, the baselined requirements are a type of contract and are legally binding.

In this context, system verification has two aspects: 1) Does the built or coded system of interest clearly represent the requirements that drove the design? Did we build the right thing? and 2) Did the build or code team follow the organizations guidelines for manufacturing and coding?

Following system verification, system validation is performed. Again, System Validation is a formal SE process and has a legal aspect where the developer is proving whether or not the built or coded and verified system, results in the intended purpose being met in the operational environment and the baselined stakeholder’s expectations and needs being met. Like the system requirements, the baselined stakeholder needs and requirements defined during the scope definition phase can also be considered part of a contract and are legally binding.

Based on this discussion, to help remove the ambiguity in the use of the terms “verification” and “validation”, the following definitions for system verification and system validation are included in terms of a product life cycle context.

System Verification: a process done after design and build or coding, making sure the designed and built or coded system meets its requirements. The focus is on the built or coded system and how well it meets the agreed-to requirement set that drove the design and fabrication. Methods used for system verification include: test, demonstration, inspection, or analysis. “Did we build the thing right?” Also included in system verification is a determination that the team responsible for building or coding the system of interests followed the organization’s rules, guidelines, and best practices associated with manufacturing and coding. The focus is on the manufacturing or coding processes. “Did we follow our organizations guidelines for manufacturing or coding correctly?”
System Validation: a process that occurs after system verification that makes sure the designed, built, and verified system meets its intended purpose in its operational environment. The focus is on the completed system and how well it meets the baselined stakeholder needs and expectations that were defined and baselined during the scope definition phase. “Did we build the right thing?”

System verification and system validation processes are directly related to the contractual obligation concept for a requirement statement and set of requirements. It is through these process activities that we prove we have met both the agreed-to requirements and the agreed-to needs of the stakeholders who are the source of or own them. This is often accomplished as part of certification and acceptance activities.

Verification and Validation and the Systems Engineering “V” model

While the previous section presented definitions that are useful in helping address the issues of ambiguity in the use of the terms verification and validation, the system engineering process is more complex. Systems Engineering is an iterative and recursive process. Requirements development and design occur top-down as shown on the left side of the SE “V” as shown in Figure 2.

Figure 2: Verification and Validation and the Systems Engineering “V” Model

Systems engineering starts with the concept stage where stakeholder needs, expectations, and requirements are elicited, documented, and baselined. This is part of defining the scope of the system to be developed. Next, the stakeholder needs, expectations and requirements are transformed into a set of system requirements which are baselined via requirements verification and requirements validation as discussed above. Once the system requirements are baselined, design results in a system architecture in which the subsystems are defined. The design at this level is baselined via the design verification and design validation process discussed above.

For each subsystem, the above cycle is repeated with the definition of stakeholder subsystem needs, expectations, and stakeholder requirements, and then transformed into subsystem requirements, which are baselined via requirements verification and requirements validation. Once the subsystem requirements are baselined, design results in a subsystem architecture in which the units/components are defined. This design is baselined via the design verification and design validation process. This process repeats until the organization makes a buy, build, code, or reuse decision.

System integration, system verification, and system validation (IV&V) occur bottom-up as shown on the right side of the SE “V” as shown in Figure 2.

Once all the components that make up the subsystems are developed, unit/component verification and unit/component validation take place as described above. Once these activities are complete, the units/components are integrated together and then the resulting subsystems are verified and validated as described above. Once the subsystem verification and subsystem validation activities are complete, the subsystems are integrated together and then system verification activities are completed. In the end, proof will be documented that can be evaluated by the customer to determine system verification activities have been completed successfully showing that the stakeholder requirements have been met (both organizational/people requirements as well as the technical requirements).

Following system verification activities, system validation activities are performed. This could be done in the form of acceptance and/or operations evaluation and validation activities. In the end, proof will be documented that can be evaluated by the customer to determine whether system validation activities have been completed successfully, stakeholder needs have been met, and the system will operate as intended in its operational environment.

Following the customer evaluation, the system can be accepted and ownership transferred to the customer.

Parting thoughts

As shown in this article, while the concepts of verification and validation are commonly used, the words are often used interchangeably and the meaning of the concepts is often misunderstood resulting in confusion. This article addressed the use of the terms verification and validation and presented their various meanings in everyday usage. In particular, it was identified that both terms are ambiguous unless a modifier is included in front of the word clearly indicating what the context in which term is referring to, specifically: requirements, the design, or to the system under development. The meaning is also ambiguous unless it is clear whether the terms are used to refer to an activity as part of a process or to an outcome of that process.

To help resolve these ambiguities, illustrations and a discussion were provided showing how the concepts of verification and validation are used throughout the product development lifecycle. Based on this discussion the point was clearly shown that the meaning of these terms changes depending on context and where you are in the system development lifecycle. Specifically, it was shown that by including a modifier with each term that clearly indicates the context in which the terms are used, the reasons for ambiguity and misunderstanding are removed. Definitions were provided in order to disambiguate the use of each term.

It is important to understand that both activities of verification and validation need to be performed continuously for work products produced across all lifecycle processes. A key advantage of using analytical models is that they enable continuous verification and validation helping to ensure correctness, completeness, and consistency of lifecycle work products. Projects need to make sure their Systems Engineering Management Plan (SEMP) addresses these concepts correctly.

A final thought: The discussion above reflects the “formal” system verification and validation processes. We must also be aware of the “informal” system validation that takes place after a product is released for use. The informal system validation is in the form of product recalls, product returns, warranty costs, reviews on social medial, and treatment in the news media. If organizations took the formal verification and validation activities and processes seriously during product development, the number of issues that occur after the product is released for use would be significantly reduced – avoiding the negative “informal” system validation we frequently observe. All product failures occurring after product release can be traced to a defective verification and validation processes.

References

INCOSE-TP-2003-002-04, 2015, Systems Engineering Handbook: A Guide for System Life Cycle Processes and Activities. Version 4, Edited by Walden, David D., et al.

INCOSE-TP-2010-006-02, 2015, Guide to Writing Requirements, Prepared by the Requirement Working Group, INCOSE.

ISO/IEC 15288, ISO/IEC 15288:2015, 2015, Systems Engineering—System Life Cycle Processes.

ISO/IEC 29148, 2011, ISO/IEC 29148 FDIS Systems and Software Engineering—Life Cycle Processes—Requirements Engineering.

Ryan, M.J., Wheatcraft, L.S.; Dick, J.; and Zinni, R., 2015. “An Improved Taxonomy for Definitions Associated with a Requirement Expression”, Systems Engineering and Test and Evaluation Conference SETE 2014, Adelaide, Australia, 28-30 April 2014 and INCOSE IS 2015, Seattle, WA 12-17 July 2015.

Ryan, M., Wheatcraft, L., “On the Use of the Terms Verification and Validation”, November 2016.

Wheatcraft, L.S., 2012, Thinking Ahead to Verification and Validation, presented at NASA’s PM Challenge 2012, February 2012, Orlando, Florida.

ARTICLE

Systems Engineering Applied to Urban Transportation

by J. Edward Anderson, PhD, P.E.

Since the early 1950’s a growing number of engineers and planners have recognized that, short of teleportation, the best way to make a significant improvement in urban transportation involves some form of Personal Rapid Transit (PRT). However, more than 40 PRT systems have appeared, some with substantial funding, and then disappeared. As Chairman of four International Conferences on PRT, Dr. J. Edward Anderson carefully studied each of these systems and found that, while they accepted and indeed promoted the basic PRT concept – small vehicles, exclusive guideways, off-line stations, and automatic control – they disappeared, usually because they either did not understand how to design the control system or they neglected some important requirement in the design of the guideway. The only PRT concept that Dr. Anderson found fully acceptable is the one designed by The Aerospace Corporation, which was formed in 1960 to manage ballistic missile programs for the United States Air Force, and likely had an outstanding systems engineering capability. The system Dr. Anderson proposes, the Intelligent Transportation Network System (ITNS), draws heavily on the Aerospace Corporation work and has been improved by further experience and by advances in technology that were not available when they initiated their work on PRT in 1968.

The presentation that is summarized in this article is based on over 40 years of involvement in PRT politics, planning, development, design, and study of almost every PRT system proposed. It is the briefest summary of Dr. Anderson’s work, which on several occasions he has presented in 10 two-hour lectures. This article provides some of the highlights of Dr. Anderson’s presentation available here (slides Suja sent to Ralph).

The next step to commercialization of ITNS is the construction of a demonstration system consisting of an oval guideway large enough to reach 35 mph. The guideway will have one off-line station and at least three vehicles operating. Such a system is sufficient to test and demonstrate all technical features of the system. A sum of US$30,000,000 will enable the construction and operation of such a system as well as providing for the development of a marketing team sufficient to secure applications and to demonstrate by computer simulation systems of any size and configuration. Recently, an investor has agreed to provide the needed funds under the condition that a local entity sign a standby letter of credit for US$3,000,000. Based on decades of work on all aspects of ITNS Dr. Anderson is extremely confident that the recommended program will succeed in commercialization of ITNS. The provider of the SBLC will also have to be convinced. Therein lies the rub. The ITNS team is working with planners in several cities in the United States. Because congestion is growing everywhere with no practical solution with conventional technology, the team is confident that it will succeed, but as is true with any new technology, there can be no guarantees. If not, cities will continue to struggle with means of urban transportation that are no different in concept from systems introduced 120 to 140 years ago. Systems engineering is the key to a bright future.

Email: jea.p.e.phd@gmail.com

Systems Engineering News

How Can We Share Solutions to Complex Systems Problems Across Domains and Application Areas?

Engineers, designers, managers, and policy-makers across all sectors need to be able to think in terms of complex systems to be able to address the problems that we are facing. For those seeking to understand, manipulate, or build complex systems, their ‘complex’ nature often demands approaches going beyond reductionist scientific models or traditional engineering design methods. These complex systems are often considered to be intractable because of their unpredictability, non-linearity, interconnectivity, heterarchy, and ‘emergence’. Although in many cases these attributes are framed as a problem, there are also cases in which they can be exploited to encourage intelligent, robust, self-organizing behaviors. To address problems with translating between disciplines, Chih-Chun Chen and Nathan Crilly at the Cambridge Engineering Design Centre have published ‘systems’. This introduces complex system constructs by building them up from basic concepts, and contrasting them with more familiar constructs that are associated with modularity.

PPI SyEN 53

Subscribe to PPI SyEN

PPI SyEN 141

PPI SyEN 140

PPI SyEN 139

PPI SyEN 138

FREE Original SE Articles and Industry News?

Subscribe to PPI

MORE About Us