Guidelines for the Verification and Validation of Expert System Software and Conventional Software
Guidelines for the Verification and Validation of Expert System Software and Conventional Software

Author:

Publisher:

Published: 1995

Total Pages: 122

ISBN-13:

DOWNLOAD EBOOK

This report presents the results of the Knowledge Base Certification activity of the expert systems verification and validation (V & V) guideline development project which is jointly funded by the US Nuclear Regulatory Commission and the Electric Power Research Institute. The ultimate objective is the formulation of guidelines for the V & V of expert systems for use in nuclear power applications. This activity is concerned with the development and testing of various methods for assuring the quality of knowledge bases. The testing procedure used was that of behavioral experiment, the first known such evaluation of any type of V & V activity. The value of such experimentation is its capability to provide empirical evidence for -- or against -- the effectiveness of plausible methods in helping people find problems in knowledge bases. The three-day experiment included 20 participants from three nuclear utilities, the Nuclear Regulatory Commission's Technical training Center, the University of Maryland, EG & G Idaho, and SAIC. The study used two real nuclear expert systems: a boiling water reactor emergency operating procedures tracking system and a pressurized water reactor safety assessment systems. Ten participants were assigned to each of the expert systems. All participants were trained in and then used a sequence of four different V & V methods selected as being the best and most appropriate for study on the basis of prior evaluation activities. These methods either involved the analysis and tracing of requirements to elements in the knowledge base (requirements grouping and requirements tracing) or else involved direct inspection of the knowledge base for various kinds of errors. Half of the subjects within each system group used the best manual variant of the V & V methods (the control group), while the other half were supported by the results of applying real or simulated automated tools to the knowledge bases (the experimental group).

Guidelines for the Verification and Validation of Expert System Software and Conventional Software
Guidelines for the Verification and Validation of Expert System Software and Conventional Software

Author:

Publisher:

Published: 1995

Total Pages: 158

ISBN-13:

DOWNLOAD EBOOK

This eight-volume report presents guidelines for performing verification and validation (V & V) on Artificial Intelligence (Al) systems with nuclear applications. The guidelines have much broader application than just expert systems; they are also applicable to object-oriented programming systems, rule-based systems, frame-based systems, model-based systems, neural nets, genetic algorithms, and conventional software systems. This is because many of the components of AI systems are implemented in conventional procedural programming languages, so there is no real distinction. The report examines the state of the art in verifying and validating expert systems. V & V methods traditionally applied to conventional software systems are evaluated for their applicability to expert systems. One hundred fifty-three conventional techniques are identified and evaluated. These methods are found to be useful for at least some of the components of expert systems, frame-based systems, and object-oriented systems. A taxonomy of 52 defect types and their delectability by the 153 methods is presented. With specific regard to expert systems, conventional V & V methods were found to apply well to all the components of the expert system with the exception of the knowledge base. The knowledge base requires extension of the existing methods. Several innovative static verification and validation methods for expert systems have been identified and are described here, including a method for checking the knowledge base {open_quotes}semantics{close_quotes} and a method for generating validation scenarios. Evaluation of some of these methods was performed both analytically and experimentally. A V & V methodology for expert systems is presented based on three factors: (1) a system's judged need for V & V (based in turn on its complexity and degree of required integrity); (2) the life-cycle phase; and (3) the system component being tested.

Verification and Validation of Rule-Based Expert Systems
Verification and Validation of Rule-Based Expert Systems

Author: Suzanne Smith

Publisher: CRC Press

Published: 2018-10-08

Total Pages: 224

ISBN-13: 149871935X

DOWNLOAD EBOOK

This book presents an innovative approach to verifying and validating rule-based expert systems. It features a complete set of techniques and tools that provide a more formal, objective, and automated means of carrying out verification and validation procedures. Many of the concepts behind these procedures have been adapted from conventional software, while others have required that new techniques or tools be created because of the uniqueness of rule-based expert systems. Verification and Validation of Rule-Based Expert Systems is a valuable reference for electrical engineers, software engineers, artificial intelligence experts, and computer scientists involved with object-oriented development, expert systems, and programming languages.

Guidelines for the Verification and Validation of Expert System Software and Conventional Software: User's Manual
Guidelines for the Verification and Validation of Expert System Software and Conventional Software: User's Manual

Author:

Publisher:

Published: 2006

Total Pages:

ISBN-13:

DOWNLOAD EBOOK

This report provides a step-by-step guide, or user manual, for personnel responsible for the planning and execution of the verification and validation (V & V), and developmental testing, of expert systems, conventional software systems, and various other types of artificial intelligence systems. While the guide was developed primarily for applications in the utility industry, it applies well to all industries. The user manual has three sections. In Section 1 the user assesses the stringency of V & V needed for the system under consideration, identifies the development stage the system is in, and identifies the component(s) of the system to be tested next. These three pieces of information determine which Guideline Package of V & V methods is most appropriate for those conditions. The V & V Guideline Packages are provided in Section 2. Each package consists of an ordered set of V & V techniques to be applied to the system, guides on choosing the review/evaluation team, measurement criteria, and references to a book or report which describes the application of the method. Section 3 presents details of 11 of the most important (or least well-explained in the literature) methods to assist the user in applying these techniques accurately.

Guidelines for the Verification and Validation of Expert System Software and Conventional Software
Guidelines for the Verification and Validation of Expert System Software and Conventional Software

Author:

Publisher:

Published: 1995

Total Pages: 99

ISBN-13:

DOWNLOAD EBOOK

This report is the fifth volume in a series of reports describing the results of the Expert System Verification C, and Validation (V & V) project which is jointly funded by the U.S. Nuclear Regulatory Commission and the Electric Power Research Institute toward the objective of formulating Guidelines for the V & V of expert systems for use in nuclear power applications. This report provides the rationale for and description of those guidelines. The actual guidelines themselves are presented in Volume 7, {open_quotes}User's Manual.{close_quotes} Three factors determine what V & V is needed: (1) the stage of the development life cycle (requirements, design, or implementation); (2) whether the overall system or a specialized component needs to be tested (knowledge base component, inference engine or other highly reusable element, or a component involving conventional software); and (3) the stringency of V & V that is needed (as judged from an assessment of the system's complexity and the requirement for its integrity to form three Classes). A V & V Guideline package is provided for each of the combinations of these three variables. The package specifies the V & V methods recommended and the order in which they should be administered, the assurances each method provides, the qualifications needed by the V & V team to employ each particular method, the degree to which the methods should be applied, the performance measures that should be taken, and the decision criteria for accepting, conditionally accepting, or rejecting an evaluated system. In addition to the Guideline packages, highly detailed step-by-step procedures are provided for 11 of the more important methods, to ensure that they can be implemented correctly. The Guidelines can apply to conventional procedural software systems as well as all kinds of Al systems.