NAP-29, Technology Readiness Assessments

NNSA POLICY LETTER NAP-29 Approved: 12-22-16 TECHNOLOGY READINESS ASSESSMENTS f,A~~~Sl§i National Nuclear Security Administration NATIONAL NUCLEAR SECURITY ADMINISTRATION Office of Cost Estimating and Program Evaluation CONTROLLED DOCUMENT OFFICE OF PRIMARY INTEREST (OPI): AVAILABLE ONLINE AT: Office of Cost Estimating and Program Evaluation https://nnsaportal.energy.gov/intrnnet/na-mb/na-mb-20/pages/nnsa-policv.aspx printed copies are uncontrolled THIS PAGE INTENTIONALLY LEFT BLANK NAP-29 12-22-16 TECHNOLOGY READINESS ASSESSMENTS 1. PURPOSE. This NNSA Policy Letter (NAP) describes the processes and procedures for conducting Technology Readiness Assessments (TRAs) for any given acquisition process. 2. CANCELLATION. None. 3. APPLICABILITY. a. Federal. This applies to all National Nuclear Security Administration (NNSA) Elements. b. Contractors. Does not apply to contractors. c. Exemption. Not applicable. d. Equivalency. In accordance with the responsibilities and authorities assigned by Executive Order 12344, codified at 50 United States Code sections 2406 and 2511, and to ensure consistency through the joint Navy/Department of Energy (DOE) Naval Nuclear Propulsion Program, the Deputy Administrator for Naval Reactors (Director) will implement and oversee requirements and practices pertaining to this NAP for activities under the Director's cognizance, as deemed appropriate. 4. SUMMARY OF CHANGES. Not applicable. 1 5. BACKGROUND. A TRA is a systematic, metrics-based process that assesses the maturity of, and the risks associated with, critical technologies used in meeting product realization goals for a specific product. TRAs inform program and project acquisition decisions and technology maturation planning using objective knowledge gathered from subject matter experts (SMEs) and compiled into a standardized scoring scale. TRAs use Technology Readiness Levels (TRLs) to measure the state of the technology development from early research on basic principles through large-scale testing and evaluation prior to commercial deployment. TRA activities start with defining the purpose and scope of the assessment, continue through the selection of critical technologies and evaluation process, and end with the final report and subsequent technology maturation planning. In conjunction with this TRA exercise, and its associated products or artifacts, an independent assessment of the TRA is executed to ensure that the TRA has been conducted in an unbiased manner and its results are acceptable to acquisition executives. This independent assessment is based on metrics and best practices from industry and its conclusions are captured in an evaluation memo. Throughout this NAP, unless otherwise specified, references to TRA requirements are the responsibility of the TRA team. 2 NAP-29 12-22-16 6. REQUIREMENTS. a. NNSA Elements must develop policy on what projects, programs, or acquisition processes require a TRA, unless otherwise directed by broader DOE or NNSA directives. These procedures will provide guidance on the execution of TRAs in any given acquisition process in support of milestone decisions or at the direction of the Administrator. b. Programs must conduct TRAs using this NAP at critical decision points for projects meeting the following criteria: (1) Prior to Phase 6.2 authorization: Major Atomic Energy Defense Acquisition (MAEDA) programs managed under the Phase 6.X process where the total program cost is greater than $500 million or the total lifetime life-cycle cost is greater than $1 billion.

(2) Prior to Critical Decision-I and Critical Decision-2: Major System Acquisitions covered by DOE Order (0) 413.3B, Program and Project Management for the Acquisition of Capital Assets (or successor order) with an estimated total project cost (TPC) greater than or equal to $750 million. (3) At other decision points for milestone approval as mandated by the NNSA Supplemental Directive (SD) 452.3-2, Phase 6.X Process and DOE 0 413.3B (or successor policies). (4) As directed by the Administrator or the Project Management Executive, . for first-of-a-kind engineering endeavors. c. The TRA must be planned and resourced for a comprehensive analysis, completed prior to the appropriate milestone as defined by the applicable acquisition policy (for example, designated acquisition milestone, gate review, critical decision event, etc.). d. TRAs must document assessment objectives to include the applicable milestone decision event or the acquisition executive direction the TRA will support. e. TRAs must have a Team Lead accountable for the TRA process and results. Additionally SMEs representing critical technologies under evaluation must be identified. In the absence of a federal Team Lead, the TRA must have a named federal employee accountable for the TRA results. f. Commensurate with the size and scope of the effort, the following TRA information is required. A single artifact or group of artifacts may be provided to satisfy these requirements: ( 1) TRA Schedule/Plan. A TRA schedule and action plan must be developed for the complete assessment process. NAP-29 12-22-16 3 (2) Assessment Tool (TRA Calculator). TRA calculator(s) must be developed and applicable to the context and Critical Technology Elements (CTEs) under analysis. The TRA calculator must follow the format of the DOE TRL scale contained in the approved DOE TRL guide (DOE Guide (G) 413.3-4A, Technology Readiness Assessment Guide or successor), and include other readiness level factors as deemed necessary to fully characterize the maturity risk of the CTE(s) for the supported decision, event, or executive tasking. Some of these factors may include: (a) Technology Readiness Level (TRL), (b) Manufacturability Readiness Levels (MRLs ), ( c) Programmatic Readiness Levels (PRLs ), ( d) System Integration Readiness Levels (SRLs ), ( e) Integration Readiness Levels (IRLs ), and (f) Nuclear Readiness Levels (NRLs). (3) TRA Final Report. The TRA must produce a TRA final report containing the results of the analysis, a list of the TRA team members and their roles, the assessed scoring per CTE to include supporting rationale or scoring justification, and the applicable TRL calculator. The report must fully characterize technology maturity risks of the applicable CTE(s). (4) Technology Maturation Plan CTMP). The TRA must produce a TMP for all technologies assessed within regulatory thresholds per cognizant acquisition policy. The TMP must include the approach, activities, resources, and timelines associated with maturing technologies to the target level. (5) Evaluation Memo. This memorandum must document an assessment of the completeness and analytical quality of the TRA process to ensure programs and projects are meeting levels of confidence associated with appropriate overall system performance. The assessment must review the TRA against best practices and applicable DOE guidance (See Appendix 1: TRA Metrics); however, it will not validate the technical findings or results of the TRA scoring.

7. RESPONSIBILITIES. a. Deputy/ Associate Administrators. (1) Assign the TRA Team Lead; (2) Ensure resources are available for the execution of the TRA; and 4 NAP-29 12-22-16 (3) Review and approve the TRA plan, schedule, TRA report, and subsequent TMP as required. Commensurate with scope and size of the effort, the Deputy/ Associate Administrator may delegate this responsibility. b. Director, Office of Cost Estimating and Program Evaluation. Reviews the TRA plan, the assessment tool, the TRA final report, and the TMP, and documents the result in the evaluation memo provided to the program or project office and Administrator. c. TRA Team Lead. ( 1) Serves as the TRA team primary point of contact; (2) Reviews TRA team members' qualifications to ensure that the team has the appropriate expertise and abilities to execute the TRA; and (3) Develops the TRA plan, TRA schedule, TRA calculator, and TRA final report and associated TMP. d. TRA Team Members. (1) Serve as SMEs in technical areas relevant to the technology under review. (2) Perform independently of the offices reviewing the work. (3) Assess technologies, determine Readiness Levels (and other relevant metrics), and document their determination. (4) Encourage participation of SMEs from the field offices to ensure relevant expertise is included in the assessment. 8. REFERENCES. a. DOE Secretarial Memorandum, Project Management Policies and Principles, 6-8-15. b. DOE 0 413.3B Admin Change 1, Program and Project Management for the Acquisition of Capital Assets, 10-22-15. c. DOE G 413.3-4A Admin Change 1, Technology Readiness Assessment Guide, 10-22-15. d. NNSA Defense Programs Technology Readiness Assessment (TRA) Implementation Guide, Revision 1, June 2015. NAP-29 12-22-16 9. DEFINITIONS. 5 a. Artifact. Any publication or document that defines the methods, processes, tools, data, or product deliverables because of, or in support of, a systems engineering life cycle, captured in electronic or hard copy format. Examples of this might be design documents, data models, workflow diagrams, test matrices and plans, setup scripts, etc. b. Critical Technology Element CCTE). A technology element is critical if the system being acquired depends on the technology element to meet operational requirements (with acceptable development, cost and schedule, and with acceptable production and operations costs) and if the technology element or its application is either new or unique. c. Integration Readiness Levels (IRLs). The TRL scale does not account for the integration of a technology into a complete system. In a system consisting of multiple component technologies, these might be at different TRLs at the beginning of the acquisition, and may mature at different rates. Systems under development frequently fail at integration points due to these disparate component technology levels of maturity. To minimize this failure, IRLs are developed to characterize the interface maturity between developing component technologies in a system. d. Major Atomic Energy Defense Acquisition (MAEDA) Program. An atomic energy defense acquisition program of which the total project cost is more than $500 million or the total lifetime cost is more than $1 billion. The term major atomic energy defense acquisition program does not include a project covered by DOE 0 413.3B (or a successor order) for the acquisition of capital assets for atomic energy defense activities.

e. Manufacturability Readiness Levels CMRLs). MRLs are a measurement scale designed to provide a common metric and vocabulary for assessing manufacturing maturity and risk of a particular technology, manufacturing process, weapon system, subsystem, or element of a legacy program at key milestones throughout the acquisition life cycle. There are 10 basic MRLs designed to be roughly congruent with comparable levels ofTRLs for ease of use and understanding. f. Nuclear Readiness Level CNRLs). In order to address the specific issues related to nuclear components and materials, the United Kingdom's (UK) National Nuclear Laboratory (NNL) developed a nuclear technology maturity scale (from 1-10), similar to the original NASA TRL scale (from 1-9). g. Programmatic Readiness Levels CPRLs). This index addresses program management concerns, such as documentation of programmatic milestones seen as vital to successful technology product development. This scale is also a nine level scale to match the TRL scale. 6 h. 1. J. k. 1. NAP-29 12-22-16 Project Management Executive. The individual designated to integrate and unify the management system for a program portfolio of projects and implement prescribed policies and practices. Formerly known as the Acquisition Executive. Subject Matter Expert (SME). Individual with sufficient technical knowledge of the critical technologies under review, commensurate with the depth and rigor of the TRA. These individuals may be either federal or contractor personnel. System Readiness Levels CSRLs). Measures the level of maturity applied at the system-level. SRLs combine TRLs with IRLs. SRLs are useful when going from the individual technology to a system context that may involve multiple technologies, as is the case with most technologies in the operational environment. Technology. A manner of accomplishing a task especially when using technical processes, methods, or knowledge. This may include hardware, software, materials, etc. Technology Development. Activities in connection with corporate or governmental innovation. m. Technology Maturation Plan CTMP). A TMP details the steps necessary for developing less mature technologies to the point where they are ready for project insertion. n. Technology Readiness Assessment (TRA). A systematic, metric-based assessment of how far technology development has progressed. It is not a pass/fail exercise, and does not intend to provide a value judgment of the technology developers or the technology development program. o. Technology Readiness Level (TRL). A metric used for describing technology maturity. It is a measure used by many U.S. government agencies to assess maturity of evolving technologies prior to incorporating that technology into a system or subsystem. 10. CONTACT. Director, Office of Cost Estimating and Program Evaluation, 202-586-6910 BY ORDER OF THE ADMINISTRATOR: j~~;ot~ Administrator Appendix 1: TRA Metrics Appendix 2: NAP Process Workflow NAP-29 12-22-16 APPENDIX 1: TRA METRICS 1. Best Practices for TRA Process - General Principles a. A TRA Lead designee. Appendix 1 APl-1 b. The TRA schedule allows adequate lead-time to socialize the TRA prior to the supported decision and develop Technology Maturation plans if required. c. Subject Matter Experts (SMEs) represent critical technologies under analysis. 2. Best Practices for TRA Process - Determining Critical Technologies

The TRA shows evidence of disciplined systems engineering and systems analysis methods for determining critical technologies (e.g., requirements analysis, functional decomposition, design synthesis, work breakdown structures, mission/performance analysis, and enterprise architecture analysis). 3. Best Practices for TRA Process -Assessing Technology Maturity a. The type of readiness assessment and Readiness Level (RL) calculator scale (e.g., TRL, MRL, SRL, IRL, PRL, or Hybrid) is clearly identified for each Critical Technology Element (CTE) (if different types ofRLs), traceable to the applicable acquisition framework, and remains stable throughout the analysis. b. The TRA clearly shows the TRL score for each CTE with substantiating analysis and data. c. The TRA analysis framework, methodology, and scoring is supported with details commensurate with the assessed levels according to the Calculator. d. TRA assessment shortfalls in the characterization of technology risk are identified. 4. Best Practices for TRA Process-TRA Reporting, Governance a. The TRA report is completed and accepted by the Deputy/Associate Administrator or delegate prior to the acquisition decision being supported. b. The TRA report contains sufficient supporting information to show how the analysis was conducted and that the results are clear and sufficiently substantiated. c. Technology shortfalls and risks identified by CTE. 5. Best Practices for TRA Process- Technology Maturation Planning The Technology Maturation Plan (TMP) contains the following: Appendix 1 APl-2 a. b. c. NAP-29 12-22-16 History of past TRL assessments and current TRL levels (shortfalls noted). Plan to mature CTEs that do not meet required levels. A comprehensive characterization of the CTE maturation risk. Ill .... 0 ......... C1J ii; > rt) .... ':l ·o tl c. 0 ·c: C1J Ill·­c Ill E <( "C <( Review TRI\ Plan/Sche..lule Review TRA Plan/Schedule ~-~-- ·-;.-_ •. 'o .. · -·_·: · :~:;>~ ·. llevPl op .·· .. l TRil -~ •· Plan/Sc hl'dule 1 . Approve TRA Plan/Schedule (", .· ~ Ensure SM E Avallablllty lctenti fy SM rs Review TRI\ Calculator Oevelop TRI\ Calculator ( l\~ess T ~hnolocvl Review TRA Final Report & TMP• Approve TRA Final Report & TMP• •As Required ;I> "'O "'O M :z ~ -;x N _, ~ "'O :::0 0 ('j M (/) (/) ~ 0 ~ t"' 0 I ~ CEPE Cost Est imating and Program Evalua tion TM P: Technology maturat ion Plan TRA Technology Readiness Assessment z t;-1 ;J> N >-rj NI I N - '° 0\ ~ ;J> g '"'Cl 0.. N - · I >< N