BOP-06.03, Independent Cost Estimates Procedure

CONTROLLED DOCUMENT OFFICE OF PRIMARY INTEREST (OPI): AVAILABLE ONLINE AT: Office of Enterprise Project Management http://hq.na.gov printed copies are uncontrolled BUSINESS OPERATING PROCEDURE Approved: 2-27-14 Independent Cost Estimates Procedure NATIONAL NUCLEAR SECURITY ADMINISTRATION Office of Acquisition and Project Management BOP-06.03 THIS PAGE INTENTIONALLY LEFT BLANK BOP-06.03 1 2-27-14 INDEPENDENT COST ESTIMATES PROCEDURE 1. PURPOSE: This Business Operating Procedure (BOP) reflects the requirements, responsibilities, and expectations relating to independent cost estimates (ICEs) on projects being executed by the National Nuclear Security Administration (NNSA). 2. CANCELLATION: BOP-50.005, National Nuclear Security Administration Establishment of an Independent Cost Estimate Policy, dated February 5, 2009. 3. APPLICABILITY: a. NNSA Applicability: (1) This procedure pertains to all capital asset projects as defined in Department of Energy (DOE) Order (O) 413.3B, Program and Project Management for the Acquisition of Capital Assets, constructed for NNSA or managed by NNSA personnel on behalf of other government agencies with an estimated Total Project Cost (TPC) greater than or equal to $20M. These projects include: Major Items of Equipment, Line Item (Capital Asset) Projects, Operation Expense Funded Projects, Lease/Alternate Finance Projects, and Work for Others projects. (2) The requirements in this procedure will be applied in conjunction with, and will not supersede, any minimum requirements established by DOE O 413.3B. Execution of project activities will follow DOE O 413.3B. (3) A Type II ICE (defined in Attachment 4) equates to an Independent Cost Review (ICR); therefore, this BOP applies to ICEs and ICRs. The applicability and specific performance of either an ICE or an ICR for each project/project phase will be determined by the Director, Office of Enterprise Project Management (NA-APM-20) in consultation with the designated Acquisition Executive (AE). b. Contractors: Does not apply to contractors. c. Exemptions: The Director of NA-APM-20 may grant waivers to this procedure. This responsibility is non-delegable. There are no other forums for redress. d. Exclusions: (1) General Plant Projects and Capital Equipment Projects are excluded, in alignment with DOE O 413.3B. (2) Financial Assistance Awards (grants and cooperative agreements) are excluded, which are covered under 10 Code of Federal Regulations (CFR) 600. 2 BOP-06.03 2-27-14 (3) Nuclear weapon research and development activities, which are managed in accordance with guidance promulgated by the Office of Defense Programs such as the Defense Programs Executive Policy, dated September 11, 2013 and the DP Program Execution Guide, dated November 20, 2013, are also excluded. e. Equivalency: In accordance with the responsibilities and authorities assigned by Executive Order 12344, codified at 50 United States Code (USC) sections 2406 and 2511 and to ensure consistency through the joint Navy/DOE Naval Nuclear Propulsion Program, the Deputy Administrator for Naval Reactors (Director) will implement and oversee requirements and practices pertaining to this Directive for activities under the Director's cognizance, as deemed appropriate.

4. BACKGROUND: An ICE is a cost estimate, prepared by an organization independent of the project sponsor, using the same detailed technical and procurement information to develop the project estimate. An ICR is an independent evaluation of a project’s cost estimate that examines the reasonableness of the estimate quality, assumptions, and risks, also prepared by an organization independent of the project sponsor. A Type II ICE equates to an ICR. When conducted in accordance with NNSA policies, an ICE or an ICR may be used to validate the project estimate and, when applicable, validate a project cost performance baseline at Critical Decision-2 (CD-2). 5. REQUIREMENTS: a. Attachment 1 and Attachment 2 enumerate different types/classes of ICEs that may be performed at various stages of line item projects within NNSA and their respective accuracy and uncertainty ranges. The estimate types/classes to be utilized for an ICE should be tailored, depending on the stage of the project and the level of documentation available; Attachment 4 contains associated cost estimating definitions. With respect to the validation of a project cost performance baseline at CD-2: (1) Prior to CD-2, the design for all NNSA projects that are Hazard Category 1, 2 or 3 nuclear facilities must be at least 90 percent complete. (2) Prior to CD-2, an ICE or an ICR will be performed by NA-APM-20 for each NNSA project with an estimated TPC between $20M and $100M to validate the project cost performance baseline; for projects with an estimated TPC between $20M and $100M, the Director of NA-APM-20, in consult with the AE, will determine if an ICE or a separate ICR is required, or if an ICR conducted as a part of an Independent Project Review (IPR) is appropriate to validate the project cost performance baseline. (3) Prior to CD-2, an ICE will be performed by DOE Acquisition and Project Management (DOE APM) for each NNSA project with an estimated TPC BOP-06.03 3 2-27-14 greater than or equal to $100M. This ICE will validate the project cost performance baseline and Life-cycle cost estimate (LCCE). b. For NNSA projects with an estimated TPC greater than or equal to $100M, a separate CD-3 ICE will be performed by the DOE/APM. c. ICEs and/or ICRs shall be identified in the planning schedule for the project. d. All successive project cost estimates, including ICEs, shall be reconciled and kept on file with previous estimates until the project is completed, thereby ensuring traceability from project start to completion. The estimate documentation file shall also include the basis for each estimate, the escalation rates used, show how the estimate was performed, and contain a risk and contingency analysis. e. All successive cost estimates shall be performed in constant year dollars (generally current fiscal year dollars), and then escalated into year-of-expenditure (generally fiscal year) dollars. DOE escalation rate indices, if available, or other customized escalation rate indices (with documented rationale and substantiating data), should be used in NNSA project cost estimates. f. In planning projects, it is acceptable to include cost estimate allowances for activities for which there is little or no design basis, especially in the earliest stages. These allowances are not considered contingency costs. Allowances should be included in early cost estimates to cover anticipated costs associated with known technical requirements or activities. Any allowances included in cost estimates should include a basis for these costs within the supporting Basis of Estimate (BOE) document.

g. Cost contingency should not to be used in any NNSA project as a substitute for well defined scope and project requirements. The point estimate at CD-2 must include sufficient contingency such that the confidence level in the estimate is no lower than 85 percent. h. ICEs shall be developed as specified in DOE O 413.3B, or in response to requests/recommendations by any of the following: the Administrator, Principal Deputy, Deputy and Associate Administrators, AEs, Program Managers, the Director of Enterprise Project Management, or Federal Project Directors. 6. RESPONSIBILITIES a. Program Office: Budgeting for conducting cost estimates and reviews is the responsibility of the cognizant Program Office. For line item projects, cost estimate and review funds shall be included in the Congressional Budget Request for each project. (1) Ensure the request for an ICE for project(s) within their program is submitted to support project progression and AE decisions. 4 BOP-06.03 2-27-14 (2) Communicate to the ICE Team the mission need and ensure the ICE is conformed on the respective mission need and acquisition strategy. (3) Assure that the information that impacts the program budget submissions and provided for AE decision making aligns with DOE policies involving cost estimating (4) Collaborate with NA-APM and the FPD to define the scope, bounds and objectives of the cost estimate review to be conducted. (5) Provide an individual to observe in the ICE/ICR Team. (6) Review ICE report for factual accuracy. (7) Provide NA-APM feedback on the ICE review for the purpose of continuous improvement. b. Office of Enterprise Project Management (NA-APM-20): (1) Ensure NNSA ICEs and/or reviews are performed in accordance with accepted cost estimating industry standards and DOE orders. (2) Serve as a focal point for all cost estimating policy and standardization within NNSA. (3) Improve cost estimating techniques and practices within NNSA. (4) Define policy and establish procedures for the implementation of independent cost estimating and project cost analysis in NNSA. (5) Help user organizations to develop customized escalation indices, as requested. (6) Ensure the 12 steps of the Government Accountability Office (GAO) High-Quality Cost Estimating Process are addressed in ICEs and reviews of project estimates including LCCE. See Attachment 5. (7) Ensure ICE Teams are comprised of individuals with appropriate industry/DOE experience with credentials (e.g. certifications, PEE, CCE, PMP, etc.) and SMEs knowledgeable in the areas addressed by the projects. (8) Organize, direct, and conduct ICEs, analyses, and reviews of project estimates, and coordinate membership for independent cost estimating task groups. (9) Reconcile ICEs with project estimates, when appropriate (e.g. ICE conducted by NA-APM). If major variances between an ICE and a project BOP-06.03 5 2-27-14 estimate exist and remain after reconciliation, approve in consultation with the cognizant program office, the disposition regarding such variances which might entail changes in or to the project cost estimate. (10) In consult with the AE, determine if an ICE or a separate ICR is required prior to CD-2 for projects with an estimated TPC between $20M and $100M to validate the project cost performance baseline. (11) Ensure that the design of each project is sufficiently mature prior to validating the project performance cost baseline at CD-2.

(12) Ensure that the projects, as they progress through the Critical Decisions, as described in DOE O 413.3B, reflect changes in the technical and program assumptions. c. Federal Project Director: (1) Develop and maintain cost estimates throughout the life of each NNSA project. (2) Maintain a record of actual project costs. (3) With the exception of “design-build” projects, ensure that sufficient design completeness has been accomplished on which a credible project performance cost baseline CD-2 estimate can be performed. (4) In consult with the cognizant Program Office and NA-APM-20, determine if an ICE or a separate ICR is required prior to CD-2 for projects with an estimated TPC between $20M and $100M to validate the project cost performance baseline. (5) Ensure funding is available and sufficient process schedule allowances are made for project ICEs and/or ICRs. (6) Incorporate this procedure’s requirements in the applicable project documentation such as the Preliminary Project Execution Plan (PPEP) or Project Execution Plan to ensure all stakeholders are aware. (7) Ensure that the project, as it progresses through the Critical Decisions, as described in DOE O 413.3B, reflect changes in the technical and program assumptions. 7. REFERENCE: a. DOE Order 413.3B, Program and Project Management for the Acquisition of Capital Assets, November 29, 2010 b. DOE Guide 413.3-21, Cost Estimating Guide, May 9, 2011 , . 6 BOP-06.03 2-27-14 c. GA0-09-3SP, GAO Cost Estimating and Assessment Guide, March 2009 d. 10 CFR 60_0, Financial Assistance Rules e. Defense Programs Executive Policy, dated September 11, 2013 f, DP Program Execution Guide, dated November 20, 2013 g. -Executive Order 12344, Naval Nuclear Propulsion Program, February 1, 1982 h. 50 United States Code Section 2406, Deputy Administrator for Naval Reactors 1. 50 United States Code Section 2511, Naval Nuclear Propulsion Program 8. DEFINITIONS: See Attachment 4. 9. CONTACT: Office of Enterprise Project Management (NA-APM-20), Victoria Premaza, (202) 586-7358. BY ORDER OF THE ADMINISTRATOR: Attachment( s): Ro ert B. Rain s Associate Administrator for Acquisition and Project Management 1. Accuracy/Uncertainty Ranges for a Process Industry Estimate 2. Types ofindependent Cost Estimates and Timing 3. Estimate Input Checklist and Maturity Matrix 4. Cost Estimating Definitions 5. Govenunent Accountability Office 12. steps of a High-Quality Cost Estimating Process BOP-06.03 Attachment 1 2-27-14 AT1-1 ATTACHMENT 1: ACCURACY/UNCERTAINTY RANGES FOR A PROCESS INDUSTRY ESTIMATE Example of the Variability in Accuracy/Uncertainty Ranges for a Process Industry Estimate (Source: AACE 18R-97) THIS PAGE INTENTIONALLY LEFT BLANK BOP-06.03 Attachment 2 2-27-14 AT2-1 ATTACHMENT 2: TYPES OF INDEPENDENT COST ESTIMATES AND TIMING Timing: Review purpose Direction Estimate Class ICE Type Prior to CD-0 Project cost magnitude range A project cost magnitude range should be established based on project alternatives. This establishes the Acquisition Authority Level for CD-0. Normally, depending on techniques used, there will be little, if any, distinction between components or categories within the cost estimate (e.g., direct costs, indirect costs, contingency, or escalation; labor, materials, equipment, etc.; types of work). The cost estimate for the Conceptual Design Review (CDR) phase of the project should be developed to a more definitive level. Class 5 Type I

At CD-0 Conceptual Phase estimate Cost estimates prepared to support achieving CD-1 will range from Class 5, Order of Magnitude to Class 3, Preliminary cost estimates, using several cost estimating techniques. For alternatives explored, varying levels of available information should be expected. Ranges should be a little more refined than those prior to CD-0, but still established based on the range of project alternatives. The cost estimate for the Preliminary Design phase of the project should be developed to a more definitive level. Class 4 Type II, III, IV, V Prior to CD-1 Alternative analysis ICE (LCCA) The cost range developed at this point in the project planning process will represent all viable alternatives considered to achieve the required performance capability. These estimates also should include costs for exploring alternative concepts and the development of solutions and alternatives during the project definition phase. Life-cycle cost estimates (LCCE) that are developed early in a project’s life may not be derived from detailed engineering, but they must be sufficiently developed to support budget requests for the remainder of the project definition phase. They should also include all anticipated resources, using appropriate estimating techniques that are necessary to acquire or meet the identified capability. Class 4 Type II, III, IV Attachment 2 BOP-06.03 AT2-2 2-27-14 Timing: Review purpose Direction Estimate Class ICE Type Prior to CD-1 During the project definition phase, at the conclusion of the concept exploration process, the alternative selected as the best solution to a mission need is presented for approval. The solution presented as a subset of a conceptual design report must include the TPC range, a schedule range with key milestones and events, and annual funding profiles. The TPC range presented must be a risk-adjusted cost estimate that defines all required resources necessary to successfully execute the planned work. Class 3 Type II, III, IV, V After CD-1 Preliminary Design Phase Estimate This is the estimated cost associated with the preliminary design and should contain activities for final design phase. This estimate will be the basis for the Federal CD decision process. Class 2 Type III, IV, V Prior to CD-2 Performance ICE Establishes the project’s performance baseline. The cost estimate shall be risk adjusted and capture the TPC to acquire the asset. This estimate should be organized using a work breakdown structure that supports earned value management reporting requirements and easily facilitates evaluation of all estimated costs. Class 1 Type III, IV, V Reconciliation between ICE and project cost estimate Reconciliation between the ICE and project estimate for the purpose of validating the BOE. This estimate will be the basis for the Federal CD decision process. Class 1 Type III, IV, V Prior to CD-3 Construction or Execution Readiness If the TPC is greater than or equal to $100M, DOE’s Office of Engineering and Construction Management will develop an ICE, if warranted by risk and performance indicators or as designated by the AE. (ICE validates the project baseline). If TPC is less than $100M, an ICR may be developed as part of an IPR by APM (validate cost performance baseline). Class 1 Type III, IV, V BOP-06.03 Attachment 3 2-27-14 AT3-1 ATTACHMENT 3: ESTIMATE INPUT CHECKLIST AND MATURITY MATRIX

Estimate Input Checklist and Maturity Matrix (Primary Classification Determinate) (Source: AACE 18R-97) Please note that per AACE 18R-97, the following refers to the notations used in Figure 2. None (blank): development of the deliverable has not begun; Started (S): work on the deliverable has begun. Development is typically limited to sketches, rough outlines, or similar levels of early completion; Preliminary (P): work on the deliverable is advanced. Interim, cross-functional reviews have usually been conducted. Development may be near completion except for final reviews and approvals; Complete (C): the deliverable has been reviewed and approved as appropriate. THIS PAGE INTENTIONALLY LEFT BLANK BOP-06.03 Attachment 4 2-27-14 AT4-1 ATTACHMENT 4: CE Definitions Acquisition Executive (AE) - The individual designated by the Secretary of Energy to integrate and unify the management system for a program portfolio of projects and implement prescribed policies and practices. Basis of Estimate (BOE) - Documentation that describes how an estimate, schedule, or other plan component was developed, and defines the information used in support of development. The basis of estimate documents the estimate assumption, exclusions, and criteria used in producing the estimate and include; the cost estimate purpose and class; work breakdown structure (WBS), including deliverables and scope of work; code of accounts; project/program requirements and milestones, including constraints, special conditions, regulatory drivers, applicable DOE Orders, and industry standards; description of assumptions and exclusions; backup data, including quantity takeoffs, calculations, commercial databases; historical data, cost estimating relationships (CERs), quotes, and other general sources of information; basis of direct costs (e.g., industry standards and historical information); basis of indirect costs (e.g., rates from a corporate perspective); basis of escalation; and basis of contingency, which may include or reference a risk analysis or risk management plan. Capital Assets - Capital Assets. Capital assets are land, structures, equipment and intellectual property, which are used by the Federal Government and have an estimated useful life of two years or more. Capital assets exclude items acquired for resale in the ordinary course of operations or held for the purpose of physical consumption such as operating materials and supplies. Capital assets may be acquired in different ways: through purchase, construction, or manufacture; through a lease-purchase or other capital lease, regardless of whether title has passed to the Federal Government; or through exchange. Capital assets include the environmental remediation of land to make it useful, leasehold improvements and land rights; assets owned by the Federal Government but located in a foreign country or held by others (such as federal contractors, state and local governments, or colleges and universities); and assets whose ownership is shared by the Federal Government with other entities. Contingency - The portion of a project budget that is available for uncertainty within the project scope, but outside the scope of the contract. Contingency is budget that is not placed on contract. Contingency is part of the TPC and government’s baseline, but is not part of the contractor’s baseline. It is an amount derived from a structured evaluation of identified risks, to cover a likely future event or condition, arising from presently known or unknown causes, within a defined project scope. Contingency is held and controlled by the government.

Critical Decision (CD) - A formal determination made by the Secretarial AE/AE at a specific point in a project’s life cycle that allows the project to proceed to the next phase or Critical Decision. Critical Decision requirements are specified in DOE O 413B. Class 5 Cost Estimates - Also known as rough order of magnitude (ROM) or top-down cost estimates, these are typically performed in the early stages of a project’s life. These cost estimates are based on readily available information and may indicate a low level of confidence or accuracy in the estimate. Techniques used to develop these estimates include stochastic, most parametric, and professional judgment (parametric, specific analogy, expert opinion, trend analysis). Attachment 4 BOP-06.03 AT4-2 2-27-14 Class 4 Cost Estimates - Include a combination of Class 5 and Class 3 cost estimates. The techniques used to develop estimates in this class vary, with more parametric (parametric, specific analogy, expert opinion, trend analysis) estimates being the norm. Class 3 Cost Estimates - Preliminary, or budgetary, cost estimates contain diverse levels of available and supporting information, use various techniques, and portray a moderate level of confidence. Various techniques, including combinations (detailed, unit-cost, or activity-based; parametric; specific analogy; expert opinion; trend analysis) are used to develop these estimates. Class 2 Cost Estimates - Intermediate cost estimates include a combination of Class 3 and Class 1 cost estimates. The techniques employed here vary, with more definitive (detailed, unit-cost, or activity-based; expert opinion; learning curve) approaches being utilized. Class 1 Cost Estimates - Definitive cost estimates, also known as detailed, detailed unit-cost, or activity-based cost estimates, are those with the most abundantly available support information using a definitive technique for development and representing a greater level of confidence. These techniques include deterministic and most definitive (detailed, unit-cost, or activity-based; expert opinion; learning curve). Design-Build - A project delivery method whereby a single contract is awarded for both design and construction. Design-Build is normally used most successfully with projects that have well- defined requirements with limited complexity and risks. Since the requirements are well-defined early in the process and much of the cost and schedule information and key design criteria are known, CD-1, CD-2 and/or even CD-3 may be accomplished simultaneously. Escalation - The provision in actual or estimated costs for an increase in the cost of equipment, material, labor, etc, due to continuing price level changes over time. Inflation may be a component of escalation, but non-monetary policy influences, such as supply-and-demand, are often components. Escalation Rate Index - The specific predicted annual increase over time of a project associated with changing economic conditions. DOE may provide escalation rate guidance. Independent Cost Estimate (ICE) - A cost estimate, prepared by an organization independent of the project proponent, using the same detailed technical and procurement information to develop the project estimate. It can be used to validate the project estimate to determine whether it is accurate and reasonable.

Independent Cost Review (ICR) - An independent evaluation of a project’s cost estimate that examines its quality and accuracy, with emphasis on specific cost and technical risks. It involves the analysis of the existing estimate’s approach and assumptions. A Type II ICE equates to an ICR. Life-cycle cost - The overall estimated cost for a particular program alternative over the time period corresponding to the life of the program, including direct and indirect initial costs plus any periodic or continuing cost of operation and maintenance. The sum total of the direct, indirect, recurring, nonrecurring, and other costs incurred or estimated to be incurred in the design, development, production, operation, maintenance, support, and final disposition of a BOP-06.03 Attachment 4 2-27-14 AT4-3 major system over its anticipated useful life span. Where system or project planning anticipates the use of existing sites or facilities, restoration, and refurbishment, costs should be included. Life-cycle cost analysis (LCCA) - An assessment of the direct, indirect, recurring, nonrecurring, and other related costs incurred or estimated to be incurred in the design, development, production, operation, maintenance, support, and final disposition of a major system over its anticipated useful life span. LCCA considers all costs (capital, operating, and decommissioning expenses for the duration of a project) for various alternative approaches, including inflation and discount rates. Line-item project - Projects that are specifically reviewed and approved by Congress. A project with total cost greater than $20M. Major Item of Equipment - Capital equipment not related to a specific construction project. In most cases, capital equipment is installed with little or no installation or construction cost. However, in cases where the equipment requires provision of foundations, utilities, structural modifications, and/or additions to a building, the project can be defined as MIE. The associated construction activities must not constitute more than 20 percent of the costs of the equipment or exceed the GPP threshold established by Congress. Project - A group of related activities that has a defined starting and end point and undertaken to create a unique product or service in support of a program. Project Traceability - A life cycle cost and schedule estimate file will be maintained on all projects. The file will contain all cost and schedule estimates from the beginning to the end of project construction. Significant variances between subsequent estimates must be explained and kept on file so that traceability can be maintained throughout the project’s life. Changes in scope, escalation assumptions, estimating methods, contingency, and schedule shall be explained, recorded, and tracked from one estimate to the next, and kept on file. Type I ICE Documentation Review - This type of review is not normally accomplished as an ICR or ICE, nor does it fulfill the requirements as specified in DOE O 413.3B, since it only consists of an assessment of the documentation available to support the estimate. It is merely an inventory of existing documents to determine that the required support documentation exists and to identify any missing data. This type of review can be beneficial for a project team facing an upcoming External Independent Review or ICE, to ensure readiness to proceed with those activities.

Type II ICE Reasonableness Review - This equates to an ICR. For this review the ICR team reviews all available project documentation, receives briefings from the project team, holds discussions with the project team, completes sufficient analysis to assess the reasonableness of the project assumptions supporting the cost and schedule estimates, ascertains the validity of those assumptions, assesses the rationale for the methodology used, and checks the completeness of the estimate, including appropriate allowances for risks and uncertainties. The result is a report that details the findings and recommendations. Type III ICE Parametric Estimating Approach - This approach, in addition to incorporating all of the activities needed for a Reasonableness Review, uses parametric techniques, factors, etc., to analyze project costs and schedules, and is usually accomplished at a summary WBS Attachment 4 BOP-06.03 AT4-4 2-27-14 level. The parametric techniques (including CERs and factors) should be based on accepted historical cost/schedule analyses. At a minimum, these tools should be based on historic estimates from which models have been derived, and, where possible, from actual completed projects. An estimate with a minimum of 75 percent of the TPC based on parametric techniques is classified as a parametric estimate. Type IV ICE Sampling Approach - This review also begins with the activities needed for a Reasonableness Review, but it also requires the ICE team to identify the key cost drivers. A “cost driver” is a major estimate element whose sensitivity significantly impacts TPC. Detailed, independent estimates should be developed for these cost drivers. Such estimates should include vendor quotes for major equipment, and detailed estimates of other materials, labor, and subcontracts. For the balance of the project costs, the project team’s estimate may be used (if deemed reasonable), or, if appropriate, parametric techniques may be used for certain portions of the project costs. An estimate which provides a detailed cost for all cost drivers is classified as a Sampling Estimate. Type V ICE Bottom-up Estimating Approach - This is the most detailed and extensive ICE effort. It begins with the activities needed for a Reasonableness Review. In addition, this approach requires a detailed bottom-up independent estimate for both cost and schedule. This will require quantity takeoffs/development, vendor quotations, productivity analysis, use of historical information, and any other means available to do a thorough and complete estimate of at least 75 percent of the project’s cost. It may not be possible to do a completely independent estimate on some portions of the project estimate, and for those portions – which should not exceed 25 percent of the total estimate – the project estimate may be used if it has passed the test of reasonableness. In all cases, the total cost (total estimated cost and TPC) should be developed. BOP-06.03 Attachment 5 2-27-14 AT5-1 ATTACHMENT 5: GOVERNMENT ACCOUNTABILITY OFFICE 12 STEPS GAO’s 12 Steps of a High Quality Cost Estimating Process The GAO Cost Estimating Process consists of 12 steps. Each step builds upon each other to develop and comprehensive and complete cost estimate. Each of the 12 steps is important for ensuring that high-quality cost estimates are developed and delivered in time to support important decisions. Step 1: Define estimate’s purpose

 Determine estimate’s purpose, required level of detail, and overall scope;  Determine who will receive the estimate. Step 2: Develop Estimating Plan  Determine the cost estimating team and develop its master schedule;  Determine who will do the ICE;  Outline the cost estimating approach;  Develop the estimate timeline. Step 3: Define Program Characteristics  In a technical baseline description document, identify the program’s purpose and its system and performance characteristics and all system configurations;  Any technology implications;  Its program acquisition schedule and acquisition strategy;  Its relationship to other existing systems, including predecessor or similar legacy systems;  Support (manpower, training, etc.) and security needs and risk items;  System quantities for development, test, and production;  Deployment and maintenance plans. Step 4: Determine Estimating Structure  Define a WBS and describe each element in a WBS dictionary (a major automated information system may have only a cost element structure);  Choose the best estimating method for each WBS element; Attachment 5 BOP-06.03 AT5-2 2-27-14  Identify potential cross-checks for likely cost and schedule drivers;  Develop a cost estimating checklist. Step 5: Identify ground rules and assumptions  Clearly define what the estimate includes and excludes;  Identify global and program-specific assumptions, such as the estimate’s base year, including time-phasing and life cycle;  Identify program schedule information by phase and program acquisition strategy;  Identify any schedule or budget constraints, inflation assumptions, and travel costs;  Specify equipment the government is to furnish as well as the use of existing facilities or new modification or development;  Identify prime contractor and major subcontractors;  Determine technology refresh cycles, technology assumptions, and new technology to be developed;  Define commonality with legacy systems and assumed heritage savings;  Describe effects of new ways of doing business. Step 6: Obtain Data  Create a data collection plan with emphasis on collecting current and relevant technical, programmatic, cost, and risk data;  Investigate possible data sources;  Collect data and normalize them for cost accounting, inflation, learning, and quantity adjustments;  Analyze the data for cost drivers, trends, and outliers and compare results against rules of thumb and standard factors derived from historical data;  Interview data sources and document all pertinent information, including an assessment of data reliability and accuracy;  Store data for future estimates. Step 7: Develop Point Estimate and Compare it to an ICE  Develop the cost model, estimating each WBS element, using the best methodology from the data collected, and including all estimating assumptions; BOP-06.03 Attachment 5 2-27-14 AT5-3  Express costs in constant year dollars;  Time-phase the results by spreading costs in the years they are expected to occur, based on the program schedule;  Sum the WBS elements to develop the overall point estimate;  Validate the estimate by looking for errors like double counting and omitted costs;  Compare estimate against the ICE and examine where and why there are differences;  Perform cross-checks on cost drivers to see if results are similar;  Update the model as more data become available or as changes occur and compare results against previous estimates.

Step 8: Conduct Sensitivity Analysis  Test the sensitivity of cost elements to changes in estimating input values and key assumptions;  Identify effects on the overall estimate of changing the program schedule or quantities;  Determine which assumptions are key cost drivers and which cost elements are affected most by changes. Step 9: Conduct Risk and Uncertainty Analysis  Determine and discuss with technical experts the level of cost, schedule, and technical risk associated with each WBS element;  Analyze each risk for its severity and probability;  Develop minimum, most likely, and maximum ranges for each risk element;  Determine type of risk distributions and reason for their use;  Ensure that risks are correlated;  Use an acceptable statistical analysis method (e.g., Monte Carlo simulation) to develop a confidence interval around the point estimate;  Identify the confidence level of the point estimate;  Identify the amount of contingency funding and add this to the point estimate to determine the risk-adjusted cost estimate;  Recommend that the project or program office develop a risk management plan to track and mitigate risks. Attachment 5 BOP-06.03 AT5-4 2-27-14 Step 10: Document the Estimate  Document all steps used to develop the estimate so that a cost analyst unfamiliar with the program can recreate it quickly and produce the same result;  Document the purpose of the estimate, the team that prepared it, and who approved the estimate and on what date;  Describe the program, its schedule, and the technical baseline used to create the estimate;  Present the program’s time-phased life-cycle cost;  Discuss all ground rules and assumptions;  Include auditable and traceable data sources for each cost element and document for all data sources how the data were normalized;  Describe in detail the estimating methodology and rationale used to derive each WBS element’s cost (prefer more detail over less);  Describe the results of the risk, uncertainty, and sensitivity;  analyses and whether any contingency funds were identified;  Document how the estimate compares to the funding profile;  Track how this estimate compares to any previous estimates. Step 11: Present Estimate to Management for Approval  Develop a briefing that presents the documented LCCE;  Include an explanation of the technical and programmatic baseline and any uncertainties;  Compare the estimate to an ICE and explain any differences;  Compare the estimate (LCCE) or ICE to the budget with enough detail to easily defend it by showing how it is accurate, complete, and high in quality;  Focus in a logical manner on the largest cost elements and cost drivers;  Make the content clear and complete so that those who are unfamiliar with it can easily comprehend the competence that underlies the estimate results;  Make backup slides available for more probing questions;  Act on and document feedback from management;  Request acceptance of the estimate. BOP-06.03 Attachment 5 2-27-14 AT5-5 Step 12: Update the Estimate to reflect Actual Costs and Changes  Update the estimate to reflect changes in technical or program assumptions or keep it current as the program passes through new phases or milestones;  Replace estimates with Earned Value Management (EVM) Estimate at Completion (EAC) and independent EAC from the integrated EVM system;  Report progress on meeting cost and schedule estimates;  Perform a post mortem and document lessons learned for elements whose actual costs or schedules differ from the estimate;  Document all changes to the program and how they affect the cost estimate.