AUDIT FACT SHEET PUBLIC LETTER BACKGROUND OBJECTIVES SCOPE & METHODOLOGY CONCLUSION EFFICIENCY MEASURES REPORT SAVINGS DUE TO ACCELERATION OF WORK STREAMLINING ENVIROMENT REVIEWS (NEPA) VALUE ANALYSIS INNOVATIVE CONTRACTING TOOLS ATTACHMENT 1 ATTACHMENT 2

CALIFORNIA DEPARTMENT OF TRANSPORTATION AUDIT OF EFFICIENCY MEASURES P3010-0641

Independent Office of Audits and Investigations

AUDIT FACT SHEET

EFFICIENCY MEASURES AUDIT

BACKGROUND
Senate Bill 1 (SB1), also known as the “Road Repair and Accountability Act of 2017, increases revenue for California’s transportation system by approximately $5.2 billion annually. SB1 requires that California Department of Transportation (Caltrans) implement efficiency measures with the goal of generating at least $100 million annually in savings to invest in maintenance and rehabilitation of the state highway system.

The Caltrans Independent Office of Audits and Investigations completed an audit on the efficiency measures for SB1. The purpose of the audit was to determine if Caltrans has internal controls in place to ensure that it meets the goal of generating at least $100 million per year to invest in maintenance and rehabilitation of the state highway system as required by SB1. The audit was limited to the evaluation of the process, policies, procedures, and methodology used in identifying and measuring savings for the following efficiencies:

  • Savings Due to Acceleration of Work
  • Streamlining Environmental Reviews – National Environmental Policy Act Assignment
  • Value Analysis
  • Innovative Contracting Tools – Construction Manager/General Contractor

Our 2018-19 Audit Plan includes an audit with similar objectives for efficiencies not previously reviewed and an audit to determine whether the savings reported by Caltrans for year 2017-18 is supported.

RECOGNITION

The audit determined that Caltrans programs were proactive in identifying areas of efficiencies and developing processes and procedures to comprehensively and effectively achieve the required SB1 efficiency measures.

KEY RECOMMENDATIONS 

Our audit also identified the following general recommendations to assist Caltrans in meeting SB1 efficiency requirement:

Report Date: July 23, 2018
Report: P3010-0641

CALIFORNIA DEPARTMENT OF TRANSPORTATION AUDIT OF EFFICIENCY MEASURES

Date: July 23, 2018
File: P3010-0641

To: LAURIE BERMAN
Director

STEVEN KECK
Chief Financial Officer

KARLA SUTLIFF
Deputy Director
Project Delivery

CARRIE POURVAHIDI
SB l Program Manager
Office of the Director

From: ERAINA ORTEGA
Inspector General
Independent Office Audits and Investigations

FINAL AUDIT REPORT ON EFFICIENCY MEASURES

Attached is the Independent Office of Audits and Investigations' final audit report on the Efficiency Measures. Your response has been included as part ofour final report. This report is intended for your information and for California Department ofTransportation (Caltrans) management.

Our 2018-19 Audit Plan includes an audit with similar objectives for efficiencies not previously reviewed and an audit to determine whether the $100 million savings reported by Caltrans for year 2017-18 is supported.

This report will be posted on the Independent Office of Audits and Investigations' website, and the Inspector General is required to report at least annually to the Governor, the Legislature, and the California Transportation Commission with a summary of audit findings and recommendations.

We thank you and your staff for their assistance provided during this audit. If you have any questions or need additional information, please contact Kevin Yee, Acting Chief, Internal Audits, at (916) 323-7929, or me at (916) 323-7863.

Attachment

c: Susan Bransen, Executive Director, California Transportation Commission
Michael R. Tritz, Deputy Secretary, California State Transportation Agency
Ryan Chamberlain, Chief Deputy Director, Caltrans
Janice Benton, Chief, Division of Design
Philip J. Stolarski, Chief, Division of Environmental Analysis
Juanita Baier, SB 1 Program, Office of the Director
William E. Lewis, Assistant Director, A&I
Kevin Yee, Acting Chief, Internal Audits, A&I

BACKGROUND

In April 2017, Governor Brown signed into law Senate Bill 1 (SB 1 ), also known as the "Road Repair and Accountability Act of 2017." SB1 increases revenue for California' s transportation system by approximately $5 .2 billion annually, over the next ten years. SB 1 requires that California Department of Transportation (Caltrans) implement efficiency measures with the goal of generating at least $100 million annually in savings to invest in maintenance and rehabilitation of the state highway system. SB 1 also requires that Caltrans report the savings to the California Transportation Commission (Commission). Caltrans will consider efficiencies that result in cost avoidance or a reduction in support or capital costs.

Caltrans will prepare an annual report and present it to the Commission no later than September 30th of every year.

The Caltrans Independent Office of Audits and Investigations (A&I) completed an audit on the efficiency measures for SB 1. The purpose of this audit is to determine if Caltrans has internal controls in place to ensure that it meets the goal of generating at least $100 million per year to invest in maintenance and rehabilitation of the state highway system as required by SB 1. A&I plans to perform additional audit engagements at various stages of SB 1 implementation to ensure Caltrans is meeting the efficiency measures required by SB 1.

OBJECTIVES

The objectives of the audit were to determine whether:

  • Caltrans has adopted a definition for efficiency savings and cost avoidance.
  • There is an oversight body responsible for establishing a structure of authority, providing guidance, and monitoring the accuracy and reliability of the efficiency savings reported.
  • There are policies, procedures, and guidelines in place to assist programs, districts, and divisions in identifying, measuring, and reporting efficiency savings.
  • The methods of calculating efficiency savings are appropriate and supported.
  • Improvements are needed on the Efficiency Report presented to the Commission.

The audit covered the period of July 1, 2017, through the end of fieldwork. We conducted our audit from January 29, 2018, through June 28, 2018. Accordingly, our conclusions do not pertain to changes arising after June 28, 2018.

SCOPE AND METHODOLOGY

As the Efficiencies Report is not due to the Commission until September 2018, the audit was limited to the evaluation ofthe process, policies, procedures, and methodology used in identifying and measuring efficiency savings identified in the Interim Efficiencies Report presented to the Commission at its January 2018 meeting. The audit was performed in accordance with the International Standards for the Professional Practice of Internal Auditing.

CONCLUSION

The audit determined that Caltrans programs were proactive in identifying areas of efficiencies and developing processes and procedures to comprehensively and effectively achieve the required SB1 efficiency measures. Specifically,

  • Caltrans created the SB 1 Program Manager position to oversee and coordinate all Caltrans SB 1 activities, including the development of policies and implementation of the SB 1 program
  • The Financial Policy Board (Board) was designated as the oversight body responsible for approving efficiencies that will be included in the annual report to the Commission.
  • Caltrans subject matter experts and the SB 1 Program Manager work together to ensure that the information presented to the public on efficiencies is consistent, accurate, and reliable.
  • Caltrans prepared and presented an Interim Efficiencies Report to the Commission at the January 2018 Commission meeting to provide information on current strategies being employed for efficiencies.

Our audit also identified the following general recommendations to assist Caltrans in meeting SB 1 efficiency requirement:

  • The list of projects used to calculate savings should be complete and accurate.
  • Costs associated with the efficiencies should be taken into consideration when calculating efficiency savings.
  • Assumptions and costs associated with efficiency savings should be noted in the report to the Commission.
  • Efficiency savings that will not be available for investment in maintenance and rehabilitation should be separately identified in efficiency reports to the Commission.
  • The Efficiency Report to the Commission should clarify when the efficiency savings are based on estimates and not actual savings.
  • The Efficiency Report should include the definition of efficiency savings as adopted by Caltrans.

Specific recommendations for each efficiency and program area reviewed are contained in this report.

VIEWS OF RESPONSIBLE OFFICIALS

We have requested and received a written response to our recommendations from the Deputy Director of Project Delivery and the SB1 Program Manager. Please see Attachments for their complete responses and action plans.

Eraina Ortega
Inspector General
Independent Office of Audits and Investigations

EFFICIENCY MEASURES REPORT

BACKGROUND

The California Department of Transportation (Caltrans) created the Senate Bill 1 (SB 1) Program Manager position to oversee all Caltrans SB 1 activities. The SB 1 Program Manager is responsible, in part, for:

  • Oversight and coordination of all departmental activities related to the SB 1 program.
  • Establishing and maintaining the SB 1 program guidance, policies, business process and procedures to support the successful development and delivery ofSB 1 programs and SB1 program compliance requirements.
  • Working with Caltrans' subject matter experts to ensure that information presented to the public is consistent, accurate, and reliable.
  • Acting in the lead role to identify and implement $100 million in required annual efficiencies by working with the Director/Chief Deputy Director and other Department Programs/Divisions.

Additionally, the Financial Policy Board (Board) was designated as the oversight body responsible for approving efficiencies that will be included in the annual report to the California Transportation Commission (Commission). The Board is made up of deputy-level representation of Caltrans' core funding programs and chaired by the Chief Deputy Director. The Board is staffed by the Chief, Division of Budgets, and his/her designees. The Board makes final decisions on fiscal policy issues. The SB 1 Program Manager will present the final efficiencies report to the Board at its quarterly meeting preceding the final date of the report.

Based on our review ofthe Interim Efficiencies Report and the review of efficiencies, we have the following recommendations to improve future reports.

RECOMMENDATIONS
We recommend the SB1 Program Manager ensure the efficiency reports identify:

  • The definition of efficiency savings as adopted by Caltrans.
  • Whether each efficiency results in cost avoidance or savings.
  • The methodology for calculating each efficiency.
  • Assumptions and costs associated with each efficiency.
  • Efficiency savings that will not be available for investment m maintenance and rehabilitation ofthe state highway system.
  • Efficiency savings that are based on estimates and not actual savings.
  • How long each efficiency has been in place.

SB 1 PROGRAM MANAGER RESPONSE

The SB 1 Program Manager concurred with our recommendations. Please see Attachment 1 for details of the response and action plan.

SAVINGS DUE TO ACCELERATION OF WORK

BACKGROUND

All project costs recommended by Caltrans to the Commission for programming are to be recommended on the basis of their fully escalated (inflated) costs. The standard escalation rate for the State Highway Operation and Protection Program (SHOPP) is the rate specified in the last Commission adopted Fund Estimate for the SHOPP. A number ofprojects have been accelerated in anticipation of new funding levels. Acceleration of projects moves up the delivery date which can yield cost savings by keeping price inflation in check.


SUMMARY

Efficiency Calculation Methodology
Caltrans district staff identified projects that could be delivered early. Projects which were advanced one fiscal year were assumed to achieve a six month time-savings, and projects that were advanced two fiscal years were assumed to achieve an 18 month time-savings. The efficiency savings was calculated by multiplying the construction capital costs by a monthly escalation rate and the estimated times savings. The escalation rate of 4.2 percent used to calculate the savings was based on the 2018 Fund Estimate for the 20 18 SHOPP. However, the majority of the accelerated projects were programmed in the 2016 SHOPP for which the approved escalation rate was 4.5 percent.

At the time of programming, an estimated ready-to-list date is included in the programming request to the Commission. Prior to construction cost allocation, an actual ready-to-list date is required. A project achieves ready-to-list status when the plans, specifi cations, and estimates are complete, environmental and right-of-way clearances are secured, and all necessary permits are obtained. The Commission will only consider an allocation for construction and/or construction support funds to projects that are ready-to-list. Since the ready-to-list date at programming and actual ready-to-list dates are ascertainable, time­ savings could be calculated based on dates rather than estimates.

Assumptions
Acceleration of projects allows construction to begin sooner. When construction begins sooner, construction costs are lower because inflationary impacts are lower.

RECOMMENDATIONS
We recommend that Project Delivery ensure:

  • That time savings is calculated based on ready-to-list dates.
  • The calculation of savings is based on updated capital construction costs and ready-to-list dates.
  • The monthly escalation rate used is based on the programmed fund estimate rate.

PROJECT DELIVERY RESPONSE

Project Delivery concurred with our recommendations. Please see Attachment 2 for details ofthe response and action plan.

STREAMLINING ENVIROMENT REVIEWS - National Environmental Policy Act (NEPA) Assignment

BACKGROUND

In 2007, Caltrans and the Federal Highway Administration (FHWA) entered into a Memorandum of Understanding (MOU). Under the MOU, Caltrans assumed the majority of FHWA's responsibility under the National Environmental Policy Act (NEPA) for highway projects in California that are funded by the FHW A. In reviewing and approving projects under NEPA, Caltrans is responsible for complying with all applicable federal environmental laws and with FHWA's NEPA regulations, policies, and guidance. Time savings in the NEPA Assignment have been achieved through the elimination of FHWA's project-specific review and approval of NEPA documents and the removal of the exchange of documents and comments between Caltrans and FHWA, thus streamlining the federal environmental review and approval process. In addition, Caltrans acting as the federal lead is able to consult directly with federal resource agencies, achieving additional time savings.

To assume FHWA's responsibilities under the NEPA Assignment Program, the State of California must waive its constitutional right under the Eleventh Amendment of the United States Constitution to sovereign immunity against suits brought in federal court. The California State Legislature passed Assembly Bills 1039, 2650, 892, and 28 to provide California with a waiver. These limited waivers were codified in Section 820.1 ofthe California Streets and Highways Code (Streets and Highways Code). Caltrans was also required to prepare reports to the Legislature by January 1of2008, 2009, 2011 , and 2016. In part, the reports were required to contain an analysis ofthe time savings achieved as a result of the NEPA Assignment and associated costs to assume the responsibilities such as personnel costs to conduct and review environmental documents, personnel costs to manage litigation, administrative costs, and litigation costs.

SUMMARY
Baseline 

The baseline required by the Streets and Highways Code was 30 projects, excluding those projects categorically excluded from environmental review, approved immediately preceding the NEPA Assignment. In 2009, Caltrans increased the number of projects included in the baseline to 39 in order to achieve a more representative mix. The 39 projects were comprised of 31 projects that did not result in a significant adverse impact on the environment and 8 projects that did cause a significant adverse impact. The sample size ofbaseline projects that result in a significant adverse impact on the environment is small since far fewer projects are classified in this category. Additionally, the timeframe for environmental approvals of these projects is significantly longer due to the sensitivity of the project in terms of the nature, quality, and extent of environmental impact, regulatory requirements, and the level of public interest and controversy.

Time Savings Estimation
To calculate time savings, the Streets and Highways Code requires that an analysis be conducted comparing the environmental review timeframes of the baseline projects with the environmental review timeframes of projects approved by Caltrans under its assumption of NEPA responsibilities. Median time savings are calculated and reported by environmental approval milestones and the type of environmental documents prepared and approved. Although the last required report to the Legislature on the NEPA Assignment was in 2016, Caltrans continues to calculate the median time savings achieved and posts the results on its website. The latest median time savings for environmental approvals is 13.3 months for projects that did not result in a significant adverse impact on the environment and 125 months for projects that did.

The Streets and Highways Code excluded projects with Categorical Exclusions from the timeframe analysis. Categorical Exclusions are a category of NEPA actions that do not have a significant individual or cumulative effect on the environment. If a project is determined to be categorically excluded, the preparation of a NEPA document is not required unless the project is controversial on environmental grounds or it is subsequently determined that a project involves significant environmental effects. Certain types of excluded projects require studies to be prepared to document that they will not result in a significant environmental effect. Time savings for categorically excluded projects have been achieved through the elimination of FHWA's review and approval of NEPA documents and removal of the exchange of documents and comments between Caltrans and FHWA. For the purpose of calculating efficiency savings, Caltrans estimates that a one month time savings has been achieved for categorically excluded projects.

Initial Efficiency Calculation Methodology
The projects included in calculating savings were a list of SHOPP projects that achieved construction contract acceptance in the past three fiscal years. Construction contract acceptance is a point at which all the payments under contract have been made and the contractor is released. Programmed budget and expended budget amounts for the list of projects were multiplied by the appropriate median time savings based on the type of environmental documents required for each project. An annual capital escalation rate of4.2 percent, adopted by the Commission as part ofthe 2018 STIP Fund Estimate, was applied to these figures, budgeted and expended. This produced the total savings for the project based on programmed budgets and total savings based on final expenditures. The totals for all projects were then divided by three to arrive at a range of annual savings between programmed budget and expenditures.

The list of projects for which efficiency savings was calculated included projects that were not subject to the NEPA Assignment. Costs associated with the NEPA Assignment were not taken into consideration in the Interim Efficiencies Report presented to the Commission at its January of 2018 meeting.

Proposed Efficiency Calculation Methodology
The list of projects will be made up of projects that complete the Project Approval and Environmental Document phase during a given fiscal year. The list will be reviewed to ensure that it only contains projects subject to the NEPA Assignment. The savings will be calculated by multiplying the programmed construction capital amount by the times savings and the monthly escalation rate. The time savings achieved for projects that did not result in a significant adverse impact on the environment will also be used for projects that did cause a significant adverse impact due to the small size of the baseline projects for the latter. For example, using the latest median time savings of 13.3 on projects that did cause a significant adverse impact, instead of 125 months.

Assumptions
Time savings during the environmental process allows construction to begin sooner. When construction begins sooner, construction costs are lower because inflationary impacts are lower.

Waiver of Sovereign Immunity
Re-enacting legislation was not passed during the 2015-2016 legislative session, as a result Streets and Highways Code Section 820.1 was repealed on January 1, 2017. Caltrans' authority under the NEPA Assignment was temporarily suspended for 90 days. On March 29, 2017, Governor Brown signed into law Assembly Bill 28 which added Section 820.1 back to the Streets and Highways Code, reinstating California's waiver of immunity and Caltrans' authority to use the NEPA Assignment. The law includes a sunset clause to repeal Section 820.1 on January 1, 2020. Ifthe waiver of sovereign immunity expires before legislation is approved, Caltrans will be unable to sign environmental approvals for all FHW A funded projects. In addition, Caltrans will not be able to consult directly with the federal resource agencies on approvals for state and local projects. The responsibility would shift back to the FHW A which has downsized their environmental division since the NEPA Assignment and would add additional time to project review and approval, thereby eliminating the time savings Caltrans has achieved in the past 11 years.

Retaining the NEPA Assignment will ensure that Caltrans continues to realize efficiency savings by avoiding delays which result in escalation costs.

RECOMMENDATIONS

We recommend that Project Delivery:

  • Ensure the list of projects used to calculate savings is complete and accurate.
  • Ensure costs associated with the NEPA Assignment are taken into consideration when calculating efficiency savings.
  • Ensure the monthly escalation rate used to calculate savings is the same rate used to program costs.
  • Ensure that significant time lags between the final approval of environmental documents and the project approval are taken into consideration when calculating savings.
  • Work with the California State Transportation Agency to extend the waiver of sovereign immunity authority beyond December 31, 2019.

PROJECT DELIVERY RESPONSE

Project Delivery concurred with our recommendations. Please see Attachment 2 for details of the response and action plan.

VALUE ANALYSIS

BACKGROUND

Title 23 United States Code, Section 106, requires value analysis on all federally funded National Highway System projects with a total project cost of $50 million or more, regardless of whether Caltrans employees, local agencies, consultants, or others are accomplishing the work. In addition, value analysis is mandated on all federally funded National Highway System bridge projects with a total project cost of $40 million or more. Caltrans also encourages value analysis studies for projects not required by the federal mandate if it is determined that the projects could benefit from the use of the study. The first study conducted by Caltrans was in 1969.

The objectives of value analysis are to identify and develop value-improving alternatives to the baseline concepts that could save cost, reduce construction time, and maintain or improve project performance. The FHW A mandates that studies be performed prior to construction.

The study is required to be conducted by a multidisciplinary team composed of individuals who are not directly involved in the planning or design of the project. The team identifies major elements of each project and the functions these elements perform. The team brainstorms design alternatives to the baseline concepts. The baseline concept is based on the preliminary designs current at the time of the study. Each alternative is evaluated in terms of its potential impact to performance, cost, time, and risk relative to the baseline concept. Some alternatives may increase cost for the project, but the overall project performance is expected to be improved.

The costs of value analysis studies are consultant costs and costs associated with Caltrans employee participation. Consultant costs include administrative fees, quality control and traveling expenses and range from $30,000 to $40,000. The typical studies have six to eight Caltrans team members and take about 40 to 60 hours to complete.

A preliminary and final study report are generated for every study. A template for both reports is on the value analysis website. The purpose of the preliminary report is to provide documentation of the alternatives to the decision makers in order to get their responses to the viability and acceptability of the alternatives. The final report documents the decision makers' comments, implementation plans, and decisions. The final study report contains detailed information such as the purpose for the proposed project, the objectives of the value analysis study, description of baseline concepts, description of alternatives, advantages and disadvantages of the alternatives, the impact ofthe alternatives on the schedule and project risks, assumptions, initial cost estimates, and estimated savings.

The FHWA annually collects information on value analysis accomplishments such as the number of studies conducted, proposed and implemented recommendations, the value of the approved recommendations, costs to conduct the studies, and total savings achieved. In 2016, the reported savings achieved by Caltrans was approximately $86,500,000.

SUMMARY

Baseline and Alternative Cost Estimates

The value analysis teams develop a cost estimate for the baseline concepts and each accepted alternative. The costs savings is the difference between the baseline concept and the alternative. Auditors noted that two studies performed on separate projects in the same area within the same year contained unit costs for pavement that were drastically different. Specifically, one cost estimate for one project contained an estimated unit cost for Jointed Plain Concrete Pavement at $150 per cubic yard and the other project quoted an estimate of $300 per cubic yard. Based on the quantity of pavement placed on the project, the difference was nearly $17 million for this item alone.

Costs
The final study reports contain attendance sheets for the value analysis team members and are used to estimate the hours staff spent on the study. Caltrans was using an average hourly rate for engineers to calculate the study costs. However, the average hourly rate used for estimating staff costs was not current and did not include associated overhead costs. Starting with the September 2018 Efficiency Report, Caltrans will be using the average loaded rate for Project Delivery staff contained in the annual budget report.

Assumptions
Assumptions related to each alternative are unique and are identified in the value analysis study report. For example, assumptions are typically related to construction item quantities, unit costs, time savings, and/or activities that must be performed.

Initial Efficiency Calculation Methodology
The savings for this efficiency measure listed in the Interim Report on Efficiencies dated January 2018 was reported to be between $20 and $50 million per year. The amount was not based on specific projects, but on a percentage of average historical savings reported to FHWA.

Proposed Efficiency Calculation Methodology
Projects that had a value analysis study completed and are ready to list in a given fiscal year will be identified for reporting. A project achieves ready-to-list status when the plans, specifications, and estimates are complete, environmental and right-of-way clearances are secured, and all necessary permits are obtained. The list of value analysis studies will include both mandated and non-mandated studies. Auditors noted that a mandated study that had been performed for a project, which is ready to list in fiscal year 2017/18, was not included on the list of projects to be reported. The list of projects was derived from contracts for delivery compiled by the districts, which did not include the aforementioned project. Caltrans is working on a methodology to identify projects to ensure that the list used to report savings is complete. The final study reports for the projects will be reviewed by Caltrans to identify accepted alternatives and verify that the alternatives will be implemented prior to reporting the corresponding savings. Additionally, Caltrans will review the cost estimates for reasonableness.

RECOMMENDATIONS
We recommend that Project Delivery ensure:

  • Baseline and alternative cost estimates are calculated in accordance with accepted guidelines and are reasonable and supported.
  • The net value of all implemented alternatives is reported, regardless of whether the alternatives result in savings or increased cost.
  • Costs associated with performing and producing the value analysis studies are taken into consideration when calculating efficiency savings.
  • The average salary rate used to calculate staff costs is supported, complete, and current.
  • Savings do not overlap with Construction Manager/General Contractor or other efficiencies.

PROJECT DELIVERY RESPONSE 

Project Delivery concurred with our recommendations. Please see Attachment 2 for details of the response and action plan.

INNOVATIVE CONTRACTING TOOLS - Construction Manager/General Contractor

BACKGROUND

Under traditional means of contracting for the construction of highway improvement projects, construction of any portion of the project cannot begin until the implementing agency has developed complete plans and specifications for the entire project, placed the contract out for bid, and awarded the contract. As a result, the contractor who will be constructing the project has no involvement during the development of the project.

The Construction Manager/General Contractor (CMGC) delivery method allows the implementing agency to engage a construction manager to provide input during the design process. The construction manager is selected based on qualifications, past experience, or a best-value basis.

Caltrans is authorized by legislation to use this method on 22 projects.

The CMGC contractor may provide constructability reviews, value engineering suggestions, construction estimates, and other construction-related recommendations. Caltrans also hires an Independent Cost Estimator (ICE) to provide independent cost estimates and to advise Caltrans on cost-related issues. The CMGC contractor and ICE each independently prepare a contractor-style, production-based cost estimate and schedule at intermediate design milestones (typically at 30 percent, 60 percent, and 90 percent design) that are based on the draft construction plans and specifications. Ifthe CMGC contractor' s estimate is not within 10 percent ofthe ICE's estimate, Caltrans staff, the CMGC contractor, and the ICE meet to review pricing assumptions and attempt to reconcile price differences. Any pricing changes are carried forward to the next estimating milestone where the previous activities are repeated.

The CMGC contractor also develops, proposes, and tracks challenges and quantifies benefits of innovations throughout the preconstruction phase, including proposing criteria to evaluate suggestions and suggests improvements that will offer the most value in terms of cost, schedule, and quality. The CMGC contractor prepares, modifies, and maintains an innovation register, which identifies the person and entity that proposed the idea, the value for the idea (in terms of cost, savings, risk reduction/mitigation, and schedule impact), and which ideas were incorporated by the project team into the final design and construction documents. The ICE also reviews the innovation register to ensure that the estimated savings are reasonable and supported.

When design is completed to about 90 to 95 percent, the CMGC contractor will provide a price to construct the project. If the price is acceptable, the CMGC contractor will become the general contractor and will construct the project. If not, the project is put out to bid using traditional procurement means.

Caltrans is required to report to the Legislature the locations, descriptions, cost, status, stage of completion, and estimated time for completion of each CMGC project no later than July 1 of each year during which any project using this method is ongoing. Additionally, for projects completed under CMGC, the report shall include: data on initial costs estimates, actual cost upon completion, and the reason for any difference, estimated time for completion, actual time of completion, and the reason for any difference, and the number and dollar value of any change orders.

SUMMARY

Baseline
The baselines are the plans and specifications developed by the design team at the 30, 60, and 90 percent design milestones.

Costs
The costs associated with the CMGC process include CMGC contractor cost, ICE consultant costs and other Cal trans support costs. Cal trans support costs may ultimately decrease due to a reduction in change orders and claims. Costs will be tracked and reported to the Commission after the projects have been completed.

Assumptions
The overall assumption is that the use of the CMGC method will result in a more constructible project and a reduction in change orders and contractor disputes. The innovation registers contain a broad range of innovation concepts which are unique to each project. However, the assumptions related to each innovation are not identified.

Efficiency Calculation Methodology
The list of projects for which efficiencies will be reported in a given year will be made up of CMGC construction contracts that have been awarded during the fiscal year. The efficiency at this stage of the project will be the innovation savings contained on the innovation registers. Caltrans staff review the innovation register prior to reporting to ensure that the savings estimated are not for areas that are part ofthe normal design process.

Any cost savings due to a reduction in change orders and contractor disputes will be reported to the Commission after the project is completed.

RECOMMENDATIONS
We recommend that Project Delivery ensure:

  • The net value of all innovations is reported, regardless of whether the innovation results in estimated savings or increased costs.
  • The costs associated with the CMGC process are reported to the Commission after the projects have been completed.
  • Savings do not overlap with Value Analysis or other efficiencies.
  • Assumptions made for each innovation in estimating savings are identified and documented.

PROJECT DELIVERY RESPONSE

Project Delivery concurred with our recommendations. Please see Attachment 2 for details of the response and action plan.

ATTACHMENT 1

Date: July 19, 2018
File: P3010-0641

To: ERAINA ORTEGA
Inspector General
Independent Office Audits and Investigations

From: LAURIE BERMAN
Director

Subject: EFFICIENCY MEASURES AUDIT

Thank you for the opportunity to provide a response to the draft audit report on Efficiency Measures. Caltrans Executive Management identified efficiency measures with the goal of generating at least $100 million in savings annually and presented an Interim Report to the California Transportation Commission at its January 2018 meeting. I appreciate the Independent Office ofAudits and Investigations reviewing the Interim Report and associated policies and procedures used to calculate the proposed efficiency savings.

The purpose ofthe audit was to determine if Caltrans has internal controls in place to ensure that it meets the goal of generating at least $100 million in efficiencies per year as required by Senate Bill 1 (SB1) The auditors concluded that Caltrans programs have been proactive in identifying areas of efficiencies and developing processes and procedures to achieve the required SB 1 efficiency measures. The draft report offers general recommendations and specific recommendations for the SB 1 Office and Project Delivery to improve future reports. Attached are the responses addressing each recommendation, estimated completion date and responsible individual.

If you have questions or need additional information, please contact me at (916) 654-6130.

Attachment

c: Ryan Chamberlain, Chief Deputy Director
Karla Sutliff, Chief Engineer, Project Delivery
Carrie Pourvahidi, Acting SB 1 Program Manager, Office of the Director
William E. Lewis, Assistant Director, Independent Office of Audits and Investigations
Kevin Yee, Audit Manager, Independent Office ofAudits and Investigations
Juanita Baier, SB1 Program, Office of the Director

Date: July 19, 2018
File: P3010-0641

To: ERAINA ORTEGA
Inspector General
Independent Office Audits and Investigations

From: Carrie Pourvahidi
Acting SB 1 Program Manager
Office of the Director

Subject: EFFICIENCY MEASURES AUDIT

Thank you for the opportunity to provide a response to the draft audit report on Efficiency Measures.

The purpose of the audit was to determine if Caltrans has internal controls in place to ensure that it meets the goal of generating at least $100 million in efficiencies per year as required by Senate Bill 1 (SB1) We are pleased that the auditors recognize that Caltrans programs have been proactive in identifying areas of efficiencies and developing processes and procedures to achieve the required SB 1 efficiency measures.

The draft audit report provides recommendations for the SB1 Program Manager and for the Project Delivery Program to improve future reports. As you will notice, the SB1 Office has already begun efforts to implement the recommendations identified in the draft report. Attached is the SB1 Office's response to the recommendations contained in the draft report.

If you have questions or need additional information, please contact me at (916) 654-6130.

Attachment

c: Ryan Chamberlain, Chief Deputy Director
Karla Sutliff, Chief Engineer, Project Delivery
William E. Lewis, Assistant Director, Independent Office of Audits and Investigations
Kevin Yee, Audit Manager, Independent Office ofAudits and Investigations
Juanita Baier, SB1 Program, Office of the Director


Independent Office of Audits and Investigations (A&I) - Response to Draft Report
Report Audit Name:  Audit of Efficiency Measures
Audit Auditee: SB1 Program Manager
Audit No: P3010-0641
EFFICIENCY MEASURES REPORT
We recommend the SB1 Program Manager ensure the efficiency reports identify:
A&I Audit Recommendation Auditee Response to Draft Report Estimated Completion Date Staff Responsible
The definition of efficiency savings as adopted by Caltrans. The Efficiency reports will include the definition of efficiency savings adopted by Caltrans. The definition was approved at the Financial Policy Board meeting on February 13, 2018. Oct-18 Juanita Baier
Whether each efficiency results in cost avoidance or savings. The Efficiency reports will specify whether each efficiency results in cost avoidance or savings. Oct-18 Juanita Baier
The methodology for calculating each efficiency. The Efficiency reports will include the methodology for calculating each efficiency. Oct-18 Juanita Baier
Assumptions and costs associated with each efficiency. The Efficiency reports will include the assumptions and costs associated with each efficiency. Oct-18 Juanita Baier
Efficiency savings that will not be available for investment in maintenance and rehabilitation of the state highway system. The Efficiency reports will specify how the savings will be invested in maintenance and rehabilitation of the state highway system, based on their funding source. Oct-18 Juanita Baier
Efficiency savings that are based on estimates and not actual savings. The Efficiency reports will identify whether the calculation is based on estimated or actual savings per efficiency. Oct-18 Juanita Baier
How long each efficiency has been in place. The Efficiency reports will specify how long each efficiency has been in place. Oct-18 Juanita Baier

ATTACHMENT 2

Date: July 12, 2018
File: P3010-0641

To: ERAINA ORTEGA
Inspector General
Independent Office Audits and Investigations

From: Karla Sutliff
Chief Engineer
Project Delivery

Subject: PROJECT DELIVERY RESPONSE TO EFFICIENCY MEASURES AUDIT

Thank you for the opportunity to provide a response to the draft audit report on Efficiency Measures. The purpose ofthe audit was to determine ifCaltrans has internal controls in place to ensure that it meets the goal ofgenerating at least $100 million in efficiencies per year as required by Senate Bill 1 (SB 1 ). The auditors focused their evaluation on the Interim Report presented to the California Transportation Commission at its January 2018 meeting. We are pleased that the auditors recognize that Caltrans' programs have been proactive in identifying areas ofefficiencies and developing processes and procedures to achieve the required SB 1 efficiency measures.

The auditors reviewed the process, policies, procedures, and methodology used in developing the Interim Report and provided general recommendations and specific recommendations. The Divisions of Project Management, Design, and Environmental, under Project Delivery, are working on incorporating the recommendations when calculating efficiency measures. Attached is Project Delivery's response to the recommendations contained in the draft report.

If you have questions or need additional information, please contact me at (916) 654-6490.

Attachment

c: Ryan Chamberlain, Chief Deputy Director
Carrie Pourvahidi, Acting SB 1 Program Manager, Office of the Director
William E. Lewis, Assistant Director, Independent Office of Audits and Investigations
Kevin Yee, Audit Manager, Independent Office ofAudits and Investigations
Juanita Baier, SB1 Program, Office of the Director


Independent Office of Audits and Investigations (A&I) - Response to Draft Report
Report Audit Name: Audit of Efficiency Measures
Audit Auditee: Project Delivery
Audit No: P3010-0641
Savings due to Acceleration of Work
We recommend that Project Delivery ensure:
A&I Audit Recommendation Auditee Response to Draft Report Estimated Completion Date Staff Responsible
That time savings is calculated based on ready to list dates. The actual ready to list date will be used in the calculation. Oct-18 Matt Bailey
The calculation of savings is based on up-to-date capital construction costs and ready to list dates. Caltrans will use the actual estimated capital cost at the time the project is ready to list. Oct-18 Matt Bailey
The monthly escalation rate used is based on the programmed fund estimate rate. Caltrans will use the approved escalation rate in effect when the project was programmed. Oct-18 Matt Bailey
Streamlining Enviromental Reviews
Ensure the list of projects used to calculate savings is complete and accurate. The list of projects meeting environmental milestones prepared by project management for the fiscal year are reviewed by the environmental program. The review is to ensure that the list of projects is complete an accurate and meets the requirements for NEPA assignment. Oct-18 Matt Bailey
Ensure costs associated with the NEPA Assignment are taken into consideration when calculating efficiency savings. Caltrans will calculate NEPA Assignment cost savings and include applicable expenses incurred including legal costs as shown in published reports to the legislature and FHWA. Oct-18 Matt Bailey
Ensure the monthly escalation rate used to calculate savings is the same rate used to program costs. Caltrans will calculate de-escalation costs based on the escalation rate in effect at the time costs were programmed. Oct-18 Matt Bailey
Ensure that significant time lags between the final approval of environmental documents and the project approval are taken into consideration when calculating savings. The average time savings, which is based on the most recent documented saving timelines in published reports (recent trend is more than 1 year) for major environmental documents, will be adjusted if there is a significant lag between environmental document approval and project approval. Oct-18 Matt Bailey
Work with the California State Transportation Agency to extend the waiver of sovereign immunity authority beyond December 31, 2019. Caltrans is actively working with CalSTA to advance the legislation forward. In order to enact legislation before the expiration date, legislation would need to be introduced by February of 2019 for the Governor to approve by the fall of 2019. Oct-18 Matt Bailey
Streamlining Enviromental Reviews
Baseline and alternative cost estimates are calculated in accordance with accepted guidelines, and are reasonable and supported. Caltrans will review value analysis reports to ensure baseline and alternative cost estimates are in accordance with acceptable estimating practices and are reasonable. Oct-18 Matt Bailey
The net value of all implemented alternatives is reported, regardless of whether the alternatives result in savings or increased cost. Caltrans will report on the net value of all innovations implemented, for each project evaluated. Oct-18 Matt Bailey
Costs associated with performing and producing the value analysis studies are taken into consideration when calculating efficiency savings. Caltrans will calculate cost savings for innovations implemented and include applicable expenses incurred including staff time for team members performing the VA study, and consultant costs to conduct VA studies. Oct-18 Matt Bailey
The average salary rate used to calculate staff costs is supported, complete and current. Caltrans will use the latest “cost of a PY” that is developed annually as part of the Capital Outlay Support finance letter. These costs are reviewed and approved by the Department of Finance and the Legislative Analyst Office as part of the budget process. Oct-18 Matt Bailey
Savings do not overlap with Construction Manager/General Contractor or other efficiencies. Caltrans will compare projects in the annual report to other efficiency cost saving innovations to ensure that the same innovation is not duplicated Oct-18 Matt Bailey
Streamlining Enviromental Reviews
The net value of all innovations is reported, regardless of whether the innovation results in estimated savings or increased costs. For each project evaluated, Caltrans will report on the net value of all innovations implemented. Oct-18 Matt Bailey
The costs associated with the CMGC process are reported to the Commission after the projects have been completed. CMGC cost savings are evaluated both at time of contract award and completion of construction.
  • At the time of contract award, there are innovations that are implemented which would provide costs savings. Caltrans will include costs incurred to compensate contractor that provided input which results in potential innovations. The reported cost savings will account for both savings and expenses at time of award.
  • At the time of completion of construction, there may be additional savings due to reduced number of claims during the project and those cost savings may be reported at that time.
Oct-18 Matt Bailey
Savings do not overlap with Value Analysis or other efficiencies. Caltrans will compare projects in the annual report to other efficiency cost saving innovations to ensure that the same innovation is not duplicated. Oct-18 Matt Bailey
Assumptions made for each innovation in estimating savings are identified and documented. The CMGC evaluation reports will include the assumptions and costs associated with each efficiency. Oct-18 Matt Bailey

CONTACT THE INDEPENDENT OFFICE OF AUDITS AND INVESTIGATIONS:

Physical Address:
California Department of Transportation - Independent Office of Audits and Investigations
1304 O Street, Suite 200
Sacramento, CA 95814

Mailing Address:
California Department of Transportation - Independent Office of Audits and Investigations
Mail Station 2
P. O. Box 942874
Sacramento, CA 94274-0001

Phone: (916) 323-7111