California Department of Transportation Efficiency Measures Verification Audit

P3010-0648
Performance Audit
Issued: 7/23/2019
Corrective Action Plan: In Process

Audit Fact Sheet

Background

In April 2017, Governor Brown signed into law Senate Bill 1 (SB1), also known as the “Road Repair and Accountability Act of 2017.”  SB1 increases revenue for California’s transportation system.  SB1 requires that Caltrans implement efficiency measures with the goal of generating at least $100 million annually in savings to invest in maintenance and rehabilitation of the state highway system.  SB1 also requires that Caltrans report the savings to the California Transportation Commission (CTC).  Caltrans will consider efficiencies that result in cost avoidance or a reduction in support or capital costs.

The Independent Office of Audits and Investigations completed an audit of the efficiency savings reported in the Annual Efficiencies Report to the CTC for the Fiscal Year 2017/18 related to the National Environmental Policy Act (NEPA) assignment, Value Analysis, and Construction Manager/General Contractor (CMGC).  The objectives of the audit were to determine whether:

  • The estimated efficiency savings reported were supported.
  • California Department of Transportation (Caltrans) has developed a methodology to determine whether savings related to Value Analysis and CMGC will result in actual savings in the future.           

Key Findings

Caltrans reported $119.5 million in efficiency savings for the areas of the NEPA assignment, Value Analysis and CMGC.  We reviewed a sample of projects for the three areas with reported savings of $117.6 million and found an overreporting of cost avoidance in the amount of $8.7 million.  Specifically,

  • NEPA - Overreporting of cost avoidance in the amount of $849,688 for our sample of 30 projects.
  • Value Analysis - Overreporting of cost avoidance in the amount of $11,064,236 for six of the nine value analysis projects reviewed.
  • CMGC - Underreporting of cost avoidance in the amount of $3,229,683 for both projects that counted toward the $100 million efficiency savings goal.Additionally, one project with savings in the amount of $1,016,314 was from funding that was not available for investment in maintenance and rehabilitation of the state highway system.

Key Recommendations

We recommend Project Delivery:

  • Ensure the list of projects used to calculate cost avoidance efficiencies for NEPA Assignment is reviewed for completeness and accuracy.Specifically, projects that are not subject to NEPA, or those without project reports, should be excluded from the list.
  • Consider reporting cost avoidance for value analysis studies once projects have been awarded using the actual bid prices.This methodology will result in a more accurate estimation of efficiency savings.
  • Ensure cost avoidance estimates for Value Analysis and CMGC are detailed, based on final unit prices, and quantities, and the estimates are updated based on final plans and specifications and plans.

We recommend the SB1 Program Manager:

  • Ensure any efficiency savings not available for investment in maintenance and rehabilitation of the state highway system are separately identified in future reports to the CTC. 
  • Ensure the efficiencies report to the CTC includes an explanation on how efficiencies from projects will be available for investment in maintenance or rehabilitation of the state highway system.

The Independent Office of Audits and Investigations completed an audit of the efficiency savings reported in the Annual Efficiencies Report to the California Transportation Commission (CTC) for fiscal year 2017-2018.  The purpose of this audit was to determine if the savings reported in the Annual Efficiencies Report are supported and available for investment in the maintenance and rehabilitation of the state highway system, as required by Senate Bill 1 (SB1).  The focus was on the efficiency areas of the National Environmental Policy Act (NEPA) Assignment, Value Analysis process and Construction Manager/General Contractor (CMGC) process, which totaled approximately $119.5 million (90%) of the reported $133 million of savings reported to the CTC. 

Our testing found that the California Department of Transportation (Caltrans) overreported savings of $8.7 million out of $117.6 million tested, as shown in the table below.

Area Reviewed Savings Reported to CTC Amount Tested Amount <over> or under reported
NEPA Assignment

$13,420,709

$10,328,465

<$849,688>

Value Analysis

$61,590,000

$62,823,000*

<$11,064,236>

CMGC

$44,469,000

$44,469,000

$3,229,683

TOTAL

$119,479,709

$117,620,465

<$8,684,241>

*Four Value Analysis studies had negative savings, thus the amount tested is larger than total reported.

Our audit found that the methodology and processes to identify, track, and support efficiencies varied between each area.  As a result, we have specific recommendations for each efficiency area within their respective section of this report.  We also found that $1,521,644, which is included in the amount above, was not available for investment in the maintenance and rehabilitation of the state highway system.  This issue is detailed separately in the Reinvestment to the State Highway System section of this report.

Acknowledgement

During our audit, we met with and obtained information from subject matter experts.  Their knowledge of  the technical aspects of the efficiency areas was valuable during the audit process. Caltrans staff also provided helpful feedback on improving the reporting process.

The objectives of the audit were to determine if:

  • The estimated efficiency savings reported by Caltrans were supported.
  • Caltrans developed a methodology to determine if savings related to Value Analysis and CMGC will result in actual savings in the future.

The audit covered the period from July 1, 2017 to June 30, 2018.  We conducted our audit from October 26, 2018, to June 11, 2019.  Changes after this date were not tested and accordingly, our conclusions do not pertain to changes arising after June 11, 2019. 

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based

on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

In April 2017, Governor Brown signed into law SB1, also known as the “Road Repair and Accountability Act of 2017.”  SB1 increases revenue for California’s transportation system and requires Caltrans implement efficiency measures with the goal of generating at least $100 million annually in savings to invest in maintenance and rehabilitation of the state highway system and to report the savings to the CTC.  Caltrans considers cost avoidance or a reduction in support or capital costs as efficiencies.

In January 2018, Caltrans provided an Interim Efficiencies Report to the CTC, which our office audited.  The scope of that audit was limited to, among other objectives, determining if the methods for calculating efficiency savings were appropriate and supported.  We did not audit the efficiency savings reported in the Interim Efficiencies Report, which was an objective of this audit.

In October 2018, Caltrans provided its first Annual Efficiencies Report to the CTC for fiscal year 2017-2018 to comply with the SB1 requirement.  The report identified six areas of efficiencies with $133 million in savings.  The largest areas of efficiencies were the NEPA Assignment, the Value Analysis process, and the CMGC process, and were tested as part of this audit.  The background for these specific areas is discussed within each section of this report.

We requested and received a written response to our recommendations from the Deputy Director of Project Delivery and the SB1 Program Manager.  Please see Attachment for their complete response and action plans.

RESULTS AND RECOMMENDATIONS

Background

In 2007, the California Department of Transportation (Caltrans) and the Federal Highway Administration (FHWA) entered into a Memorandum of Understanding (MOU) whereby, Caltrans assumed the majority of FHWA’s responsibility under NEPA for highway projects in California funded by FHWA.  This action resulted in time savings through the elimination of FHWA’s project-specific reviews and approval of NEPA documents, which streamlined the federal environmental review and approval process.  In addition, it allowed Caltrans to act as the federal lead and communicate directly with federal resource agencies to achieve additional time savings.

Time Savings Estimation

To assume FHWA’s responsibilities under the NEPA Assignment Program, Caltrans was mandated to prepare biennial reports to the Legislature that, in part, were required to contain an analysis of the time savings achieved as a result of the NEPA Assignment.   Although the last required report to the Legislature was in 2016, Caltrans continues to calculate the median time savings achieved and posts the results on its website. 

The report to the Legislature excluded projects with Categorical Exclusions from the analysis.  Categorical Exclusions are a category of NEPA actions that do not have a significant individual or cumulative effect on the environment.  However, Caltrans estimates that a one-month time savings was  achieved for each categorically excluded project.  

Waiver of Sovereign Immunity

To assume FHWA’s responsibilities under the NEPA Assignment Program, the State of California must waive its constitutional right under the Eleventh Amendment of the United States Constitution to sovereign immunity against suits brought in federal court.  Re-enacting legislation was not passed during the 2015-2016 legislative session, and as a result, the Streets and Highways Code Section 820.1 was repealed on January 1, 2017, and Caltrans’ authority under the NEPA Assignment was temporarily suspended for 90 days.  On March 29, 2017, Governor Brown signed into law Assembly Bill 28 which added Section 820.1 back to the Streets and Highways Code, reinstating California’s waiver of immunity and Caltrans’ authority to use the NEPA Assignment.  The law includes a sunset clause to repeal Section 820.1 on January 1, 2020.  If the waiver of sovereign immunity expires before legislation is approved, Caltrans will be unable to sign environmental approvals for all FHWA funded projects.  In addition, Caltrans will not be able to communicate directly with the federal resource agencies on approvals for state and local projects.  The responsibility would shift back to FHWA and would add additional time to project review and approvals, thereby eliminating the time savings achieved by Caltrans in the past 12 years.    

Caltrans, by retaining the NEPA Assignment, will ensure that it continues to realize efficiency savings by avoiding delays that result in project escalation costs. 

Caltrans’ Methodology for NEPA Assignment Efficiency Calculation

The list of NEPA projects consists of those that completed the Project Approval and Environmental Document (PA/ED) phase in a given fiscal year.  The list is reviewed by Caltrans staff to ensure that it only contains projects subject to the NEPA Assignment.  The savings is calculated by multiplying the programmed construction capital amount by the time savings and a monthly escalation rate.  All project costs recommended by Caltrans to the California Transportation Commission (CTC) for programming (the identification and commitment of funds for a project) are based on fully escalated (inflated) costs.  The time savings for environmental assessments is estimated at 13 months and the time savings for categorically excluded projects is one month.  The monthly escalation rate is based on the annual escalation rate approved by the CTC. 

Audit Methodology

We reviewed the list of projects Caltrans used to calculate the efficiency savings in its Annual Efficiencies Report to the CTC for the Fiscal Year 2017-2018.  Specifically, we evaluated the projects to determine if:

  • The projects contained on the list were subject to NEPA.
  • The efficiency savings for the projects would be available for investment in the maintenance and rehabilitation of the state highway system.
  • The correct programmed capital construction costs, the correct time savings, and the correct escalation rate were used to calculate the efficiency savings.
  • Any of the projects reported under the Acceleration of Work efficiency were not also reported under the NEPA Assignment efficiency.(Acceleration of Work is another efficiency area included in the Annual Efficiencies Report, but not included in our testing.)
  • There was a significant time lag, greater than 90 days, between the environmental document date and the project approval date.

Results

The list of projects used to calculate efficiency savings for the 2017-2018 Efficiencies Report contained 222 projects with categorical exclusions and 10 projects with environmental assessments with a reported cost avoidance of $13,420,706.  We determined Caltrans used the correct time savings and annual escalation rate of 4.5 percent, adopted by the CTC for 2016.  We also tested 30 projects with an associated savings of approximately $10.3 million reported to the CTC.    

Our review found that the cost avoidance reported for NEPA Assignment efficiencies was materially supported for our sample of projects tested.  However, we noted an overreporting of cost avoidance in the amount of $849,688 as follows:

  • We found one project included in the list of projects for the NEPA Assignment efficiencies was also reported under the Acceleration of Work efficiencies.The savings for this project reported under the NEPA Assignment efficiency was $35,644 which should have been excluded from the total NEPA savings.
  • Three projects out of the 30 sampled should have been excluded from the calculation as follows:
    • One project, with a savings of $88,920, was not subject to NEPA.
    • One project was funded by a source for which the savings of $292,152 would not be available for investment in maintenance and rehabilitation of the state highway system.(See Reinvestment to the State Highway System section below.)
    • One project, with a savings of $907,330, did not have a project report prepared, and therefore did not reach the PA/ED phase.
  • The calculation used to determine the savings did not use the programmed capital construction amount, but based the calculation on the estimated capital construction amount. This resulted in underreporting the amount of savings by $474,358 for our sample of 30 projects.

Finally, we found that 5 out of the 30 projects sampled had a time lag of more than 90 days between the final approval of environmental documents and the project approval ranging from 98 days to 238 days.  Although the delays were explained and the time savings were not affected, we encourage the districts to track the time delays in order to determine if efficiencies in this area can be achieved in the future.

Recommendations

We recommend the Project Delivery Program ensure:

  • The efficiency savings calculation is based on the programmed capital construction amount and not the estimated capital construction amount.
  • The list of projects included in the calculation is reviewed to ensure that projects reported under the Acceleration of Work are not also reported under NEPA.
  • The list of projects included in the calculation is reviewed for completeness and accuracy.Specifically, projects that are not subject to NEPA or those without project reports should be excluded from the list.
  • Time lags between the final approval of environmental documents and the project approval are tracked to determine if efficiencies in this area can be achieved in the future.

Our recommendation on efficiency savings not being available for investment in the maintenance and rehabilitation of the state highway system is addressed in the last section of this report.

Project Delivery Program Response

The Project Delivery Program agreed with our recommendations.  Please see Attachment for their detailed response and action plan.

Background

Title 23 United States Code, Section 106, requires value analysis on all federally funded National Highway System projects with a total project cost of $50 million or more, regardless of whether Caltrans employees, local agencies, consultants, or others are accomplishing the work.  In addition, value analysis is mandated on all federally funded National Highway System bridge projects with a total project cost of $40 million or more. Caltrans also encourages value analysis studies for projects not required by the federal mandate if it is determined that the projects could benefit from the use of the study. 

The objectives of value analysis studies are to identify and develop value-improving alternatives to the baseline concepts that could lower cost, reduce construction time, and maintain or improve project performance.  The FHWA mandates that studies be performed prior to construction. 

The study is required to be conducted by a multidisciplinary team composed of individuals who are not directly involved in the planning or design of the project.  The team identifies major elements of each project and the functions these elements perform.  The team brainstorms design alternatives to the baseline concepts.  The baseline concept is based on the preliminary designs created by Caltrans at the time of the study.  Each alternative is evaluated and some alternatives may increase costs for a project, but the overall project performance is expected to improve.

A preliminary and final study report are generated for every study.  A template for both reports is on the value analysis website.  The purpose of the preliminary report is to provide documentation of the alternatives to the decision makers in order to get their responses to the viability and acceptability of the alternatives.  The final study report documents the decision makers’ comments, implementation plans, and decisions.  The final study report contains detailed information such as the purpose for the proposed project, the objectives of the value analysis study, description of baseline concepts, description of alternatives, advantages and disadvantages of the alternatives, the impact of the alternatives on the schedule, and project risks, assumptions, initial cost estimates, and estimated savings.  

Caltrans’ Methodology for Value Analysis Efficiency Calculation

Projects that had a value analysis study completed and are ready to list in a given fiscal year are identified for reporting the efficiency savings.  A project achieves ready to list status when the plans, specifications, and estimates are complete; environmental and right-of-way clearances are secured; and all necessary permits are obtained.  The list of value analysis studies includes both mandated and non-mandated studies.  The value analysis teams develop initial cost estimates for the baseline concepts and each accepted alternative.  The cost savings is the difference between the cost for the baseline concept and the cost for the alternative.    The final study reports for the projects are reviewed by Caltrans to identify the accepted alternatives and verify that the alternatives will be implemented prior to reporting the corresponding savings.  Additionally, Caltrans reviews the cost estimates for reasonableness.

Audit Methodology

We reviewed the 2017-2018 Efficiencies Report to obtain the list of value analysis projects.  We selected a sample of six out of nine studies completed and requested the districts provide updated cost savings estimates.  We reviewed the initial estimates provided by the districts to ensure:

  • Only implemented alternatives were included in estimating efficiency savings reported to the CTC, and material quantities were updated based on plans and specifications.  This information was provided by the subject matter experts.
  • Estimates were updated based on the engineer’s estimate or the low bid for awarded projects.  The engineer’s estimate is known at the time the project is ready to list, but the actual cost of bid unit prices is known only after the project is awarded.
  • Baseline and alternative unit prices for the same material matched.
  • The cost estimates included preparation work required for using different material. 
  • Correct mark-up (e.g., contingency percentage) was applied.
  • The efficiency savings for the projects would be available for investment in the maintenance and rehabilitation of the state highway system.

Results

We reviewed six of the nine value analysis projects contained in the 2017-2018 Efficiencies Report with an associated reported savings totaling $62.8 million.  We identified an overreporting of cost avoidance in the amount of $11,064,236 as follows:

  • The estimates were not updated to reflect changes in the plans and specifications, and current unit prices which resulted in overreporting of $10,851,058.
  • One project was partially funded by a source for which the savings in the amount of $213,178 would not be available for investment in the maintenance and rehabilitation of the state highway system.(See Reinvestment to the State Highway System section below.)

The 2017-2018 Efficiencies Report was the first official report to the CTC and the process for updating value analysis information was not fully developed and communicated to the districts at the time.    As a result, we found the cost estimates in the final study reports and the requested updates were inconsistent among districts based on the level of detail included, the areas updated, and the mark-up applied. 

Recommendations

We recommend the Project Delivery Program:

  • Consider reporting cost avoidance for value analysis studies once the projects have been awarded using actual bid prices.This methodology will result in a more accurate estimation of efficiency savings because actual unit prices will be used rather than the engineer’s estimated unit prices.
  • Provide guidance to districts in order to ensure consistent methodology in updating initial cost estimates.
  • Perform quality assurance on the updated estimates prepared by the districts.Specifically, ensure:
    • Baseline and alternative unit prices for the same materials are the same. 
    • The alternative estimate includes preparation work and materials, and the baseline mark-up should only reflect similar work and materials necessary to implement the baseline concept. The work and materials included in the mark-up should be specified.
    • Quantities should be updated for the alternatives based on plans and specifications.  Also, if the baseline quantities contained in the study are not reasonable, they should be updated.

Project Delivery Program Response

The Project Delivery Program agreed with our recommendations.  Please see Attachment for their detailed response and action plan.

Background

Under traditional means of contracting for the construction of highway improvement projects, construction of any portion of the project cannot begin until the implementing agency has developed complete plans and specifications for the entire project, advertised the contract for bid, and awarded the contract.  As a result, the contractor who will be constructing the project is not involved with the development and design of the project.

The CMGC delivery method allows the implementing agency to engage a construction manager to provide input during the design process, per the Public Contract Code (PCC) section 6700. The construction manager is selected based on qualifications or on a best-value basis, as permitted under PCC section 6703.

The CMGC contractor may provide constructability reviews, value engineering suggestions, construction estimates, and other construction-related recommendations.  Caltrans also hires an Independent Cost Estimator (ICE) to provide independent cost estimates and to advise Caltrans on cost related issues.  The CMGC contractor and ICE each independently prepare a contractor-style, production-based cost estimate and schedule at intermediate design milestones (typically at 30 percent, 60 percent, and 90 percent design). These cost estimates are based on the draft construction plans and specifications.  

The CMGC contractor also develops, proposes, tracks challenges, and quantifies benefits of innovations throughout the preconstruction phase.  The CMGC contractor prepares, modifies, and maintains an innovation matrix, which identifies the person and entity that proposed the innovation, the value of the innovation (in terms of cost, savings, risk reduction/mitigation, and schedule impact), and which innovations were incorporated by the project team into the final design and construction documents.  The ICE also reviews the innovation matrix to ensure that the estimated savings are reasonable and supported.    

When design is about 90 to 95 percent complete, the CMGC contractor will provide a price to construct the project.  If the price is acceptable to Caltrans, the CMGC contractor will become the general contractor and will construct the project.  If not, the project is put out to bid using traditional procurement means.

CMGC Efficiency Calculation Methodology

The list of projects and their efficiencies that are reported in a given year is made up of CMGC construction contracts that have been awarded during the fiscal year.  The efficiency at this stage of the project is the innovation savings contained in an innovation matrix.  Caltrans CMGC Program reviews the innovation matrix prior to reporting to ensure that the savings estimated are not for areas that are part of the normal design process.  The baselines are the plans and specifications developed by the design team.

Audit Methodology

We reviewed the 2017-2018 Efficiencies Report to obtain the list of projects with reported efficiency savings and selected the two projects categorized as highway facility rehabilitation, reconstruction, or replacement projects.  We relied on the subject matter expert’s conclusions regarding whether innovations for both projects were implemented in the final plans and specifications and reviewed the innovation matrices provided by the CMGC Program to ensure that:

  • Estimates were updated based on the final unit prices.
  • Baseline and innovation unit prices for the same material matched.
  • Material quantities were updated based on approved plans and specifications.  This information was provided by the subject matter experts. 
  • The efficiency savings for the projects would be available for investment in maintenance and rehabilitation of the state highway system.

We met with the CMGC Program and the ICE to review the innovation matrices provided to obtain additional information and clarification on the innovations.  We requested updated matrices from the CMGC Program for the two projects based on the final unit prices. 

Results

We reviewed the two CMGC projects with an associated reported savings totaling $44.5 million and found the initial innovation matrices provided by the CMGC Program did not contain enough information to support the estimated efficiency savings.  For example, some innovations did not provide the quantity and unit price information and only referenced the estimated savings amount.  Also, the assumptions made to determine the efficiency savings were not detailed and some innovations lacked enough description information to enable a reviewer without design knowledge to determine how the innovation would result in an efficiency.  We also found the cost saving estimates were not updated to reflect the final unit price which resulted in an underreporting of cost avoidance in the amount of $3,229,683.  This includes one project that was partially funded by sources for which the savings in the amount of $1,016,314 would not be available for investment in the maintenance and rehabilitation of the state highway system.  (See Reinvestment to the State Highway System section below.)

Recommendations

We recommend the Project Delivery Program:

  • Update the innovation matrix template to ensure that:
    • Innovations implemented into the final plans and specifications include quantity and price information for both the baseline concept and the innovation.
    • Assumptions made for each implemented innovation are identified and documented to support how the innovation resulted in an efficiency.
  • Update the innovation matrices based on the final unit prices.

Project Delivery Program Response

The Project Delivery Program agreed with our recommendations.  Please see Attachment for their detailed response and action plan.

Background

SB1 requires that Caltrans implement efficiency measures with the goal of generating at least $100 million annually in savings to invest in the maintenance and rehabilitation of the state highway system.  The State Highway Operation and Protection Program (SHOPP) is Caltrans’ funding mechanism for the rehabilitation of all state highways and bridges.   Caltrans created the SB1 Program Manager position to oversee all Caltrans SB1 activities, which includes coordinating and reporting efficiency measures with savings to invest in the maintenance and rehabilitation of the state highway system. 

Results

We identified four projects with savings that appeared not to be available for investment in the maintenance and rehabilitation of the state highway system, as they were not included in the SHOPP. We requested the Division of Budgets, Division of Transportation Programming, and Division of Project Management (Divisions) review the list of projects to determine if the savings from these four projects would be available for investment in the maintenance and rehabilitation of the state highway system.

After a review of the project information, the Divisions determined that savings for three out of the four projects would not be available for investment in the maintenance and rehabilitation of the state highway system as follows:

Project Name

Efficiency Area

Amount of Savings Reported for Project

Not Available for Maintenance and Rehabilitation

Feather River Aquatic Passage

NEPA Assignment

$292,152

$292,152*

Sac 5 Corridor Enhancement Project

Value Analysis

$4,161,000

$213,178*

Barton Road

CMGC

$3,203,000

$1,016,314*

TOTAL

$7,656,152

$1,521,644

*Results were identified in the appropriate efficiency sections above. 

Recommendations

We recommend the SB1 Program Manager:

  • Ensure that any efficiency savings not available for investment in maintenance and rehabilitation of the state highway system are separately identified in future reports to the CTC. 
  • Ensure that the efficiencies reported to the CTC include an explanation on how efficiencies will be available for investment in the maintenance or rehabilitation of the state highway system.

SB1 Program Response Manager

The SB1 Program Manager agreed with our recommendations.  Please see Attachment for the detailed response and action plan.

Audit Finding #1

Maintenance and Rehabilitation 

1.1 IOAI Audit Recommendation

Ensure that any efficiency savings not available for investment in maintenance and rehabilitation of the state highway system are separately identified in future reports to the California Transportation Commission (CTC).  

Auditee Response to Draft Report

The SB 1 Program will ensure that the next Annual Efficiencies Report and subsequent reports identify efficiency savings not available for investment in maintenance and rehabilitation of the state highway system.

Estimated Completion Date

10/2019

Staff Responsible

Juanita Baier

1.2 IOAI Audit Recommendation

Ensure that the efficiencies report to the CTC includes an explanation on how efficiencies from non-State Highway Operation and Protection Program projects will be available for investment in maintenance or rehabilitation of the state highway system. 

Auditee Response to Draft Report

The SB 1 Program will ensure that the next Annual Efficiencies Report and subsequent reports explain whether efficiencies identified from non- SHOPP projects are available for investment in maintenance and rehabilitation of the state highway system.

Estimated Completion Date

10/2019

Staff Responsible

Juanita Baier

Audit Finding #1

National Environmental Policy Act (NEPA) Assignment

1.1 IOAI Audit Recommendation

The efficiency calculation is based on the programmed capital construction amount and not the estimated capital construction amount.

Auditee Response to Draft Report

Agree. Project Delivery will calculate efficiencies based on the programmed capital construction amount and not the estimated capital construction amount all future fiscal year calculations.

Estimated Completion Date

10/2019

Staff Responsible

Scott Williams

1.2 IOAI Audit Recommendation

The list of projects included in the calculation is reviewed to ensure that projects reported under the Acceleration of Work are not reported under NEPA.

Auditee Response to Draft Report

Agree. Project Delivery will ensure that projects included on the NEPA list are not duplicating efficiencies found elsewhere

Estimated Completion Date

10/2019

Staff Responsible

Scott Williams

1.3 IOAI Audit Recommendation

The list of projects included in the calculation is reviewed for completeness and accuracy.  Specifically, projects that are not subject to NEPA, or those without project reports, should be excluded from the list.  

Auditee Response to Draft Report

Agree. Caltrans will exclude projects that are not subject to NEPA.

Estimated Completion Date

10/2019

Staff Responsible

Scott Williams

1.4 IOAI Audit Recommendation

That time lags between the final approval of environmental documents and the project approval are tracked to determine if efficiencies in this area can be achieved in the future.

Auditee Response to Draft Report

Agree. Caltrans will track the duration between final environmental document and project approval to determine if efficiencies can be achieved in this area.

Estimated Completion Date

10/2019

Staff Responsible

Scott Williams

Audit Finding #2

Value Analysis

2.1 IOAI Audit Recommendation

Consider reporting cost avoidance for value analysis studies once the projects have been awarded using the actual bid prices.  This methodology will result in a more accurate estimation of efficiency savings.

Auditee Response to Draft Report

Project Delivery agrees with this recommendation and will begin transitioning to reporting projects at Award using actual bid prices instead of at Ready to List.  Savings from Value Analysis studies for some projects have already been reported for the 2018-19 fiscal year.  Those projects will continue to be reported at Ready to List for the current fiscal year.  Project Delivery will be fully compliant with this recommendation for the 2019-20 fiscal year.

Estimated Completion Date

6/2020

Staff Responsible

Belinda Hon

2.2 IOAI Audit Recommendation

Provide guidance to districts in order to ensure consistent methodology in updating initial cost estimates.

Auditee Response to Draft Report

Project Delivery will provide guidance to districts on cost savings methodology for Value Analysis studies in order to ensure consistent methodology in reporting cost savings.  

Estimated Completion Date

12/2019

Staff Responsible

Belinda Hon

2.3 IOAI Audit Recommendation

Perform quality assurance on the updated estimates prepared by the districts.  Specifically, ensure:

a. Baseline and alternative unit prices for the same materials are the same.

b. The alternative estimate includes preparation work and materials, and the baseline markup should only reflect similar work and materials necessary to implement the baseline concept. The work and materials included in the markup should be specified.

c. Quantities should be updated for the alternatives based on specifications and plans.  Also, if the baseline quantities contained in the study were not reasonable, they should also be updated.

Auditee Response to Draft Report

Project Delivery agrees to perform qualify assurance on Value Analysis estimates prepared by the districts.  In particular, Project Delivery commits to

a. Check for consistency in unit prices between the baseline estimate and the alternative estimate

b. Ensure that baseline estimates reflect all work necessary to implement the baseline and to provide more details on work included in any markup used.

c. Ensure quantities for Value Analysis alternatives are updated once the plans and specifications have been completed and validate that the baseline quantities are reasonable.

Estimated Completion Date

10/2019

Staff Responsible

Belinda Hon

Audit Finding #3

Construction Manager/General Contractor (CMGC)

3.1 IOAI Audit Recommendation

Update the innovation matrix template to ensure that:

a. Innovations implemented into the final plans and specifications include quantity and price information for both the baseline concept and the innovation.

b. Assumptions made for each implemented innovation are identified and documented.

Auditee Response to Draft Report

Project Delivery agrees to update the innovation matrix template and create backup documentation templates to ensure that:

a. quantity and price information for both the baseline concept and the implemented innovation are documented.

b. Assumptions for each implemented innovation are documented.

Estimated Completion Date

12/2019

Staff Responsible

Ray Tritt

3.2 IOAI Audit Recommendation

Update the innovation matrices based on the final unit prices.

Auditee Response to Draft Report

Project Delivery agrees with the recommendation and will ensure that innovation matrices are updated to reflect final agreed unit prices.

Estimated Completion Date

10/2019

Staff Responsible

Ray Tritt