City of Santa Cruz 

The Independent Office of Audits and Investigations (IOAI), California Department of Transportation (Caltrans), audited the City of Santa Cruz's (City) project deliverables (i.e. outputs) and outcomes of Active Transportation Program (ATP) agreement ATPL-5025(065) (Agreement), Project Branciforte Creek Bicycle/Pedestrian Bridge (Project). We found that the initial pedestrian and bicyclist (User) counts, project deliverables, and outcomes from the project allocation of $1,800,000 were not consistent with the executed Agreement, ATP Application, and the California Transportation Commission (CTC) ATP Guidelines. The audit of the project outcomes was limited to the City's initial User counts as the one and five-year outcome milestones are not yet due.

We performed our audit to determine whether project deliverables and outcomes were consistent with the project scope, schedule, and benefits described in the Agreement or approved amendments, the ATP Application, and the CTC ATP Guidelines. Our audit period was from June 30, 2016, to October 10, 2018.

The Agreement was between Caltrans and the City. The audit was limited to compliance activities. Our field work was completed on October 10, 2018. Transactions occurring subsequent to this date were not tested and, accordingly, are not reflected in our conclusion.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. The audit was less in scope than an audit performed for the purpose of expressing an opinion on the financial statements of the City. Therefore, we did not audit and are not expressing an opinion on the City's financial statements.

The audit included interviews of City staff, review of prior audit reports, assessment of internal controls, review of project deliverables and outcomes, and obtaining an understanding of the City's grant management system. The audit also included review of project reports and a visit to the project site. An audit includes examining, on a test basis, evidence supporting the amounts and disclosures in the data and the records selected.

The ATP was created by Senate Bill 99 (Chapter 359, Statutes of 2013) and Assembly Bill 101 (Chapter 354, Statutes of 2013) to encourage increased use of active modes of transportation. The ATP consolidated various transportation programs into a single program and was originally funded at about $123 million a year from a combination of state and federal funds. Recently, the Road Repair and Accountability Act of 2017, Senate Bill 1 (Chapter 5, Statutes of 2017), added approximately $100 million per year in available funds for the ATP. 

The Project consisted of constructing a 12-foot wide bicycle/pedestrian bridge over Branciforte Creek and a 12-foot wide multi-use path under the Soquel Avenue bridge to meet the existing 12-foot wide multi-use path.

Based on our audit, we determined that the project was built consistent with the Agreement, however, not all deliverables and outcomes to date were consistent with the project scope and benefits described in the Agreement, ATP application, and the CTC ATP Guidelines. Final outcomes could not be fully assessed as the milestones were estimated for one and five-year periods after the project becomes operable; therefore, at the time of this audit, the outcomes were not yet known. 

Specifically, we determined that:

• The City did not correctly calculate beginning Users to estimate project outcomes.
• The first Project Progress Report due six months after project allocation was not submitted for eight months, one day before project was accepted by the City.

Our findings and recommendations take in account the City's response dated December 10, 2018, to our December 3, 2018, draft report. Our findings and recommendations, the City's response, and our analysis of the response are set forth in the Findings and Recommendations section of this report. The report is a matter of public record and will be placed on IOAI's webpage which can be viewed at: ig.dot.ca.gov.

The City of Santa Cruz (City) did not correctly estimate the beginning daily pedestrians and bicyclists (Users) for the Branciforte Creek Bicycle/Pedestrian Bridge Project (Project). The City used the National Bicycle and Pedestrian Documentation Project model to estimate Users, however, the City did not use the model correctly. The City only used the hourly adjustment factor and did not include the required daily adjustment factor (1.05) in their calculation. In addition, the model "strongly encourages" User estimates be based on the average of at least two, preferably three, counts. The City's estimation was based on only one count. 

By not using the recalculation model correctly to determine the beginning number of Users, the City will not be able to accurately compare to the outcomes and evaluate the effectiveness of the Project.

Program Supplement No. Q00 to Administering Agency-State Agreement for State Funded Project No. 00244S states, "This project will be administered in accordance with the applicable CTC STIP Guidelines and the Active Transportation Program Guidelines as adopted or amended, the LAPM and this Program Supplement."

Local Assistance Programs Guidelines (LAPG), Chapter 22 effective March 26, 2015, Section 22.17, "The Active Transportation Program will be evaluated for its effectiveness in increasing the use of active modes of transportation in California. Evaluation is a requirement on all infrastructure and non-infrastructure projects. 1. For ATP bike, pedestrian, and trail projects: the implementing agency must submit before and after pedestrian and/or bicycle counts, and an explanation of the methodology used for conducting the counts."

National Bicycle and Pedestrian Documentation Project Count Adjustment Factors March 2009 states, "Given the variability of bicycle and pedestrian activity, we strongly encourage that all estimates be based on the average of at least two (2) and preferably three (3) counts during the same time period and week, especially for lower volume areas.... Your next step is to multiply those counts by 1.05."

Recommendation:

We recommend the City establish policies and procedures to properly administer ATP projects and train staff accordingly to correctly follow methodologies used to estimate and evaluate project Users.

Summary of Response from the City of Santa Cruz

The City states that a single hourly count was performed and although two or three counts are strongly encouraged, it is not a requirement of the National Bicycle and Pedestrian Documentation Project (NBPDP) Model. The City acknowledged the daily adjust factor (1.05 or 5%) step was accidently missed in project user calculation. The NBPDP now has a fillable spreadsheet that can be downloaded and performs the calculations.

See Attachment I for the City's full response

The City did not submit their first semi-annual Project Progress Report for the Project until February 12, 2018, which was eight months after the City was allocated their funds. The City Council accepted the project on February 13, 2018.

The City relied on the instructions from the ATP Guidelines, but the guidelines did not include instructions on how to prepare and submit semi-annual reports. By not submitting timely PPRs, Caltrans may not be able to ensure projects are executed in a timely fashion and within scope and budget.

2015 Active Transportation Guideline Section 23 states, "As a condition of the project allocation, the Commission requires the implementing agency to submit semi-annual reports on the activities and progress made toward implementation of the project...."

Recommendation:

We recommend the City submit future Project Progress Reports timely.

Summary of Response From City Of Santa Cruz:

The City acknowledges the late submittal of the semi-annual report by two months. They also recognize that Caltrans now has an established submittal process with Caltrans standard forms.

See Attachment I for the City's full response.

Analysis Of The City Of Santa Cruz Response:

We appreciate the City's efforts to implement a report submittal process. Any procedures implemented subsequent to our fieldwork has not been audited or reviewed.

Thank you for the Audit Report of project agreement AT PL-5025(065) dated December 2018. The City of Santa Cruz does not dispute the two findings in the report but would like to provide this letter as additional context.

Finding 1 - Unreliable Estimation of Project Users.

A singe hourly count was performed for the ATP Cycle 2 grant application and although two or three counts are strongly encouraged, it is not a requirement of the National Bicycle and Pedestrian Documentation Project (NBPDP) model. The City acknowledges that the daily adjust factor (1.05 or 5%) step was accidently missed in project user calculation thus creating a relatively small error. The NBPDP now has a fillable spreadsheet that can be downloaded and performs the calculations for you, unfortunately at the time of the application this spreadsheet did not exist. For future ATP cycle guidelines it would be helpful to have specific and consistent count methodologies to use for the grant application. Additionally as noted in the Caltrans Active Transportation Program Report presented at the December 5-6, 2018 California Transportation Commission meeting, Caltrans will be attempting to improve and standardize counting methods, and offered the following as to potential reasons why numerous ATP projects did not see increases in after counts:

• Size of the projects - Only 11 of the 65 projects that submitted completed Project Completion Reports have a total project cost over $1 million, with three of those projects having a total project cost over $2 million. Some examples of these smaller projects include installing a HAWK pedestrian-activated crossing signal at an unsafe crossing location, filling in sidewalk gaps and striping bike lanes along a single corridor, improving pedestrian-scale safety lighting, and safe routes to school education programs.
• Inconsistent counting methodology - Since many of these completed projects are smaller in scope and scale, the before and after project counts submitted by their project sponsors demonstrate an inconsistent trend of mode shift. After project counts for some projects show a significant increase in bicycling and especially in walking, while after project counts for other projects indicate no change or a decrease in walking or bicycling. Part of the inconsistent trend in after project count data could be attributed to an inconsistency in count methodologies used for before and after project counts. Also, research has shown that travel behavior will change significantly when smaller projects are connected to larger bicycling and walking facility networks in the community.
• Reporting time horizon — These after counts were taken within one year of project completion. Getting an accurate count of new travel behavior resulting from a project requires ample time after the project is open to the public for users to adjust their travel behavior. It is customary to use a minimum of a two-year history to accurately measure mode shift changes and a minimum of a three-year history to accurately analyze safety improvement data. Therefore, additional time will be needed to accurately quantify and report the actual long-term benefits of a project.
• Other benefits not captured by user counts - These projects may also exhibit other program benefits that are not captured well in the Project Completion Report. For example, projects that primarily aim to increase safety for people walking and bicycling, such as safe routes to school projects, may not have a significant impact on mode shift but could be providing a significant, low-cost safety benefit.

Finding 2 — Late Submission of Semi-Annual Project Progress Report.

The City acknowledges the late submittal of the semi-annual report by two months. As the audit report states, the ATP Guidelines (and website) did not have any instructions, forms, or procedures in place to submit project progress reports. This has since been resolved now that there is an established submittal process with Caltrans standard forms.

The City appreciates the time and effort to perform the audit and will improve our pre project counts and submit reports on time for future cycles of the Active Transportation Program.Accordion text body will be here.