Why create an Evaluation and Investigative Services division in the Auditor’s Office?
It will capitalize on an opportunity to fill an information void by retaining analytical and investigative skills the City otherwise will lose when the Auditor’s Independent Police Review ceases operations on June 30, 2022. Evaluation and Investigative services will be an independent source of rigorous analysis and investigative expertise that will help policymakers, managers, and community members make government smarter and more accountable. The division serves the added purpose of incentivizing the employees to stay in their jobs and complete their police accountability work.
How is evaluation different than auditing?
Auditors and evaluators share an interest in more efficient, effective, and equitable government but evolved from different intellectual disciplines. Auditing grew out of accounting, and evaluation from the social sciences. Auditors focus on areas that could put government at risk of failure. They rely on interviews, documents, and datasets to determine whether programs and operations are well managed. Evaluators can have a broader scope, depending on the question stakeholders want answered. They use structured observations and larger volumes of data with an eye toward detecting statistically significant findings related to policies, program design, better processes, and the efficacy of interventions. In reality, the disciplines are complementary and borrow techniques from each other depending on the context and circumstances.
What value would the City and community get from evaluations?
The complexity of problems the City is expected to solve grows faster than the resources it has to solve them. Solutions require coordination and collaboration across economic sectors and government jurisdictions. Inaction or choosing the wrong intervention can be catastrophic for individuals, communities, neighborhoods, or the planet. Unlike auditing, evaluation requires ongoing stakeholder input and identification of data needs at the outset of an engagement. This makes evaluation better suited for cross-jurisdictional collaboration to ensure questions about policies and programs can be answered with sound evidence from an array of sources.
Are evaluations common in local government?
They are more common at the federal level. Evaluations of federal agencies were prioritized in the Obama administration’s 21st Century Management Agenda, which wanted to rigorously test programs and interventions that were most effective at achieving important goals and scaling up those that worked best. Offices of Inspectors General use multidisciplinary approaches to audits, evaluations, and investigations. The Idaho Legislature has the Office of Program Evaluations. That office has conducted studies that could replicated locally, such as the effect of tax preferences on revenue, the effect of state mandates on Portland government, or the City’s preparedness for its aging population generally and its workforce specifically.
What are some examples of types of evaluations the new division might conduct?
Any municipal matter with policy or programmatic questions could be well-served by evaluations. Possibilities include analyses of policy options under consideration by Council, analyses of systems the City shares with other governments, analyses of functions shared by bureaus, effects of individual bureau decisions on Citywide goals, evaluations of current programs or interventions, and equity analyses.
How would the evaluation topics be chosen?
We would adapt the process we already use to select audit topics and apply it to evaluation topics. The Charter makes the Auditor responsible for selecting audit topics. We solicit and encourage the public, employees, and elected officials to submit audit ideas year-round. Audit selection criteria emphasizes focusing our limited resources on high-risk areas. Evaluation selection would emerge from engagement on matters of concern to historically underserved communities, Council priorities, and issues that rely on collaboration with other governmental jurisdictions. The goal would be for evaluations to be timely, credible, relevant, and useful.
Why put Evaluation and Investigative Services in the Auditor’s Office?
The employees with analytical and investigative skills are already employed in the Auditor’s Office. Their responsibilities are narrowly focused on reviewing police policies and practices and investigating police misconduct complaints, but their skillsets are transferrable to broader subject areas. The Auditor’s independence from City management is the foundation of the office’s credibility. People trust our work because we don’t have an interest in any particular outcome, and that would apply to evaluations, too.
What types of investigations would be done?
The Auditor’s Office investigates fraud hotline tips, lobbyist reporting violations, campaign regulation violations, and community complaints about City administrative actions. Investigators could also contribute their skills in interviewing, observation, and writing to evaluations.
When would the division begin?
Two analysts would begin July 1, 2021, to engage the community, elected officials, and City staff as the division is established. They also will help draft City Code and administrative rules to formalize its authority. The investigators and managers would begin a year later as they wind down their commitments in Independent Police Review. Find more information on the transition on our website.
What if Council does not approve the proposal?
If the City does not incentivize Independent Police Review employees to stay and complete their police accountability work, it is likely they immediately will begin finding other jobs. That would jeopardize the City’s ability to regain substantial compliance with its settlement agreement with the U.S. Department of Justice and meet its obligation to the public to provide civilian police oversight. The City would also lose institutional knowledge and skills it could reassign to build capacity for policy and program evaluation.