The presentation, delivered by members of the City’s Digital Services team, shared insights from a recent pilot of a generative AI chatbot designed to help customers schedule the right type of permit appointment. Joining them were two partners from US Digital Response, a nonprofit agency that helps governments use technology to better serve their communities.
The presentation was titled “Building Better Access: Portland’s GenAI Pilot for Smarter Permitting Appointments” and was hosted by InnovateUS, an organization that trains public service professionals. The interactive workshop showed how Digital Services’ approach balances cutting-edge technology with human-centered design principles to improve an important civic service, one chatbot conversation at a time.
Wrong place, right time
In Portland’s complex permitting environment, residents too often struggle to book the right kind of 15-minute appointment. Normally, these appointments are for customers to get one-on-one support from a permitting specialist who has a specific area of expertise, like city planning, zoning, or commercial building code.
Customers in the community can get a lot of value out of these conversations because it gives them the knowledge to finish their projects more quickly and easily.
But because permitting can be so complex, and customers’ projects can be so particular, choosing the right appointment can sometimes get confusing. After all, customers are looking for an appointment because they don’t know where to start in the first place.
That confusion leads to misrouted appointments, staff time spent redirecting customers, and project delays that can stretch for weeks.
“Users could get so confused by the website that they would actually make multiple, different kinds of appointments, just to cover their bases,” explained Cristy Rowley, a user research fellow with USDR. “This meant that staff lost time rerouting customers, and some appointments went unused entirely, so that again wasted a lot of time.”
To solve this, the team set out to build a chatbot that could navigate complex information and point customers in the right direction. But to do this, they didn’t start with code—they started with people.
A human-centered approach to the machines
Before building the bot, the team conducted interviews with permitting technicians to understand the biggest pain points in the process. These conversations helped confirm the project’s scope and offered clearer insights into how and where things break down for customers. This, coupled with user research done in the course of a larger project to improve the permitting experience, gave the team important insights into what community members and staff expect of the process.
“Even though this project was focused on building and offering an AI chatbot, we saw that human-centered design is still key to making AI useful,” said Evan Bowers, designer and researcher for Digital Services.
Using these conversations with technicians, the team developed clear and structured information that would help the chatbot give the right answers. “If your content is confusing or conflicting or poorly structured,” Bowers said, “AI doesn't have a solid foundation to work from.”
Artificial intelligence, authentic information
With data from over 2,400 real interactions with the City’s permit help desk, the team used AI to generate about 200 “synthetic examples”—completely fictional questions based on real-life situations. Subject matter experts then classified each question with the correct appointment type.
Like blueprints in a construction project, the team could use these synthetic examples as a source of truth for building and testing the accuracy of the chatbot.
Armed with this structured knowledge, the team then developed a prototype of the chatbot using Google’s Dialogflow, an AI conversation tool. They then embedded the chatbot behind a login wall for internal testing by staff, with built-in feedback tools that allow experts to rate the bot’s responses to give feedback to the team.
Within the AI tool, Digital Services and USDR were then able to write, and rewrite, prompts that would help the AI answer questions accurately and helpfully.
“The key thing here is to have some place where anybody within the team could suggest edits to the prompt,” said Christopher Fan, the USDR product manager volunteer. “And luckily, because we had examples, and we can talk to our stakeholders, we can make edits. Prompts are only as good as the inputs you put into it.”
Evaluating progress in conversation
After Digital Services’ presentation, public service professionals from all levels of government posed many thoughtful questions about the team’s process, and more generally, how AI tools can best serve the public. Conversations like these serve as an important opportunity to advance ideas and debates about the responsible use of artificial intelligence in public sector work.
Though the pilot is still under development, the results are promising so far. By keeping the scope of the project to appointments, the team is learning how they could put AI in even more complex processes.
“It might sound hard to do at first,” said Fan, “but through these practices of scoping down, getting some problem examples, and just iterating, you could actually see improvement.”
In testing, the pilot improved booking accuracy and gave staff more confidence in both the chatbot and the process behind it. It also gave the team a reusable set of tools and practices for future AI projects, including prompt libraries, benchmarking methods, and evaluation techniques.
Next steps into the future
The next steps will be to start all over. While keeping their learnings and continuing to refine the overall process, the development team may plug in different AI tools to reduce costs and to have more control over how the service will function. While still in an early stage, the team sees the potential for AI to become a bigger part of customer conversations, in permitting and beyond.
Throughout the presentation, the message was clear: AI should never replace the human connection in public service. But it can amplify human connection, when it’s thoughtfully applied.
“Let’s all work together to make public services easier, one conversation at a time,” said Hilaire Brockmeyer, manager of Digital Services. “That conversation might be with a chatbot, it might be conversations you’re having with your subject matter experts, or involving community members as you’re thinking through how to use something like generative AI to solve problems for people who are seeking services from the government.”
These conversations will continue.
Watch: Digital Services and US Digital Response at InnovateUS