City leaders often test innovations through pilot programs. Even when pilots successfully achieve their stated goals, scaling them into full-time programs can be difficult.

Smart Cities Dive caught up with Jason Cooley, chief innovation officer for the city of Frisco, Texas; Garfield Swaby, vice president of information technology at the New York Public Library; and Julia Murphy, deputy chief sustainability officer for the city of San Antonio, at the annual Smart Cities Connect Conference and Expo near Washington, D.C., this week.

These three leaders shared their thoughts on the biggest challenges they have faced in advancing pilots to permanent programs and their solutions to overcoming those barriers.

What is the biggest challenge you have faced in advancing innovative projects from a pilot to a permanent program?

Jason Cooley: The biggest challenge is trying to identify the pilot that actually works in your city and determine its success, said Cooley. Pilot projects do not always meet their stated goals. 

“I think in some cases, we move forward with pilots that actually didn’t succeed in their mission and we move forward with it, and it’s a failure,” said Cooley. 

Another challenge is cost, he said. The city understands that it can launch a pilot, but when it comes time to implement a permanent program, there are cost constraints.

The last challenge is effectively communicating a new technology’s purpose and how it addresses a particular issue. This communication helps align internal and external expectations about the potential benefits of the technology, he said.

Garfield Swaby: In April, the New York Public Library launched a pilot testing Citizens Broadband Radio Service, or CBRS, as a means of bridging the digital divide, Swaby said. The program involved installing radio towers on the roofs of five of the library system’s 92 libraries and letting patrons pick up kits with Chromebook computers and devices that turn the radio signals into a Wi-Fi internet connection. Library officials are evaluating the pilot, which recently wrapped up, and considering its expansion. 

By testing the technology as a pilot program, Swaby said the quasi-city agency was able to “cut out a lot of red tape” and avoid a lengthy procurement process, allowing the pilot to get “all of our vendors on board that we wanted to be a part of that trial.”

The library’s goal was to evaluate the vendors during the pilot as well, potentially allowing them to skip a formal procurement process if they were to implement it at a larger scale. “We hope this evaluation serves as the due diligence required under the normal RFP process for a full-blown implementation,” he said. 

The library couldn’t undercut all the red tape, however. They still had to conduct legal and privacy-policy reviews to protect patrons’ information. That process will be even more pervasive when they implement the program long-term, said Swaby. 

Julia Murphy: In September 2021, the city of San Antonio launched a program that installed electric vehicle charging stations at some city-owned sites such as libraries, parks and parking garages as a way of encouraging their adoption among residents, said Murphy. A year later, the city is looking to expand the program with more chargers at more public facilities. 

While the program was not a pilot, it allowed city officials to test the approach in a controlled manner before expanding it later on, she said.

Pilot programs are a “great way to demonstrate what good can come out of the goal that we’re trying to achieve,” Murphy said. But finding partners to move it onto the next phase requires establishing the scope of work, addressing legal and procurement concerns and making it attractive to the private sector, she said.

What are some solutions to addressing those challenges?

Cooley: Everyone affected by the innovative product, including department directors and citizens, needs to be brought in at the very beginning, said Cooley. All information must be shared so they understand what to expect during the pilot and after it ends.

“We have to have clear expectations,” said Cooley. Projects fail when the expectations don’t correspond with what the vendor can deliver. 

“To move forward, I think we just have to take it in incremental steps,” he said. “If it works here, we expand the process; if it doesn’t, we tweak it and make sure we go forward the right way.”

Swaby: Swaby said he plans to recommend the library system conduct a phased expansion of its CBRS program. Under his proposal, library officials will expand the program to several other locations, allowing them to maintain the technology in-house. But, as they expand the program further, they will have to use managed service providers or come up with solutions allowing them to effectively operate as an internet service provider. 

The library system has never taken that approach, so library officials are thinking and learning about the technology and building good relationships with the vendor community.

Murphy: A challenge the city faces internally in implementing the EV charging station program is communicating to other city departments that the effort is citywide and an amenity that the public wants, said Murphy. 

“Once everybody understands, that they get on, board and everybody is really working together,” she said. The city’s utility company has also worked with the city and helped project managers connect to new and existing utility infrastructure.

“The private sector was instrumental in making this project successful,” Murphy said. “Their willingness to work with the city and its guidelines and rules was foundational to the success of the program.”

Externally, the challenge is making residents aware of the benefits that EVs can bring to the community, such as improving air quality and public health.