Page 15 - AI Vol 2: Risks of AI

improve and are integrated into the lives of more
people, these perceptions may shift. Agencies
may experience the general public sentiment to
shift from “Why are you using AI given all of
its flaws” to “Why aren’t you using AI given its
benefits, particularly in terms of cost savings.”
For example, the public may question why
agency resources and staff time are being used for
a certain task when available AI systems could
significantly decrease the time and cost required
for completion.
AI will also impact day-to-day operations
of agencies, as the vendors they deal with
increasingly utilize these systems both in their
product offerings and in their own internal
operations. For example, a vendor of finance
software may utilize AI in an effort to provide
better financial planning strategies without
directly informing the client. The agency could
thus find itself faced with many of the problems
associated with AI information and decision-
making discussed above, but without the
opportunity to evaluate the risks. As an example
of how a vendor’s internal use could impact
RISKS OF AI | LOZANOSMITH.COM
AGENCIES SHOULD EXPECT THEY
WILL INCREASINGLY BE UNDER
SCRUTINY BY INDIVIDUALS AND
GROUPS LEVERAGING AI FOR
THEIR OWN PURPOSES.
agencies, a vendor could use AI to sneak an
unfavorable contract provision into an agreement
in such a way that the agency might not notice it
on a quick review.
There is also the potential for groups and
individuals to leverage AI systems to assess legal
vulnerabilities of public agencies. LLMs can
be used to analyze vast amounts of information
about public agencies, whether publicly available
on the internet or produced in response to CPRA
requests and compare it against databases
of laws and agency policies to identify legal
vulnerabilities. No agency is perfect and minor
legal missteps are not uncommon. AI systems
will be able to thoroughly scan for these issues
and identify concerns that likely would have
been overlooked by a non-expert human. The
purpose of this use could be fundamentally fair,
such as a disappointed bidder using AI to scan
the low bid for an agency contract in order to
identify grounds to submit a bid protest. Another
relatively innocuous example would be a non-
profit concerned with compliance in certain areas
of law (e.g., prevailing wage regulations) using
AI to identify potential instances of agency non-
compliance. However, this practice could be
“weaponized” by individuals or groups looking
to target a particular agency. Agencies should
VOLUME 2 | 15