Page 11 - AI Vol 2: Risks of AI

results, whereas some tasks are just too complex
for the current AI system to provide effective
assistance. Continually monitoring and evaluating
AI performance ensures that the agency can adapt
their use of AI systems (or evaluate whether other
systems are more appropriate). In addition to the
general liability risks discussed in this document,
an agency which deploys but fails to appropriately
monitor an AI system could be found liable under
a negligence theory. This liability may arise if
the AI model generates harmful or inappropriate
responses, whether directed towards a member of
the public, a student, or an employee relying on the
AI system for decision-making. Such liability could
be attributed to the agency's failure to oversee the
deployment adequately, especially if the potential
risks were foreseeable with proper monitoring.
Related issues include overreliance and the
subsequent degradation of human skills.
Overreliance refers to situations where AI systems
are excessively depended upon for decision-
making, potentially leading to diminished human
engagement and a failure to critically assess
AI outputs. This reliance can result in a lack of
questioning of AI-generated results, allowing
unchallenged errors or biases to influence
decisions. Moreover, an important concern
that arises with the overuse of AI is the risk of
skill degradation among staff. When employees
rely heavily on AI tools, they may experience
a decline in critical skills, particularly in areas
requiring complex analysis and decision-making.
This degradation not only diminishes individual
capabilities but also impacts the overall resilience
and adaptability of the agency. These issues are
very related, as overreliance can lead to skill
degradation, and skill degradation can lead to
RISKS OF AI | LOZANOSMITH.COM
WHEN EMPLOYEES RELY
HEAVILY ON AI TOOLS, THEY
MAY EXPERIENCE A DECLINE IN
CRITICAL SKILLS, PARTICULARLY
IN AREAS REQUIRING COMPLEX
ANALYSIS AND DECISION-MAKING.
overreliance. In addition to this situation leading
to poor decision making and performance of the
public agency, it may also lead to legal liability
to the extent that the agency implements flawed,
biased, or legally non-compliant decisions based
on AI advice. It is thus critical for agencies to
implement protocols to guard against these risks.
AI system stability and long term sustainability
is also important to consider, particularly in the
context of overreliance and skill degradation.
While still in the infancy of widespread AI
development and deployment, there are a large
number of companies attempting to build and
develop businesses and AI products, some of
which may not remain viable over time. Over
the Thanksgiving break in 2023, OpenAI saw
its CEO fired and the vast majority of its staff
threatened to quit in protest, creating concern
among the business built upon or simply relying
on OpenAI’s services. While that situation was
ultimately resolved without interruption of
OpenAI’s services, the episode illustrates that
agencies need to think critically regarding the
extent they intend to rely on any one AI service
and have backup plans in the event a service is
no longer available. Boardroom conflicts are just
one of the potential events that could lead to the
discontinuation of AI services. AI models are
trained on vast amounts of information primarily
VOLUME 2 | 11