Page 4 - AI Vol 2: Risks of AI
P. 4
raises questions about accountability and trust,
TESTING AI MODELS FOR BIAS particularly in scenarios where decisions have
INVOLVES ANALYZING HOW substantial impacts on individuals or communities.
THE SYSTEM MAKES DECISIONS Without transparency, it is difficult to ascertain
ACROSS DIFFERENT GROUPS whether decisions are fair, free from bias, or
AND SCENARIOS. even aligned with the agency's goals and legal
obligations. The lack of transparency exacerbates
the issue of identifying and addressing bias in AI
can deepen existing inequalities or create new systems. If the decision-making process is unclear,
forms of discrimination. Imagine an AI system it becomes challenging to determine whether a
designed to identify students needing additional biased reasoning pattern exists and, if so, what
academic support. If this system is trained on data is causing it. This is particularly problematic in
that inadvertently prioritizes certain indicators situations where decisions may be influenced
of performance – which may be influenced by by subtle forms of bias that are not immediately
socio-economic status, access to resources, or apparent.
other external factors – it might consistently
recommend more advanced resources for students For public agencies employing AI systems,
from more affluent backgrounds while relegating it is vital to be aware of the potential for bias
those from underprivileged backgrounds to an within these models and understand the general
academic intervention program. Over time, this approaches for mitigating it. One strategy is early
can widen the educational achievement gap, testing of AI models to identify potential biases.
as the AI's decisions reinforce and exacerbate Testing AI models for bias involves analyzing how
existing disparities. the system makes decisions across different groups
and scenarios. This process helps identify if the
The nature of AI, particularly in advanced and AI system is unfairly favoring or disadvantaging
complex systems, often involves a level of opacity certain groups. Since most public agencies may
that makes it challenging to understand how not have the technical capacity to test and correct
decisions are made, thus obscuring whether and AI models internally, it is advisable to engage
how biased reasoning might be influencing AI- with developers, vendors, or third-party auditors
driven decisions. Extremely complex computations who have the necessary expertise. Additionally,
make it difficult to trace how inputs are transformed agencies could require that entities they contract
into outputs. In simpler terms, these systems can with conduct thorough bias testing as part of their
become 'black boxes' where the reasoning behind service agreement. This can include periodic
a specific decision is not transparent. reviews and audits of the AI systems to identify
and address any emerging biases. Public agencies
For public agencies, the inability to fully can establish standards for the models they utilize,
understand or explain the decision-making process including requirements for transparency regarding
of AI systems poses significant challenges. It bias testing procedures and corrective measures.
4 | VOLUME 2 RISKS OF AI | LOZANOSMITH.COM
