Page 7 - AI Vol 2: Risks of AI
P. 7
01 B. PRIVACY
REAL WORLD USERS PROVIDING
AI SYSTEMS WITH FEEDBACK IS
Like any software application or cloud service, VERY VALUABLE FOR FUTURE AI
users must be able to trust and rely on the DEVELOPMENT.
companies they are entrusting with sensitive
information and ensure there are adequate data
protections. This concern is particularly acute in it was trained on when faced with similar queries
the context of AI and LLMs, as user inputs and from different users.
responses may be used to train current or future
models. One of the ways that AI models advance Imagine a public agency implementing an AI-
is through reinforcement learning, discussed in powered chatbot to assist citizens with inquiries.
more detail above, whereby the model generates An employee engages with the chatbot to input
one or several responses and a human user diverse citizen questions and receive appropriate
rates the output. The model incorporates human responses. Although the chatbot does not retain
feedback and adjusts the model to perform closer the exact questions or answers, if the employee
to the desired result in the future. User interactions identifies a unique citizen concern and seeks
with LLMs and other AI systems can be used to clarification, the chatbot may recognize similar
provide some of this reinforcement learning, as issues in subsequent interactions and offer
users can either directly provide feedback on a tailored assistance.
model’s response to a prompt (e.g., a “thumbs
up” or “thumbs down” in the ChatGPT interface) While LLMs are generally trained to learn the
or the AI system administrator may be able to patterns of the data they are trained on, rather than
discern whether the user was satisfied with the to “memorize” exact copies, there are instances
response based on subsequent interactions (e.g., when this can happen. LLMs memorizing their
the user thanks the model for the help, or has training data is called “overfitting” and is a problem
to repeatedly clarify their questions and prompt developers guard against as it makes the models
to get to a desired output). Real world users less useful. However, in rare circumstances the
providing this kind of feedback is very valuable models may memorize portions of their training
for future AI development. data and reproduce it in an output to a user request.
One study found that an attacker could cause
As discussed in Volume 1, LLMs typically do not ChatGPT and other LLMs to reproduce verbatim
store information in a database or memorize the training data in an output, including personally
data they're trained on. Instead, they learn from identifiable information. The New York Times
patterns within the data. So, while a model trained used ChatGPT outputs which included verbatim
on user inputs shouldn't directly reproduce those text from New York Times articles as part of the
prompts for other users, it can still understand basis of their copyright lawsuit against OpenAI
and replicate the underlying information. In other and Microsoft. While the risk of information
words, it may generate responses similar to those contained in user inputs being outputted to other
RISKS OF AI | LOZANOSMITH.COM VOLUME 2 | 7
