Page 16 - AI Vol 1: Foundations of AI
P. 16
are greater numbers of specialized, fine-tuned, developers and trainers interact with the model,
and domain specific models to take advantage providing guidance on its responses which helps
of. Second, the open-source community has been the model learn the art of conversation, not just
particularly interested in developing smaller as a language tool, but as an interactive entity
models needing less computational power, such capable of assisting, advising, and entertaining.
that the models can be run locally on generally
available consumer hardware, such as the Mixtral In addition to learning from conversational
model which is capable of running on mid-range examples and feedback, LLM models are also
laptops. A locally run LLM eliminates many of guided by explicit system-level instructions.
the privacy concerns that exist with large closed- These instructions define the boundaries of the
source models which require you to transmit data model’s responses, ensuring they adhere to ethical
to their company servers and then may utilize that guidelines, avoid misinformation, and stay within
data to further train their models. A locally run the scope of appropriate and safe interactions.
model never transfers data outside of the user’s
system. Finally, open-source models continue to Fine-tuning is a process where a pre-trained
work even if the developer discontinues support. foundational model is further trained on a specific
This mitigates a concern that was recently data set to tailor its capabilities to particular needs
raised when OpenAI’s board of directors fired or domains. This could involve training the model
their CEO, threatening the future availability of on domain-specific information, to create an
ChatGPT and the GPT-4 model which could have expert domain-specific model, or on data from a
been a substantial problem for the companies particular company or agency such that the model
that built software utilizing OpenAI’s service or is able to perform specific tasks for the entity
entities which integrated OpenAI services into with greater precision. A key advantage of fine-
their operations. tuning is the ability to maintain the AI’s general
capabilities while enhancing its performance in
6. PRE-TRAINING, FINE-TUNING, AND specific areas. Unlike a model trained solely on
GUARDRAIL STRATEGIES domain-specific data, a fine-tuned model retains
As previously discussed, LLMs start with training its extensive base knowledge. This dual capability
on large and varied data sets enabling the model allows the model to provide specialized assistance
to understand and generate human-like text. while still being able to draw upon a broad range
Transitioning from a word prediction generator of general knowledge when needed.
to an AI assistant or chatbot involves teaching
the model the format and flow of conversations. Guardrails and other procedures are critical to
This is where pre-training evolves into a more ensure LLM systems are safe for deployment.
focused learning phase. The model is exposed to One common method is known as “red teaming”
examples of dialogues and question-answer pairs, wherein the developers and outside experts utilize
simulating real-world conversational patterns. An models to actively try to exploit weaknesses and
integral part of this phase is human feedback. AI produce unexpected behaviors by simulating
16 | VOLUME 1 FOUNDATIONS OF AI | LOZANOSMITH.COM
