Page 13 - AI Vol 1: Foundations of AI

another intriguing example from the same
paper, researchers showed that GPT-4 was able
to suggest a stable way to stack a set of objects
provided by the researchers (whereas GPT-3 was

not), suggesting the model was able to utilize
“common sense” if not a deeper understanding
of objects and how they behave in the physical
world. As previously mentioned, while these are
intriguing case studies, it is a matter of debate
whether these models are actually demonstrating
these capabilities, or just have become very good
at simulating them.

In practice, LLMs have demonstrated abilities
akin to human reasoning, creativity, problem
solving, and the ability to synthesize large
amounts of information from diverse sources.
These are all powerful capabilities that hold the
promise of improving agency functions, while 4. MODEL TYPES
also presenting new challenges as LLMs become It is useful to understand the growing diversity

more widely distributed among the public. of LLM model types, both because it provides
Emergent properties may manifest themselves a better understanding of how these systems
in ways that developers cannot predict, largely function generally, and because it will provide
owing to the complex and opaque nature of how public agency officials with the basic knowledge
these abilities are developed in the first place, needed to evaluate LLM based products. We
presenting risks that are difficult to anticipate. will provide a brief overview of the three most
Originally, emergent properties were associated prominent LLM model types: foundational
with large-scale LLMs, with capabilities in larger models, domain-specific models, and multi-
models absent in smaller counterparts. However, expert models.

recent developments indicate the possibility of
emergent behaviors in smaller models, expanding Foundational models are versatile, large-scale
the scope of research and potential applications. models designed to handle a broad spectrum of
Accordingly, we are likely to see more advanced tasks. Examples include GPT-3, GPT-4, Claude2,
models with new and more capable emergent and Google’s Gemini models. These models are
properties as model sizes increase and further trained on a large and varied data set, providing
technical development can produce these abilities them with generalist capabilities and making them

even at smaller scales. adaptable for a variety of use cases. However,

FOUNDATIONS OF AI | LOZANOSMITH.COM VOLUME 1 | 13