What Parrots? 🦜

Released in 2021 and leading to the dismissal of one of its authors from Google, "On the Dangers of Stochastic Parrots" is a formative paper in the field of AI. Written by Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and (Sh)margaret (Sch)mitchell, the paper presented a critique of GPTs at the dawning of its age. Its title has become shorthand for those denigrating the latest models and a source of angst for boomers of AI technology. What follows are my notes on the article.

Following an introduction and background focused on GenAI development, the authors outline their major arguments against generative AI:

(1) environmental risk - Research "estimate[s] that an increase in 0.1 BLEU score using neural architecture search for English to German translation results n an increase of $150,000 compute cost in addition to the carbon emissions" (612).

(2) the scale of training data - The data drawn is biased: "from initial participation in Internet fora, to continued presence there, to the collection and finally the filtering of training data, current practice privileges the hegemonic viewpoint" (614).

(3) the inherent limitations of LMs and the associated meaning-ization of LLM responses - "The training data for LMs is only form; they do not have access to meaning" (615) this leads to an inherent "teaching to the test" aspect of constructing models. Here arises the philosophical issues of what language and understanding are. The paper argues that "coherence is in fact in the eye of the beholder" (616) and human communication is "a jointly-constructed activity" (ibid). LMs - by their nature - do not have a grounded intent, model of the world, or model of the interlocutor, as such they cannot truly communicate. Instead, humans construct the meaning on their side for the ungrounded model, which "stitches" together a response using probability, alignment, training, and its dataset.

Given the authors' (1) and (2) concerns and their understanding of the construction (3) of models, there arises a series of possible harms: replicating and promoting the biases, allowing the dissemination of bias in their training data, manipulation by malicious actors to generate meaning-seeming texts and content, and the hallucination problem (assured and seemingly meaningful content from a model) going unchecked.

Recommendations focus on the development of smaller models with specific datasets, careful planning and premortems, and value-sensitive design. Of course, since 2021, the AI development world has followed an "All You Need is Scale" route, ignoring such recommendations to seek AGI. (Others have criticized this since 2021).

The piece essentially serves as another sign that a philosophical conversation on the nature of meaning, language, and the mind is needed to truly understand the AI moment.