Why ChatGPT Is More Than Autocomplete
Author(s): GSO1 Originally published on Towards AI. Why ChatGPT Is More Than Autocomplete Figure by the author with assistance from Claude (Anthropic) Calling a large language model (LLM) like ChatGPT “autocomplete” is not exactly wrong, but it is deeply misleading. Most of …
The Flow of Attention
Author(s): GSO1 Originally published on Towards AI. The Flow of Attention Introduction Picture an input prompt to a large language model as a cloud of points in a high-dimensional vector space E, one point for each token. As the model processes the …
The Geometry of Attention: One Space, Two Operators
Author(s): GSO1 Originally published on Towards AI. How two operators in one space reveal what four projections hide The goal of single-headed attention is to enhance the contextual awareness of a fixed token, or query, based on content from other tokens. This …