Emergent Abilities

In depth

Capabilities that are absent in smaller language models but appear in larger ones, so they cannot be predicted by extrapolating from small-model performance. Wei et al. coined the term in 2022 to describe tasks where accuracy stays near random until a scale threshold and then jumps. A 2023 follow-up argued many of these jumps come from choosing all-or-nothing metrics, and that smoother metrics show gradual improvement instead.

Primary source: arxiv.org