Base Model

In depth

A language model that has only been pretrained to predict the next token on a large text corpus, with no instruction tuning or preference training on top. It has broad knowledge but tends to continue whatever text you give it rather than follow a request, so prompts work best as documents to complete or as few-shot examples. Labs often release a base checkpoint next to a chat version, as Meta did with Llama 2 and Llama 2-Chat, and base models are the usual starting point for fine-tuning and RL.

Primary source: arxiv.org