What is in-context learning?

In-context learning is the LLM's ability to learn a task from examples provided in the prompt, without any weight updates. The term comes from the GPT-3 paper, which showed that large models can pick up new tasks from a handful of in-context examples. In-context learning is what makes few-shot prompting possible and is the foundation of most prompt engineering techniques.