What is positional encoding and why is it needed?

Positional encoding adds information about token position to embeddings, because self-attention by itself is order-agnostic and treats input as a set rather than a sequence. Without position information, the model could not distinguish "dog bites man" from "man bites dog". Modern LLMs use rotary position embeddings, which encode position through rotation in vector space.