What is catastrophic forgetting?

Catastrophic forgetting is when a model loses previously learned capabilities after being fine-tuned on new data. For example, fine-tuning a general model heavily on legal documents might make it forget how to write code. It can be mitigated by mixing some general data into the fine-tuning set, using lower learning rates, or using PEFT methods that preserve original weights.