Parameter-Efficient Fine Tuning(PEFT)

Parameter-Efficient Fine Tuning is a set of techniques used to fine-tune large pre-trained models e.g GPT, BERT without updating all parameters. Instead, PEFT methods update only a small subset of parameters. Thus reducing the computational and memory cost of fine tuning

PEFT makes LLM customization more accessible while creating outputs that are comparable to a traditional fine-tuned model.

Traditional fine-tuning makes adjustments to an LLM by further training the entire model. This requires intensive computational resources, data, and time.

Comparatively, PEFT only modifies a small portion of parameters within a model, making it generally more accessible for organizations without extensive resources.

Why PEFT?

Large models can have billions of parameters. Fine-tuning all of them is:

  • Expensive - in time and compute
  • Storage-heavy - you need to store a new copy for every use case
  • Risky - more prone to overfitting on small datasets
  • Catastrophic forgetting - happens when the model forgets the knowledge it’s already learned when provided with new training data.

PEFT helps by:

  • Faster training speed: When fewer parameters are updated, PEFT allows for quicker experimentation and iteration.
  • Resource-efficient: PEFT uses much less GPU memory than traditional fine-tuning and can run on consumer-grade hardware. This means you can train an LLM on a laptop rather than needing a dedicated server.
  • Ability to overcome PEFT helps models avoid catastrophic forgetting because it only updates a few parameters rather than the whole model.
  • Accessible: Teams and organizations with fewer computational resources can fine-tune models and still achieve a desirable result.

PEFT Methods

  1. Adapters
  • Small neural networks inserted between layers of the pre-trained model.
  • Only adapter layers are trained; base model remains frozen.
  1. LoRA (Low-Rank Adaptation)
  • Instead of updating weight matrices directly, it adds low-rank matrices whose product approximates the update.
  • Extremely memory-efficient and effective
  1. QLoRA (Quantized Low-Rank Adaptation)
  • Combines 4-bit quantization of the base model with LoRA to drastically reduce memory usage.
  • Enables fine-tuning large models (e.g., LLaMA 65B) on a single GPU by training only small adapter modules.
  1. DoRA (Weight-Decomposed Low-Rank Adaptation)
  • Improves on LoRA by decomposing and fine-tuning the direction of weight updates instead of the full delta.
  • Achieves better performance with the same or fewer trainable parameters, especially in low-data settings.
  1. Prefix Tuning
  • Adds trainable “prefix tokens” to the input of each layer.
  • The model is guided by these tokens without modifying its weights.
  1. Prompt Tuning / P-Tuning
  • Optimizes a continuous prompt (embedding vectors) rather than model weights.
  • Works well for language models conditioned on inputs.

Example

Imagine you’re using a 7B parameter model for sentiment analysis. With PEFT:

  • Load the 7B model, but don’t touch its weights (they're frozen).

  • Insert small trainable modules into key parts of the model:

    For LoRA: Tiny low-rank matrices (like 8x64 or 16x16) into attention or FFN layers.

    For Adapters: Mini neural nets (like bottlenecks) between layers.

Only these small new modules—10 million parameters—are trained.

Back to code!