Overview

Transformers 4.30 from Hugging Face, released on May 20, 2023, integrates PEFT for efficient fine-tuning and improves large language model support.

Main Features

PEFT integration

PEFT (Parameter-Efficient Fine-Tuning) integration enables fine-tuning massive models with LoRA, drastically reducing required memory.

python 
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import LoraConfig, get_peft_model

model = AutoModelForCausalLM.from_pretrained('gpt2')
tokenizer = AutoTokenizer.from_pretrained('gpt2')

# LoRA configuration for efficient fine-tuning
lora_config = LoraConfig(
    r=8, lora_alpha=32,
    target_modules=['c_attn'],
    lora_dropout=0.1,
)
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()
# trainable params: 0.1% of total

Improved LLM support

Large language model (LLM) support is improved with 4-bit/8-bit loading, generation with KV-cache, and conversation pipelines.

python 
from transformers import pipeline

# Text generation pipeline
generator = pipeline(
    'text-generation',
    model='gpt2',
    max_new_tokens=50,
)

result = generator('Python is a language')
print(result[0]['generated_text'])

Sources