Overview

Hugging Face Transformers 4.10, released on July 21, 2021, improves the Trainer and introduces quantization support to reduce model sizes.

Main Features

Trainer improvements

The Trainer gains new callback options, better logging, and native mixed-precision training support via fp16.

python 
from transformers import Trainer, TrainingArguments

# Trainer configuration with mixed precision
args = TrainingArguments(
    output_dir='./results',
    num_train_epochs=3,
    per_device_train_batch_size=16,
    fp16=True,  # automatic mixed precision
    logging_steps=100,
    evaluation_strategy='epoch',
)

# trainer = Trainer(
#     model=model,
#     args=args,
#     train_dataset=train_ds,
#     eval_dataset=eval_ds,
# )
# trainer.train()

Quantization

Dynamic quantization support reduces model sizes by 2 to 4 times with minimal accuracy loss, making deployment on resource-constrained devices easier.

python 
from transformers import AutoModelForSequenceClassification
import torch

# Load a model
model = AutoModelForSequenceClassification.from_pretrained(
    'distilbert-base-uncased'
)

# Dynamic quantization
quantized = torch.quantization.quantize_dynamic(
    model, {torch.nn.Linear}, dtype=torch.qint8
)

# Size comparison
import os
torch.save(model.state_dict(), '/tmp/original.pt')
torch.save(quantized.state_dict(), '/tmp/quantized.pt')
print(f'Original:  {os.path.getsize("/tmp/original.pt") / 1e6:.1f} MB')
print(f'Quantized: {os.path.getsize("/tmp/quantized.pt") / 1e6:.1f} MB')

Sources