Overview

Transformers 4.46, released on November 15, 2024, introduces the modular transformers architecture for easier model composition.

Main Features

Modular transformers

The modular approach allows composing models from reusable blocks, simplifying the creation of custom architectures.

python 
from transformers import AutoModel, AutoTokenizer

# Modular loading
tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')
model = AutoModel.from_pretrained('bert-base-uncased')

inputs = tokenizer('Modular transformers', return_tensors='pt')
outputs = model(**inputs)
print(f'Hidden states: {outputs.last_hidden_state.shape}')

Sources