Overview

Transformers 4.40, released on May 10, 2024, adds support for Phi-3, Microsoft's small language model.

Main Features

Phi-3 support

Phi-3 is natively integrated, enabling inference and fine-tuning of a compact high-performance model.

python 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    'microsoft/Phi-3-mini-4k-instruct',
    device_map='auto',
    torch_dtype='auto',
)
tokenizer = AutoTokenizer.from_pretrained(
    'microsoft/Phi-3-mini-4k-instruct'
)

inputs = tokenizer('Explain Python in one sentence.',
                   return_tensors='pt')
outputs = model.generate(**inputs, max_new_tokens=100)

Sources