Overview

Pandas 1.5, released on September 20, 2022, introduces experimental Copy-on-Write and improves the PyArrow backend.

Main Features

Experimental Copy-on-Write

Copy-on-Write mode defers data copying until an actual modification, improving memory performance.

python 
import pandas as pd

pd.set_option('mode.copy_on_write', True)

df = pd.DataFrame({'a': [1, 2, 3], 'b': [4, 5, 6]})
df2 = df[['a']]  # no immediate copy
df2.iloc[0] = 100  # copy triggered here only
print(df)   # original unchanged
print(df2)  # modified

PyArrow backend

The PyArrow data type backend offers better nullable type handling and improved string performance.

python 
import pandas as pd

# PyArrow backend for strings
df = pd.DataFrame({
    'name': pd.array(['Alice', 'Bob', None], dtype='string[pyarrow]'),
    'age': pd.array([30, 25, None], dtype='int64[pyarrow]'),
})
print(df.dtypes)
print(df)

Sources