Overview

PyArrow 12.0, released on May 3, 2023, integrates the Acero engine for query execution and improves CSV features.

Main Features

Acero engine

The Acero execution engine enables running query plans on Arrow data with optimized filtering, joining, and aggregation operations.

python 
import pyarrow as pa
import pyarrow.compute as pc

# Create an Arrow table
table = pa.table({
    'name': ['Alice', 'Bob', 'Charlie', 'Diana'],
    'score': [85, 92, 78, 95],
    'group': ['A', 'B', 'A', 'B'],
})

# Filtering and aggregation via Acero
mask = pc.greater(table['score'], 80)
result = table.filter(mask)
print(result.to_pandas())

Improved CSV features

The CSV reader now supports incremental reading, automatic type detection, and new encoding options.

python 
import pyarrow.csv as pv

# CSV reading with type detection
table = pv.read_csv(
    'data.csv',
    convert_options=pv.ConvertOptions(
        auto_dict_encode=True,
        timestamp_parsers=['%Y-%m-%d', '%d/%m/%Y'],
    ),
)
print(table.schema)

Sources