Overview

Dask 2024.1, released on January 15, 2024, makes Dask Expressions the default backend, improving DataFrame performance.

Main Features

Dask Expressions default

The expression engine optimizes DataFrame queries before execution, reducing memory and computation time.

python 
import dask.dataframe as dd

# Dask Expr optimizes automatically
df = dd.read_parquet('data/*.parquet')
result = df[df['col'] > 0].groupby('cat').mean()

# Plan is optimized before execution
print(result.explain())  # show plan
print(result.compute())

Improved query API

The .query() method benefits from automatic optimization and column projections are pushed down to the read level.

python 
import dask.dataframe as dd

df = dd.read_parquet('sales/*.parquet')

# Optimized projection and filtering
result = (
    df[['product', 'amount']]
    .query('amount > 100')
    .groupby('product')
    .sum()
)
print(result.compute())

Sources