Overview

Ray 2.5, released on June 15, 2023, improves Ray Data for large-scale distributed data processing.

Main Features

Improved Ray Data

Ray Data offers a high-level API for distributed data processing, with improved data source support and transformations.

python 
import ray

ray.init()

ds = ray.data.read_parquet('s3://bucket/data/')
ds = ds.filter(lambda row: row['score'] > 80)
ds = ds.map(lambda row: {
    **row,
    'category': 'excellent' if row['score'] > 90 else 'good'
})
print(ds.count())

Improved ML integration

Integration with ML frameworks (PyTorch, TensorFlow) is simplified for distributed data preprocessing.

python 
import ray

ds = ray.data.read_images('s3://bucket/images/')

def preprocess(batch):
    batch['image'] = [img / 255.0 for img in batch['image']]
    return batch

ds = ds.map_batches(preprocess)
print(f'Images: {ds.count()}')

Sources