Overview

Ray 2.9, released on February 15, 2024, improves autoscaling and simplifies distributed cluster deployment.

Main Features

Improved autoscaling

The autoscaler reacts faster to load spikes and handles scale-down better to reduce cloud costs.

python 
import ray

ray.init()  # autoscaling configured in cluster.yaml

@ray.remote
def process(x):
    return x ** 2

# Launches 1000 tasks, autoscaler adds nodes
futures = [process.remote(i) for i in range(1000)]
results = ray.get(futures)
print(f'Total: {sum(results)}')

Simplified Ray Serve

Ray Serve improves ML model deployment with more flexible routing and better monitoring.

python 
from ray import serve

@serve.deployment(num_replicas=2)
class Model:
    def __init__(self):
        self.model = ...  # load model

    async def __call__(self, request):
        data = await request.json()
        return {'prediction': 42}

serve.run(Model.bind())

Sources