Overview

Airflow 2.9, released on April 9, 2024, introduces multi-asset scheduling for data pipelines.

Main Features

Multi-asset scheduling

DAGs can now be triggered by multiple datasets, enabling reactive data-driven pipelines.

python 
from airflow.datasets import Dataset
from airflow.decorators import dag, task
from datetime import datetime

sales = Dataset('s3://bucket/sales')
inventory = Dataset('s3://bucket/inventory')

@dag(schedule=[sales, inventory],
     start_date=datetime(2024, 1, 1))
def report_pipeline():
    @task
    def build_report():
        print('Both datasets updated')

    build_report()

report_pipeline()

Sources