Overview

Apache Airflow 2.7, released on October 13, 2023, introduces setup and teardown tasks for managing resource lifecycle in DAGs.

Main Features

The setup and teardown tasks guarantee resource allocation and cleanup, even when intermediate tasks fail.

python 
from airflow.decorators import dag, task, setup, teardown
from datetime import datetime

@dag(schedule=None, start_date=datetime(2023, 1, 1))
def my_dag():
    @setup
    @task
    def create_cluster():
        return 'cluster-id'

    @teardown
    @task
    def delete_cluster(cluster_id):
        pass

    @task
    def process(cluster_id):
        pass

    cid = create_cluster()
    process(cid) >> delete_cluster(cid)

Sources