Overview

Apache Airflow 2.2, released on October 12, 2021, introduces custom timetables and deferrable operators for more flexible orchestration.

Main Features

Custom timetables

Custom Timetable objects replace cron expressions for complex scheduling: holidays, stock market calendars, or irregular intervals.

python 
from airflow import DAG
from airflow.operators.empty import EmptyOperator
from datetime import datetime

# DAG with custom timetable
with DAG(
    dag_id='daily_pipeline',
    start_date=datetime(2021, 10, 1),
    schedule='@daily',  # or custom timetable
    catchup=False,
) as dag:
    start = EmptyOperator(task_id='start')
    process = EmptyOperator(task_id='process')
    end = EmptyOperator(task_id='end')

    start >> process >> end

Deferrable operators

Deferrable operators free the worker while waiting for an external event (file, API, sensor), reducing cluster resource consumption.

python 
from airflow.sensors.base import BaseSensorOperator
from airflow.triggers.temporal import TimeDeltaTrigger
from datetime import timedelta

class DeferrableSensor(BaseSensorOperator):
    """Sensor that frees the worker while waiting."""

    def execute(self, context):
        if not self.condition_met():
            self.defer(
                trigger=TimeDeltaTrigger(timedelta(minutes=5)),
                method_name='check',
            )

    def check(self, context, event=None):
        if not self.condition_met():
            self.defer(
                trigger=TimeDeltaTrigger(timedelta(minutes=5)),
                method_name='check',
            )

    def condition_met(self):
        return True  # verification logic

Sources