Apache Airflow 2.4.0

New Features

• Add Data-aware Scheduling (https://github.com/apache/airflow/pulls?q=is%3Apr+is%3Amerged+label%3AAIP-48+milestone%3A%22Airflow+2.4.0%22)
• Add @task.short_circuit TaskFlow decorator (#25752)
• Make execution_date_or_run_id optional in tasks test command (#26114)
• Automatically register DAGs that are used in a context manager (#23592, #26398)
• Add option of sending DAG parser logs to stdout. (#25754)
• Support multiple DagProcessors parsing files from different locations. (#25935)
• Implement ExternalPythonOperator (#25780)
• Make execution_date optional for command dags test (#26111)
• Implement expand_kwargs() against a literal list (#25925)
• Add trigger rule tooltip (#26043)
• Add conf parameter to CLI for airflow dags test (#25900)
• Include scheduled slots in pools view (#26006)
• Add output property to MappedOperator (#25604)
• Add roles delete command to cli (#25854)
• Add Airflow specific warning classes (#25799)
• Add support for TaskGroup in ExternalTaskSensor (#24902)
• Add @task.kubernetes taskflow decorator (#25663)
• Add a way to import Airflow without side-effects (#25832)
• Let timetables control generated run_ids. (#25795)
• Allow per-timetable ordering override in grid view (#25633)
• Grid logs for mapped instances (#25610, #25621, #25611)
• Consolidate to one schedule param (#25410)
• DAG regex flag in backfill command (#23870)
• Adding support for owner links in the Dags view UI (#25280)
• Ability to clear a specific DAG Run's task instances via REST API (#23516)
• Possibility to document DAG with a separate markdown file (#25509)
• Add parsing context to DAG Parsing (#25161)
• Implement CronTriggerTimetable (#23662)
• Add option to mask sensitive data in UI configuration page (#25346)
• Create new databases from the ORM (#24156)
• Implement XComArg.zip(*xcom_args) (#25176)
• Introduce sla_miss metric (#23402)
• Implement map() semantic (#25085)
• Add override method to TaskGroupDecorator (#25160)
• Implement expand_kwargs() (#24989)
• Add parameter to turn off SQL query logging (#24570)
• Add DagWarning model, and a check for missing pools (#23317)
• Add Task Logs to Grid details panel (#24249)
• Added small health check server and endpoint in scheduler(#23905)
• Add built-in External Link for ExternalTaskMarker operator (#23964)
• Add default task retry delay config (#23861)
• Add clear DagRun endpoint. (#23451)
• Add support for timezone as string in cron interval timetable (#23279)
• Add auto-refresh to dags home page (#22900, #24770)

Improvements

• Add more weekday operator and sensor examples #26071 (#26098)
• Add subdir parameter to dags reserialize command (#26170)
• Update zombie message to be more descriptive (#26141)
• Only send an SlaCallbackRequest if the DAG is scheduled (#26089)
• Promote Operator.output more (#25617)
• Upgrade API files to typescript (#25098)
• Less hacky double-rendering prevention in mapped task (#25924)
• Improve Audit log (#25856)
• Remove mapped operator validation code (#25870)
• More DAG(schedule=...) improvements (#25648)
• Reduce operator_name dupe in serialized JSON (#25819)
• Make grid view group/mapped summary UI more consistent (#25723)
• Remove useless statement in task_group_to_grid (#25654)
• Add optional data interval to CronTriggerTimetable (#25503)
• Remove unused code in /grid endpoint (#25481)
• Add and document description fields (#25370)
• Improve Airflow logging for operator Jinja template processing (#25452)
• Update core example DAGs to use @task.branch decorator (#25242)
• Update DAG audit_log route (#25415)
• Change stdout and stderr access mode to append in commands (#25253)
• Remove getTasks from Grid view (#25359)
• Improve taskflow type hints with ParamSpec (#25173)
• Use tables in grid details panes (#25258)
• Explicitly list @dag arguments (#25044)
• More typing in SchedulerJob and TaskInstance (#24912)
• Patch getfqdn with more resilient version (#24981)
• Replace all NBSP characters by whitespaces (#24797)
• Re-serialize all DAGs on airflow db upgrade (#24518)
• Rework contract of try_adopt_task_instances method (#23188)
• Make expand() error vague so it's not misleading (#24018)
• Add enum validation for [webserver]analytics_tool (#24032)
• Add dttm searchable field in audit log (#23794)
• Allow more parameters to be piped through via execute_in_subprocess (#23286)
• Use func.count to count rows (#23657)
• Remove stale serialized dags (#22917)
• AIP45 Remove dag parsing in airflow run local (#21877)
• Add support for queued state in DagRun update endpoint. (#23481)
• Add fields to dagrun endpoint (#23440)
• Use sql_alchemy_conn for celery result backend when result_backend is not set (#24496)

Bug Fixes

• Have consistent types between the ORM and the migration files (#24044, #25869)
• Disallow any dag tags longer than 100 char (#25196)
• Add the dag_id to AirflowDagCycleException message (#26204)
• Properly build URL to retrieve logs independently from system (#26337)
• For worker log servers only bind to IPV6 when dual stack is available (#26222)
• Fix TaskInstance.task not defined before handle_failure (#26040)
• Undo secrets backend config caching (#26223)
• Fix faulty executor config serialization logic (#26191)
• Show DAGs and Datasets menu links based on role permission (#26183)
• Allow setting TaskGroup tooltip via function docstring (#26028)
• Fix RecursionError on graph view of a DAG with many tasks (#26175)
• Fix backfill occasional deadlocking (#26161)
• Fix DagRun.start_date not set during backfill with --reset-dagruns True (#26135)
• Use label instead of id for dynamic task labels in graph (#26108)
• Don't fail DagRun when leaf mapped_task is SKIPPED (#25995)
• Add group prefix to decorated mapped task (#26081)
• Fix UI flash when triggering with dup logical date (#26094)
• Fix Make items nullable for TaskInstance related endpoints to avoid API errors (#26076)
• Fix BranchDateTimeOperator to be timezone-awreness-insensitive (#25944)
• Fix legacy timetable schedule interval params (#25999)
• Fix response schema for list-mapped-task-instance (#25965)
• Properly check the existence of missing mapped TIs (#25788)
• Fix broken auto-refresh on grid view (#25950)
• Use per-timetable ordering in grid UI (#25880)
• Rewrite recursion when parsing DAG into iteration (#25898)
• Find cross-group tasks in iter_mapped_dependants (#25793)
• Fail task if mapping upstream fails (#25757)
• Support / in variable get endpoint (#25774)
• Use cfg default_wrap value for grid logs (#25731)
• Add origin request args when triggering a run (#25729)
• Operator name separate from class (#22834)
• Fix incorrect data interval alignment due to assumption on input time alignment (#22658)
• Return None if an XComArg fails to resolve (#25661)
• Correct json arg help in airflow variables set command (#25726)
• Added MySQL index hint to use ti_state on find_zombies query (#25725)
• Only excluded actually expanded fields from render (#25599)
• Grid, fix toast for axios errors (#25703)
• Fix UI redirect (#26409)
• Require dag_id arg for dags list-runs (#26357)
• Check for queued states for dags auto-refresh (#25695)
• Fix upgrade code for the dag_owner_attributes table (#25579)
• Add map index to task logs api (#25568)
• Ensure that zombie tasks for dags with errors get cleaned up (#25550)
• Make extra link work in UI (#25500)
• Sync up plugin API schema and definition (#25524)
• First/last names can be empty (#25476)
• Refactor DAG pages to be consistent (#25402)
• Check expand_kwargs() input type before unmapping (#25355)
• Filter XCOM by key when calculating map lengths (#24530)
• Fix ExternalTaskSensor not working with dynamic task (#25215)
• Added exception catching to send default email if template file raises any exception (#24943)
• Bring MappedOperator members in sync with BaseOperator (#24034)

Misc/Internal

• Add automatically generated ERD schema for the MetaData DB (#26217)
• Mark serialization functions as internal (#26193)
• Remove remaining deprecated classes and replace them with PEP562 (#26167)
• Move dag_edges and task_group_to_dict to corresponding util modules (#26212)
• Lazily import many modules to improve import speed (#24486, #26239)
• FIX Incorrect typing information (#26077)
• Add missing contrib classes to deprecated dictionaries (#26179)
• Re-configure/connect the ORM after forking to run a DAG processor (#26216)
• Remove cattrs from lineage processing. (#26134)
• Removed deprecated contrib files and replace them with PEP-562 getattr (#26153)
• Make BaseSerialization.serialize "public" to other classes. (#26142)
• Change the template to use human readable task_instance description (#25960)
• Bump moment-timezone from 0.5.34 to 0.5.35 in /airflow/www (#26080)
• Fix Flask deprecation warning (#25753)
• Add CamelCase to generated operations types (#25887)
• Fix migration issues and tighten the CI upgrade/downgrade test (#25869)
• Fix type annotations in SkipMixin (#25864)
• Workaround setuptools editable packages path issue (#25848)
• Bump undici from 5.8.0 to 5.9.1 in /airflow/www (#25801)
• Add custom_operator_name attr to _BranchPythonDecoratedOperator (#25783)
• Clarify filename_template deprecation message (#25749)
• Use ParamSpec to replace ... in Callable (#25658)
• Remove deprecated modules (#25543)
• Documentation on task mapping additions (#24489)
• Remove Smart Sensors (#25507)
• Fix elasticsearch test config to avoid warning on deprecated templa...

Apache Airflow 2.3.4

Bug Fixes

• Disable attrs state management on MappedOperator (#24772)
• Serialize pod_override to JSON before pickling executor_config (#24356)
• Fix pid check (#24636)
• Rotate session id during login (#25771)
• Fix mapped sensor with reschedule mode (#25594)
• Cache the custom secrets backend so the same instance gets re-used (#25556)
• Add right padding (#25554)
• Fix reducing mapped length of a mapped task at runtime after a clear (#25531)
• Fix airflow db reset when dangling tables exist (#25441)
• Change disable_verify_ssl behaviour (#25023)
• Set default task group in dag.add_task method (#25000)
• Removed interfering force of index. (#25404)
• Remove useless logging line (#25347)
• Adding mysql index hint to use index on task_instance.state in critical section query (#25673)
• Configurable umask to all daemonized processes. (#25664)
• Fix the errors raised when None is passed to template filters (#25593)
• Allow wildcarded CORS origins (#25553)
• Fix "This Session's transaction has been rolled back" (#25532)
• Fix Serialization error in TaskCallbackRequest (#25471)
• fix - resolve bash by absolute path (#25331)
• Add __repr__ to ParamsDict class (#25305)
• Only load distribution of a name once (#25296)
• convert TimeSensorAsync target_time to utc on call time (#25221)
• call updateNodeLabels after expandGroup (#25217)
• Stop SLA callbacks gazumping other callbacks and DOS'ing the DagProcessorManager queue (#25147)
• Fix invalidateQueries call (#25097)
• airflow/www/package.json: Add name, version fields. (#25065)
• No grid auto-refresh for backfill dag runs (#25042)
• Fix tag link on dag detail page (#24918)
• Fix zombie task handling with multiple schedulers (#24906)
• Bind log server on worker to IPv6 address (#24755) (#24846)
• Add %z for %(asctime)s to fix timezone for logs on UI (#24811)
• TriggerDagRunOperator.operator_extra_links is attr (#24676)
• Send DAG timeout callbacks to processor outside of prohibit_commit (#24366)
• Don't rely on current ORM structure for db clean command (#23574)
• Clear next method when clearing TIs (#23929)
• Two typing fixes (#25690)

Doc only changes

• Update set-up-database.rst (#24983)
• Fix syntax in mysql setup documentation (#24893 (#24939)
• Note how DAG policy works with default_args (#24804)
• Update PythonVirtualenvOperator Howto (#24782)
• Doc: Add hyperlinks to Github PRs for Release Notes (#24532)

Misc/Internal

• Remove depreciation warning when use default remote tasks logging handlers (#25764)
• clearer method name in scheduler_job.py (#23702)
• Bump cattrs version (#25689)
• Include missing mention of external_executor_id in sql_engine_collation_for_ids docs (#25197)
• Refactor DR.task_instance_scheduling_decisions (#24774)
• Sort operator extra links (#24992)
• Extends resolve_xcom_backend function level documentation (#24965)
• Upgrade FAB to 4.1.3 (#24884)
• Limit Flask to <2.3 in the wake of 2.2 breaking our tests (#25511)
• Limit astroid version to < 2.12 (#24982)
• Move javascript compilation to host (#25169)
• Bump typing-extensions and mypy for ParamSpec (#25088)

Apache Airflow 2.3.3

Bug Fixes

• Fix exception in mini task scheduler (#24865)
• Fix cycle bug with attaching label to task group (#24847)
• Fix timestamp defaults for sensorinstance (#24638)
• Move fallible ti.task.dag assignment back inside try/except block (#24533) (#24592)
• Add missing types to FSHook (#24470)
• Mask secrets in stdout for airflow tasks test (#24362)
• DebugExecutor use ti.run() instead of ti._run_raw_task (#24357)
• Fix bugs in URI constructor for MySQL connection (#24320)
• Missing scheduleinterval nullable true added in openapi (#24253)
• Unify return_code interface for task runner (#24093)
• Handle occasional deadlocks in trigger with retries (#24071)
• Remove special serde logic for mapped op_kwargs (#23860)
• ExternalTaskSensor respects soft_fail if the external task enters a failed_state (#23647)
• Fix StatD timing metric units (#21106)
• Add cache_ok flag to sqlalchemy TypeDecorators. (#24499)
• Allow for LOGGING_LEVEL=DEBUG (#23360)
• Fix grid date ticks (#24738, #24849)
• Debounce status highlighting in Grid view (#24710)
• Fix Grid vertical scrolling (#24684)
• don't try to render child rows for closed groups (#24637)
• Do not calculate grid root instances (#24528)
• Maintain grid view selection on filtering upstream (#23779)
• Speed up grid_data endpoint by 10x (#24284)
• Apply per-run log templates to log handlers (#24153)
• Don't crash scheduler if exec config has old k8s objects (#24117)
• TI.log_url fix for map_index (#24335)
• Fix migration 0080_2_0_2 - Replace null values before setting column not null (#24585)
• Patch sql_alchemy_conn if old Postgres schemes used (#24569)
• Seed log_template table (#24511)
• Fix deprecated log_id_template value (#24506)
• Fix toast messages (#24505)
• Add indexes for CASCADE deletes for task_instance (#24488)
• Return empty dict if Pod JSON encoding fails (#24478)
• Improve grid rendering performance with a custom tooltip (#24417, #24449)
• Check for run_id for grid group summaries (#24327)
• Optimize calendar view for cron scheduled DAGs (#24262)
• Use get_hostname instead of socket.getfqdn (#24260)
• Check that edge nodes actually exist (#24166)
• Fix useTasks crash on error (#24152)
• Do not fail re-queued TIs (#23846)
• Reduce grid view API calls (#24083)
• Rename Permissions to Permission Pairs. (#24065)
• Replace use_task_execution_date with use_task_logical_date (#23983)
• Grid fix details button truncated and small UI tweaks (#23934)
• Add TaskInstance State REMOVED to finished states and success states (#23797)
• Fix mapped task immutability after clear (#23667)
• Fix permission issue for dag that has dot in name (#23510)
• Fix closing connection dbapi.get_pandas_df (#23452)
• Check bag DAG schedule_interval match timetable (#23113)
• Parse error for task added to multiple groups (#23071)
• Fix flaky order of returned dag runs (#24405)
• Migrate jsx files that affect run/task selection to tsx (#24509)
• Fix links to sources for examples (#24386)
• Set proper Content-Type and chartset on grid_data endpoint (#24375)

Doc only changes

• Fix doc description of [core] parallelism config setting (#23768)
• Update templates doc to mention extras and format Airflow Vars / Conns (#24735)
• Document built in Timetables (#23099)
• Alphabetizes two tables (#23923)
• Clarify that users should not use Maria DB (#24556)
• Add imports to deferring code samples (#24544)
• Add note about image regeneration in June 2022 (#24524)
• Small cleanup of get_current_context() chapter (#24482)
• Fix default 2.2.5 log_id_template (#24455)
• Update description of installing providers separately from core (#24454)
• Mention context variables and logging (#24304)

Misc/Internal

• Do not support MSSQL less than v2017 in code (#24095)
• Drop Python 3.6 compatibility objects/modules (#24048)
• Remove upper-binding for SQLAlchemy (#24819)
• Remove internet explorer support (#24495)
• Removing magic status code numbers from api_connexion (#24050)
• Upgrade FAB to 4.1.2 (#24619)
• Switch Markdown engine to markdown-it-py (#19702)
• Update rich to latest version across the board. (#24186)
• Get rid of TimedJSONWebSignatureSerializer (#24519)
• Update flask-appbuilder authlib/ oauth dependency (#24516)
• Upgrade to webpack 5 (#24485)
• Add typescript (#24337)
• The JWT claims in the request to retrieve logs have been standardized: we use nbf and aud claims for
  maturity and audience of the requests. Also "filename" payload field is used to keep log name. (#24519)
• Address all yarn test warnings (#24722)
• Upgrade to react 18 and chakra 2 (#24430)
• Refactor DagRun.verify_integrity (#24114)
• Upgrade FAB to 4.1.1 (#24399)
• We now need at least Flask-WTF 0.15 (#24621)

Apache Airflow 2.3.2

Bug Fixes

• Run the check_migration loop at least once
• Fix grid view for mapped tasks (#24059)
• Icons in grid view for different DAG run types (#23970)
• Faster grid view (#23951)
• Disallow calling expand with no arguments (#23463)
• Add missing is_mapped field to Task response. (#23319)
• DagFileProcessorManager: Start a new process group only if current process not a session leader (#23872)
• Mask sensitive values for not-yet-running TIs (#23807)
• Add cascade to dag_tag to dag foreign key (#23444)
• Use --subdir argument value for standalone dag processor. (#23864)
• Highlight task states by hovering on legend row (#23678)
• Fix and speed up grid view (#23947)
• Prevent UI from crashing if grid task instances are null (#23939)
• Remove redundant register exit signals in dag-processor command (#23886)
• Add __wrapped__ property to _TaskDecorator (#23830)
• Fix UnboundLocalError when sql is empty list in DbApiHook (#23816)
• Enable clicking on DAG owner in autocomplete dropdown (#23804)
• Simplify flash message for _airflow_moved tables (#23635)
• Exclude missing tasks from the gantt view (#23627)

Doc only changes

• Add column names for DB Migration Reference (#23853)

Misc/Internal

• Remove pinning for xmltodict (#23992)

Apache Airflow 2.3.1

Bug Fixes

• Automatically reschedule stalled queued tasks in CeleryExecutor (#23690)
• Fix expand/collapse all buttons (#23590)
• Grid view status filters (#23392)
• Expand/collapse all groups (#23487)
• Fix retrieval of deprecated non-config values (#23723)
• Fix secrets rendered in UI when task is not executed. (#22754)
• Fix provider import error matching (#23825)
• Fix regression in ignoring symlinks (#23535)
• Fix dag-processor fetch metadata database config (#23575)
• Fix auto upstream dep when expanding non-templated field (#23771)
• Fix task log is not captured (#23684)
• Add reschedule to the serialized fields for the BaseSensorOperator (#23674)
• Modify db clean to also catch the ProgrammingError exception (#23699)
• Remove titles from link buttons (#23736)
• Fix grid details header text overlap (#23728)
• Ensure execution_timeout as timedelta (#23655)
• Don't run pre-migration checks for downgrade (#23634)
• Add index for event column in log table (#23625)
• Implement send_callback method for CeleryKubernetesExecutor and LocalKubernetesExecutor (#23617)
• Fix PythonVirtualenvOperator templated_fields (#23559)
• Apply specific ID collation to root_dag_id too (#23536)
• Prevent KubernetesJobWatcher getting stuck on resource too old (#23521)
• Fix scheduler crash when expanding with mapped task that returned none (#23486)
• Fix broken dagrun links when many runs start at the same time (#23462)
• Fix: Exception when parsing log #20966 (#23301)
• Handle invalid date parsing in webserver views. (#23161)
• Pools with negative open slots should not block other pools (#23143)
• Move around overflow, position and padding (#23044)
• Change approach to finding bad rows to LEFT OUTER JOIN. (#23528)
• Only count bad refs when moved table exists (#23491)
• Visually distinguish task group summary (#23488)
• Remove color change for highly nested groups (#23482)
• Optimize 2.3.0 pre-upgrade check queries (#23458)
• Add backward compatibility for core__sql_alchemy_conn__cmd (#23441)
• Fix literal cross product expansion (#23434)
• Fix broken task instance link in xcom list (#23367)
• Fix connection test button (#23345)
• fix cli airflow dags show for mapped operator (#23339)
• Hide some task instance attributes (#23338)
• Don't show grid actions if server would reject with permission denied (#23332)
• Use run_id for ti.mark_success_url (#23330)
• Fix update user auth stats (#23314)
• Use <Time /> in Mapped Instance table (#23313)
• Fix duplicated Kubernetes DeprecationWarnings (#23302)
• Store grid view selection in url params (#23290)
• Remove custom signal handling in Triggerer (#23274)
• Override pool for TaskInstance when pool is passed from cli. (#23258)
• Show warning if '/' is used in a DAG run ID (#23106)
• Use kubernetes queue in kubernetes hybrid executors (#23048)
• Add tags inside try block. (#21784)

Doc only changes

• Move dag_processing.processor_timeouts to counters section (#23393)
• Clarify that bundle extras should not be used for PyPi installs (#23697)
• Synchronize support for Postgres and K8S in docs (#23673)
• Replace DummyOperator references in docs (#23502)
• Add doc notes for keyword-only args for expand() and partial() (#23373)
• Document fix for broken elasticsearch logs with 2.3.0+ upgrade (#23821)

Misc/Internal

• Add typing for airflow/configuration.py (#23716)
• Disable Flower by default from docker-compose (#23685)
• Added postgres 14 to support versions(including breeze) (#23506)
• add K8S 1.24 support (#23637)
• Refactor code references from tree to grid (#23254)

Apache Airflow Helm Chart 1.6.0

New Features

• Support annotations on volumeClaimTemplates (#23433)
• Add support for topologySpreadConstraints to Helm Chart (#22712)
• Helm support for LocalKubernetesExecutor (#22388)
• Add securityContext config for Redis to Helm chart (#22182)
• Allow annotations on Helm DAG PVC (#22261)
• enable optional subPath for DAGs volume mount (#22323)
• Added support to override auth_type in auth_file in PgBouncer Helm configuration (#21999)
• Add extraVolumeMounts to Flower (#22414)
• Add webserver PodDisruptionBudget (#21735)

Improvements

• Ensure the messages from migration job show up early (#23479)
• Allow migration jobs and init containers to be optional (#22195)
• Use jobs check command for liveness probe check in Airflow 2 (#22143)

Doc only changes

• Adds resultBackendSecretName warning in Helm production docs (#23307)

Misc

• Update default Airflow version to 2.3.0 (#23386)
• Move the database configuration to a new section (#22284)
• Disable flower in chart by default (#23737)

Apache Airflow 2.3.0

New Features

• Add dynamic task mapping (https://github.com/apache/airflow/pulls?q=is%3Apr+is%3Amerged+label%3AAIP-42+milestone%3A%22Airflow+2.3.0%22)
• New Grid View replaces Tree View (#18675)
• Templated requirements.txt in Python Operators (#17349)
• Allow reuse of decorated tasks (#22941)
• Move the database configuration to a new section (#22284)
• Add SmoothOperator (#22813)
• Make operator's execution_timeout configurable (#22389)
• Events Timetable (#22332)
• Support dag serialization with custom ti_deps rules (#22698)
• Support log download in task log view (#22804)
• support for continue backfill on failures (#22697)
• Add dag-processor cli command (#22305)
• Add possibility to create users in LDAP mode (#22619)
• Add ignore_first_depends_on_past for scheduled jobs (#22491)
• Update base sensor operator to support XCOM return value (#20656)
• Add an option for run id in the ui trigger screen (#21851)
• Enable JSON serialization for connections (#19857)
• Add REST API endpoint for bulk update of DAGs (#19758)
• Add queue button to click-on-DagRun interface. (#21555)
• Add list-import-errors to airflow dags command (#22084)
• Store callbacks in database if standalone_dag_processor config is True. (#21731)
• Add LocalKubernetesExecutor (#19729)
• Add celery.task_timeout_error metric (#21602)
• Airflow db downgrade cli command (#21596)
• Add ALL_SKIPPED trigger rule (#21662)
• Add db clean CLI command for purging old data (#20838)
• Add celery_logging_level (#21506)
• Support different timeout value for dag file parsing (#21501)
• Support generating SQL script for upgrades (#20962)
• Add option to compress Serialized dag data (#21332)
• Branch python operator decorator (#20860)
• Add Audit Log View to Dag View (#20733)
• Add missing StatsD metric for failing SLA Callback notification (#20924)
• Add ShortCircuitOperator configurability for respecting downstream trigger rules (#20044)
• Allow using Markup in page title in Webserver (#20888)
• Add Listener Plugin API that tracks TaskInstance state changes (#20443)
• Add context var hook to inject more env vars (#20361)
• Add a button to set all tasks to skipped (#20455)
• Cleanup pending pods (#20438)
• Add config to warn public deployment exposure in UI (#18557)
• Log filename template records (#20165)
• Added windows extensions (#16110)
• Showing approximate time until next dag_run in Airflow (#20273)
• Extend config window on UI (#20052)
• Add show dag dependencies feature to CLI (#19985)
• Add cli command for 'airflow dags reserialize` (#19471)
• Add missing description field to Pool schema(REST API) (#19841)
• Introduce DagRun action to change state to queued. (#19353)
• Add DAG run details page (#19705)
• Add role export/import to cli tools (#18916)
• Adding dag_id_pattern parameter to the /dags endpoint (#18924)

Improvements

• Show schedule_interval/timetable description in UI (#16931)
• Added column duration to DAG runs view (#19482)
• Enable use of custom conn extra fields without prefix (#22607)
• Initialize finished counter at zero (#23080)
• Improve logging of optional provider features messages (#23037)
• Meaningful error message in resolve_template_files (#23027)
• Update ImportError items instead of deleting and recreating them (#22928)
• Add option --skip-init to db reset command (#22989)
• Support importing connections from files with ".yml" extension (#22872)
• Support glob syntax in .airflowignore files (#21392) (#22051)
• Hide pagination when data is a single page (#22963)
• Support for sorting DAGs in the web UI (#22671)
• Speed up has_access decorator by ~200ms (#22858)
• Add XComArg to lazy-imported list of Airflow module (#22862)
• Add more fields to REST API dags/dag_id/details endpoint (#22756)
• Don't show irrelevant/duplicated/"internal" Task attrs in UI (#22812)
• No need to load whole ti in current_state (#22764)
• Pickle dag exception string fix (#22760)
• Better verification of Localexecutor's parallelism option (#22711)
• log backfill exceptions to sentry (#22704)
• retry commit on MySQL deadlocks during backfill (#22696)
• Add more fields to REST API get DAG(dags/dag_id) endpoint (#22637)
• Use timetable to generate planned days for current year (#22055)
• Disable connection pool for celery worker (#22493)
• Make date picker label visible in trigger dag view (#22379)
• Expose try_number in airflow vars (#22297)
• Add generic connection type (#22310)
• Add a few more fields to the taskinstance finished log message (#22262)
• Pause auto-refresh if scheduler isn't running (#22151)
• Show DagModel details. (#21868)
• Add pip_install_options to PythonVirtualenvOperator (#22158)
• Show import error for airflow dags list CLI command (#21991)
• Pause auto-refresh when page is hidden (#21904)
• Default args type check (#21809)
• Enhance magic methods on XComArg for UX (#21882)
• py files don't have to be checked is_zipfiles in refresh_dag (#21926)
• Fix TaskDecorator type hints (#21881)
• Add 'Show record' option for variables (#21342)
• Use DB where possible for quicker airflow dag subcommands (#21793)
• REST API: add rendered fields in task instance. (#21741)
• Change the default auth backend to session (#21640)
• Don't check if py DAG files are zipped during parsing (#21538)
• Switch XCom implementation to use run_id (#20975)
• Action log on Browse Views (#21569)
• Implement multiple API auth backends (#21472)
• Change logging level details of connection info in get_connection() (#21162)
• Support mssql in airflow db shell (#21511)
• Support config worker_enable_remote_control for celery (#21507)
• Log memory usage in CgroupTaskRunner (#21481)
• Modernize DAG-related URL routes and rename "tree" to "grid" (#20730)
• Move Zombie detection to SchedulerJob (#21181)
• Improve speed to run airflow by 6x (#21438)
• Add more SQL template fields renderers (#21237)
• Simplify fab has access lookup (#19294)
• Log context only for default method (#21244)
• Log trigger status only if at least one is running (#21191)
• Add optional features in providers. (#21074)
• Better multiple_outputs inferral for @task.python (#20800)
• Improve handling of string type and non-attribute template_fields (#21054)
• Remove un-needed deps/version requirements (#20979)
• Correctly specify overloads for TaskFlow API for type-hinting (#20933)
• Introduce notification_sent to SlaMiss view (#20923)
• Rewrite the task decorator as a composition (#20868)
• Add "Greater/Smaller than or Equal" to filters in the browse views (#20602) (#20798)
• Rewrite DAG run retrieval in task command (#20737)
• Speed up creation of DagRun for large DAGs (5k+ tasks) by 25-130% (#20722)
• Make native environment Airflow-flavored like sandbox (#20704)
• Better error when param value has unexpected type (#20648)
• Add filter by state in DagRun REST API (List Dag Runs) (#20485)
• Prevent exponential memory growth in Tasks with custom logging handler (#20541)
• Set default logger in logging Mixin (#20355)
• Reduce deprecation warnings from www (#20378)
• Add hour and minute to time format on x-axis of all charts using nvd3.lineChart (#20002)
• Add specific warning when Task asks for more slots than pool defined with (#20178)
• UI: Update duration column for better human readability (#20112)
• Use Viewer role as example public role (#19215)
• Properly implement DAG param dict copying (#20216)
• ShortCircuitOperator push XCom by returning python_callable result (#20071)
• Add clear logging to tasks killed due to a Dagrun timeout (#19950)
• Change log level for Zombie detection messages (#20204)
• Better confirmation prompts (#20183)
• Only execute TIs of running DagRuns (#20182)
• Check and run migration in commands if necessary (#18439)
• Log only when Zombies exists (#20118)
• Increase length of the email and username (#19932)
• Add more filtering options for TI's in the UI (#19910)
• Dynamically enable "Test Connection" button by connection type (#19792)
• Avoid littering postgres server logs with "could not obtain lock" with HA schedulers (#19842)
• Renamed Connection.get_hook parameter to make it the same as in SqlSensor and SqlOperator. (#19849)
• Add hook_params in SqlSensor using the latest changes from PR #18718. (#18431)
• Speed up webserver boot time by delaying provider initialization (#19709)
• Configurable logging of XCOM value in PythonOperator (#19378)
• Minimize production js files (#19658)
• Add hook_params in BaseSqlOperator (#18718)
• Add missing "end_date" to hash components (#19281)
• More friendly output of the airflow plugins command + add timetables (#19298)
• Add sensor default timeout config (#19119)
• Update taskinstance REST API schema to include dag_run_id field (#19105)
• Adding feature in bash operator to append the user defined env variable to system env variable (#18944)
• Duplicate Connection: Added logic to query if a connection id exists before creating one (#18161)

Bug Fixes

• Use inherited 'trigger_tasks' method (#23016)
• In DAG dependency detector, use class type instead of class name (#21706)
• Fix tasks being wrongly skipped by schedule_after_task_execution (#23181)
• Fix X-Frame enabled behaviour (#23222)
• Allow extra to be nullable in connection payload as per schema(REST API). (#23183)
• Fix dag_id extraction for dag level access checks in web ui (#23015)
• Fix timezone display for logs on UI (#23075)
• Include message in graph errors (#23021)
• Change trigger dropdown left position (#23013)
• Don't add planned tasks for legacy DAG runs (#23007)
• Add dangling rows check for TaskInstance references (#22924)
• Validate the input params in connection CLI command (#22688)
• Fix trigger event payload ...

Apache Airflow 2.2.5

Bug Fixes

• Check and disallow a relative path for sqlite (#22530)
• Fixed dask executor and tests (#22027)
• Fix broken links to celery documentation (#22364)
• Fix incorrect data provided to tries & landing times charts (#21928)
• Fix assignment of unassigned triggers (#21770)
• Fix triggerer --capacity parameter (#21753)
• Fix graph autorefresh on page load (#21736)
• Fix filesystem sensor for directories (#21729)
• Fix stray order_by(TaskInstance.execution_date) (#21705)
• Correctly handle multiple '=' in LocalFileSystem secrets. (#21694)
• Log exception in local executor (#21667)
• Disable default_pool delete on web ui (#21658)
• Extends typing-extensions to be installed with python 3.8+ #21566 (#21567)
• Dispose unused connection pool (#21565)
• Fix logging JDBC SQL error when task fails (#21540)
• Filter out default configs when overrides exist. (#21539)
• Fix Resources __eq__ check (#21442)
• Fix max_active_runs=1 not scheduling runs when min_file_process_interval is high (#21413)
• Reduce DB load incurred by Stale DAG deactivation (#21399)
• Fix race condition between triggerer and scheduler (#21316)
• Fix trigger dag redirect from task instance log view (#21239)
• Log traceback in trigger excs (#21213)
• A trigger might use a connection; make sure we mask passwords (#21207)
• Update ExternalTaskSensorLink to handle templated external_dag_id (#21192)
• Ensure clear_task_instances sets valid run state (#21116)
• Fix: Update custom connection field processing (#20883)
• Truncate stack trace to DAG user code for exceptions raised during execution (#20731)
• Fix duplicate trigger creation race condition (#20699)
• Fix Tasks getting stuck in scheduled state (#19747)
• Fix: Do not render undefined graph edges (#19684)
• Set X-Frame-Options header to DENY only if X_FRAME_ENABLED is set to true. (#19491)

Doc only changes

• adding on_execute_callback to callbacks docs (#22362)
• Add documentation on specifying a DB schema. (#22347)
• Fix postgres part of pipeline example of tutorial (#21586)
• Extend documentation for states of DAGs & tasks and update trigger rules docs (#21382)
• DB upgrade is required when updating Airflow (#22061)
• Remove misleading MSSQL information from the docs (#21998)

Misc

• Add the new Airflow Trove Classifier to setup.cfg (#22241)
• Rename to_delete to to_cancel in TriggerRunner (#20658)
• Update Flask-AppBuilder to 3.4.5 (#22596)

Apache Airflow Helm Chart 1.5.0

New Features

• Add support for custom command and args in jobs (#20864)
• Support for priorityClassName (#20794)
• Add envFrom to the Flower deployment (#21401)
• Add annotations to cleanup pods (#21484)

Improvements

• Speedup liveness probe for scheduler and triggerer (#20833, #21108, #22041)
• Update git-sync to v3.4.0 (#21309)
• Remove default auth backend setting (#21640)

Bug Fixes

• Fix elasticsearch URL when username/password are empty (#21222)
• Mount airflow.cfg in wait-for-airflow-migrations containers (#20609)
• Grant pod log reader to triggerer serviceaccount (#21111)

Doc only changes

• Simplify chart docs for configuring Airflow (#21747)
• Add extra information about time synchronization needed (#21685)
• Fix extra containers docs (#20787)
• Fix example DAGs env var in quickstart (#21990)

Misc

• Use 2.2.4 as default Airflow version (#21745)
• Change Redis image to bullseye (#21875)

Apache Airflow 2.2.4

Bug Fixes

• Adding missing login provider related methods from Flask-Appbuilder (#21294)
• Fix slow DAG deletion due to missing dag_id index for job table (#20282)
• Add a session backend to store session data in the database (#21478)
• Show task status only for running dags or only for the last finished dag (#21352)
• Use compat data interval shim in log handlers (#21289)
• Fix mismatch in generated run_id and logical date of DAG run (#18707)
• Fix TriggerDagRunOperator extra link (#19410)
• Add possibility to create user in the Remote User mode (#19963)
• Avoid deadlock when rescheduling task (#21362)
• Fix the incorrect scheduling time for the first run of dag (#21011)
• Fix Scheduler crash when executing task instances of missing DAG (#20349)
• Deferred tasks does not cancel when DAG is marked fail (#20649)
• Removed duplicated dag_run join in Dag.get_task_instances() (#20591)
• Avoid unintentional data loss when deleting DAGs (#20758)
• Fix session usage in /rendered-k8s view (#21006)
• Fix airflow dags backfill --reset-dagruns errors when run twice (#21062)
• Do not set TaskInstance.max_tries in refresh_from_task (#21018)
• Don't require dag_id in body in dagrun REST API endpoint (#21024)
• Add Roles from Azure OAUTH Response in internal Security Manager (#20707)
• Allow Viewing DagRuns and TIs if a user has DAG "read" perms (#20663)
• Fix running airflow dags test <dag_id> <execution_dt> results in error when run twice (#21031)
• Switch to non-vendored latest connexion library (#20910)
• Bump flask-appbuilder to >=3.3.4 (#20628)
• upgrade celery to 5.2.3 (#19703)
• Bump croniter from <1.1 to <1.2 (#20489)
• Lift off upper bound for MarkupSafe (#20113)
• Avoid calling DAG.following_schedule() for TaskInstance.get_template_context() (#20486)
• Fix(standalone): Remove hardcoded Webserver port (#20429)
• Remove unnecssary logging in experimental API (#20356)
• Un-ignore DeprecationWarning (#20322)
• Deepcopying Kubernetes Secrets attributes causing issues (#20318)
• Fix(dag-dependencies): fix arrow styling (#20303)
• Adds retry on taskinstance retrieval lock (#20030)
• Correctly send timing metrics when using dogstatsd (fix schedule_delay metric) (#19973)
• Enhance multiple_outputs inference of dict typing (#19608)
• Fixing ses email backend (#18042)
• Pin Markupsafe until we are able to upgrade Flask/Jinja (#21664)

Doc only changes

• Added explaining concept of logical date in DAG run docs (#21433)
• Add note about Variable precedence with env vars (#21568)
• Update error docs to include before_send option (#21275)
• Augment xcom docs (#20755)
• Add documentation and release policy on "latest" constraints (#21093)
• Add a link to the DAG model in the Python API reference (#21060)
• Added an enum param example (#20841)
• Compare taskgroup and subdag (#20700)
• Add note about reserved params keyword (#20640)
• Improve documentation on Params (#20567)
• Fix typo in MySQL Database creation code (Set up DB docs) (#20102)
• Add requirements.txt description (#20048)
• Clean up default_args usage in docs (#19803)
• Add docker-compose explanation to conn localhost (#19076)
• Update CSV ingest code for tutorial (#18960)
• Adds Pendulum 1.x -> 2.x upgrade documentation (#18955)
• Updating explicit arg example in TaskFlow API tutorial doc (#18907)
• Adds back documentation about context usage in Python/@task (#18868)
• Clean up dynamic start_date values from docs (#19607)
• Docs for multiple pool slots (#20257)
• Update upgrading.rst with detailed code example of how to resolve post-upgrade warning (#19993)

Misc

• Deprecate some functions in the experimental API (#19931)
• Deprecate smart sensors (#20151)