feat(airflow): Update docker.io/apache/airflow Docker tag to v2.11.0 - #507
Open
plural-copilot[bot] wants to merge 1 commit into
Open
feat(airflow): Update docker.io/apache/airflow Docker tag to v2.11.0#507plural-copilot[bot] wants to merge 1 commit into
plural-copilot[bot] wants to merge 1 commit into
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
plural-copilot
Bot
force-pushed
the
renovate/docker.io-apache-airflow-2.x
branch
from
February 25, 2025 14:25
faf7b51 to
66c9c03
Compare
plural-copilot
Bot
force-pushed
the
renovate/docker.io-apache-airflow-2.x
branch
from
May 20, 2025 10:41
66c9c03 to
0414143
Compare
plural-copilot
Bot
changed the title
| @@ -1,4 +1,4 @@ | |||
| FROM docker.io/apache/airflow:2.8.2-python3.10 | |||
| FROM docker.io/apache/airflow:2.11.0-python3.10 | |||
Check notice
Code scanning / Trivy
No HEALTHCHECK defined Low
plural-copilot
Bot
force-pushed
the
renovate/docker.io-apache-airflow-2.x
branch
from
July 11, 2025 21:33
0414143 to
66536cb
Compare
Signed-off-by: plural-copilot[bot] <172153387+plural-copilot[bot]@users.noreply.github.com>
plural-copilot
Bot
force-pushed
the
renovate/docker.io-apache-airflow-2.x
branch
from
July 12, 2025 00:21
66536cb to
c876e4f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR contains the following updates:
2.8.2-python3.10->2.11.0-python3.10Release Notes
apache/airflow (docker.io/apache/airflow)
v2.11.0Compare Source
Significant Changes
^^^^^^^^^^^^^^^^^^^
DeltaTriggerTimetablefor trigger-based scheduling (#47074)"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
This change introduces DeltaTriggerTimetable, a new built-in timetable that complements the existing suite of
Airflow timetables by supporting delta-based trigger schedules without relying on data intervals.
Airflow currently has two major types of timetables:
- Data interval-based (e.g.,
CronDataIntervalTimetable,DeltaDataIntervalTimetable)- Trigger-based (e.g.,
CronTriggerTimetable)However, there was no equivalent trigger-based option for delta intervals like
timedelta(days=1).As a result, even simple schedules like
schedule=timedelta(days=1)were interpreted through a data intervallens—adding unnecessary complexity for users who don't care about upstream/downstream data dependencies.
This feature is backported to Airflow 2.11.0 to help users begin transitioning before upgrading to Airflow 3.0.
By flipping this config in 2.11, users can preview and adopt the new scheduling behavior in advance — minimizing surprises during upgrade.
Consistent timing metrics across all backends (#39908, #43966)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Previously, Airflow reported timing metrics in milliseconds for
StatsDbut in seconds for other backendssuch as
OpenTelemetryandDatadog. This inconsistency made it difficult to interpret or comparetiming metrics across systems.
Airflow 2.11 introduces a new config option:
[metrics] timer_unit_consistency(default:Falsein 2.11,Trueand dropped in Airflow 3.0).When enabled, all timing metrics are consistently reported in milliseconds, regardless of the backend.
This setting has become mandatory and always
Truein Airflow 3.0 (the config will be removed), soenabling it in 2.11 allows users to migrate early and avoid surprises during upgrade.
Ease migration to Airflow 3
"""""""""""""""""""""""""""
This release introduces several changes to help users prepare for upgrading to Airflow 3:
execution_datenow also include alogical_datefield. Airflow 3 dropsexecution_dateentirely in favor oflogical_date(#44283)airflow config lintandairflow config updatecommands in 2.11 to help audit and migrate configs for Airflow 3.0. (#45736, #50353, #46757)Python 3.8 support removed
""""""""""""""""""""""""""
Support for Python 3.8 has been removed, as it has reached end-of-life.
Airflow 2.11 requires Python 3.9, 3.10, 3.11, or 3.12.
New Features
""""""""""""
DeltaTriggerTimetable(#47074)airflow config updateandairflow config lintchanges to ease migration to Airflow 3 (#45736, #50353)Bug Fixes
"""""""""
ti.log_urltimestamp format from"%Y-%m-%dT%H:%M:%S%z"to"%Y-%m-%dT%H:%M:%S.%f%z"(#50306)airflow.cfgcontains a randomfernet_keyandsecret_key(#47755)rendered_map_indexvia internal api (#49057)TaskInstancePydanticintoTaskInstance(#48571)log_urlproperty onTaskInstancePydantic(Internal API) (#50560)TypeErrorwhen deserializing task withexecution_timeoutset toNone(#46822)check_query_existsreturns a bool (#46707)/xcom/listgot exception when applying filter on thevaluecolumn (#46053)Miscellaneous
"""""""""""""
logical_dateto models usingexecution_date(#44283)BaseOperatorLink.get_linksignature (#46448)Doc Only Changes
""""""""""""""""
airflow.cfgvariable (#48084)XComdocs to show examples of pushing multipleXComs(#46284, #47068)v2.10.5Compare Source
Significant Changes
^^^^^^^^^^^^^^^^^^^
Ensure teardown tasks are executed when DAG run is set to failed (#45530)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Previously when a DAG run was manually set to "failed" or to "success" state the terminal state was set to all tasks.
But this was a gap for cases when setup- and teardown tasks were defined: If teardown was used to clean-up infrastructure
or other resources, they were also skipped and thus resources could stay allocated.
As of now when setup tasks had been executed before and the DAG is manually set to "failed" or "success" then teardown
tasks are executed. Teardown tasks are skipped if the setup was also skipped.
As a side effect this means if the DAG contains teardown tasks, then the manual marking of DAG as "failed" or "success"
will need to keep the DAG in running state to ensure that teardown tasks will be scheduled. They would not be scheduled
if the DAG is directly set to "failed" or "success".
Bug Fixes
"""""""""
trigger_rule=TriggerRule.ALWAYSin a task-generated mapping within bare tasks (#44751)ONE_DONE) in a mapped task group (#44937)FileTaskHandleronly read from default executor (#46000)skip_ifandrun_ifdecorators before TaskFlow virtualenv tasks are run (#41832) (#45680)rendered_map_index(#45109) (#45122)max_form_parts,max_form_memory_size(#46243) (#45749)executesafeguard mechanism (#44646) (#46280)Miscellaneous
"""""""""""""
conffrom Task Context (#44993)v2.10.4Compare Source
Significant Changes
^^^^^^^^^^^^^^^^^^^
TaskInstance
priority_weightis capped in 32-bit signed integer ranges (#43611)"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Some database engines are limited to 32-bit integer values. As some users reported errors in
weight rolled-over to negative values, we decided to cap the value to the 32-bit integer. Even
if internally in python smaller or larger values to 64 bit are supported,
priority_weightiscapped and only storing values from -2147483648 to
2147483.Bug Fixes
^^^^^^^^^
trigger_rule="always"in a dynamic mapped task (#43810)trigger_rule=TriggerRule.ALWAYSin a task-generated mapping within bare tasks (#44751)Doc Only Changes
""""""""""""""""
Miscellaneous
"""""""""""""
v2.10.3Compare Source
Significant Changes
^^^^^^^^^^^^^^^^^^^
No significant changes.
Bug Fixes
"""""""""
stringifiedobjects to UI via xcom if pickling is active (#42388) (#42486)selectinloadinstead ofjoinedload(#40487) (#42351)TrySelectorfor Mapped Tasks in Logs and Details Grid Panel (#43566)scheduler_loop_duration(#42886) (#43544)Miscellaneous
"""""""""""""
dompurifyfrom 2.2.9 to 2.5.6 in /airflow/www (#42263) (#42270)4.5.2(#43309) (#43318)Doc Only Changes
""""""""""""""""
v2.10.2Compare Source
Significant Changes
^^^^^^^^^^^^^^^^^^^
No significant changes.
Bug Fixes
"""""""""
renderedTemplatesas keys to skipcamelCasing(#42206) (#42208)camelcasexcom entries (#42182) (#42187)Miscellaneous
"""""""""""""
0.2.4as it breaks our integration (#42101)LibCST(#42089)--treeflag fortasks listcli command (#41965)Doc Only Changes
""""""""""""""""
security_model.rstto clear unauthenticated endpoints exceptions (#42085)v2.10.1Compare Source
Significant Changes
^^^^^^^^^^^^^^^^^^^
No significant changes.
Bug Fixes
"""""""""
__name__(#41699)tojsonfilter to example_inlet_event_extra example dag (#41890)Miscellaneous
"""""""""""""
Doc Only Changes
""""""""""""""""
keycloak(#41791)v2.10.0Compare Source
Significant Changes
^^^^^^^^^^^^^^^^^^^
Scarf based telemetry: Airflow now collect telemetry data (#39510)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Airflow integrates Scarf to collect basic usage data during operation. Deployments can opt-out of data collection by
setting the
[usage_data_collection]enabledoption toFalse, or theSCARF_ANALYTICS=falseenvironment variable.Datasets no longer trigger inactive DAGs (#38891)
"""""""""""""""""""""""""""""""""""""""""""""""""
Previously, when a DAG is paused or removed, incoming dataset events would still
trigger it, and the DAG would run when it is unpaused or added back in a DAG
file. This has been changed; a DAG's dataset schedule can now only be satisfied
by events that occur when the DAG is active. While this is a breaking change,
the previous behavior is considered a bug.
The behavior of time-based scheduling is unchanged, including the timetable part
of
DatasetOrTimeSchedule.try_numberis no longer incremented during task execution (#39336)""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Previously, the try number (
try_number) was incremented at the beginning of task execution on the worker. This was problematic for many reasons.For one it meant that the try number was incremented when it was not supposed to, namely when resuming from reschedule or deferral. And it also resulted in
the try number being "wrong" when the task had not yet started. The workarounds for these two issues caused a lot of confusion.
Now, instead, the try number for a task run is determined at the time the task is scheduled, and does not change in flight, and it is never decremented.
So after the task runs, the observed try number remains the same as it was when the task was running; only when there is a "new try" will the try number be incremented again.
One consequence of this change is, if users were "manually" running tasks (e.g. by calling
ti.run()directly, or command lineairflow tasks run),try number will no longer be incremented. Airflow assumes that tasks are always run after being scheduled by the scheduler, so we do not regard this as a breaking change.
/logoutendpoint in FAB Auth Manager is now CSRF protected (#40145)"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
The
/logoutendpoint's method in FAB Auth Manager has been changed fromGETtoPOSTin all existingAuthViews (
AuthDBView,AuthLDAPView,AuthOAuthView,AuthOIDView,AuthRemoteUserView), andnow includes CSRF protection to enhance security and prevent unauthorized logouts.
OpenTelemetry Traces for Apache Airflow (#37948).
"""""""""""""""""""""""""""""""""""""""""""""""""
This new feature adds capability for Apache Airflow to emit 1) airflow system traces of scheduler,
triggerer, executor, processor 2) DAG run traces for deployed DAG runs in OpenTelemetry format. Previously, only metrics were supported which emitted metrics in OpenTelemetry.
This new feature will add richer data for users to use OpenTelemetry standard to emit and send their trace data to OTLP compatible endpoints.
Decorator for Task Flow
(@​skip_if, @​run_if)to make it simple to apply whether or not to skip a Task. (#41116)""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
This feature adds a decorator to make it simple to skip a Task.
Using Multiple Executors Concurrently (#40701)
""""""""""""""""""""""""""""""""""""""""""""""
Previously known as hybrid executors, this new feature allows Airflow to use multiple executors concurrently. DAGs, or even individual tasks, can be configured
to use a specific executor that suits its needs best. A single DAG can contain tasks all using different executors. Please see the Airflow documentation for
more details. Note: This feature is still experimental. See
documentation on Executor <https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/executor/index.html#using-multiple-executors-concurrently>_ for a more detailed description.New Features
""""""""""""
AIP-61 <https://github.com/apache/airflow/pulls?q=is%3Apr+label%3Aarea%3Ahybrid-executors+is%3Aclosed+milestone%3A%22Airflow+2.10.0%22>_)AIP-62 <https://github.com/apache/airflow/pulls?q=is%3Apr+is%3Amerged+label%3AAIP-62+milestone%3A%22Airflow+2.10.0%22>_)AIP-64 <https://github.com/apache/airflow/pulls?q=is%3Apr+is%3Amerged+label%3AAIP-64+milestone%3A%22Airflow+2.10.0%22>_)AIP-44 <https://github.com/apache/airflow/pulls?q=is%3Apr+label%3AAIP-44+milestone%3A%22Airflow+2.10.0%22+is%3Aclosed>_)accessorsto read dataset events defined as inlet (#39367)dag test(#40010)endDatein task instance tooltip. (#39547)accessorsto read dataset events defined as inlet (#39367, #39893)run_if&skip_ifdecorators (#41116)Improvements
""""""""""""
renderedjsoncomponent (#40964)get_extra_dejsonmethod with nested parameter which allows you to specify if you want the nested json as string to be also deserialized (#39811)__getattr__to task decorator stub (#39425)RemovedIn20Warninginairflow taskcommand (#39244)db migrateerror messages (#39268)suppress_and_warnwarning (#39263)declarative_basefromsqlalchemy.orminstead ofsqlalchemy.ext.declarative(#39134)on_task_instance_failedaccess to the error that caused the failure (#38155)output_processorparameter toBashProcessor(#40843)Bug Fixes
"""""""""
never_failin BaseSensor (#40915)start_date(#40878)external_task_group_idtoWorkflowTrigger(#39617)Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Enabled.
♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR has been generated by Renovate Bot.