Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[airflow]: extend removed names (AIR302) #14804

Merged
merged 10 commits into from
Dec 6, 2024

Conversation

Lee-W
Copy link
Contributor

@Lee-W Lee-W commented Dec 6, 2024

Summary

Airflow 3.0 removes various deprecated functions, members, modules, and other values. They have been deprecated in 2.x, but the removal causes incompatibilities that we want to detect. This PR deprecates the following names.
The full list of rules we will extend apache/airflow#44556

package

  • airflow.contrib.*

module

  • airflow.operators.subdag.*

class

  • airflow.sensors.external_task.ExternalTaskSensorLinkairflow.sensors.external_task.ExternalDagLin
  • airflow.operators.bash_operator.BashOperatorairflow.operators.bash.BashOperator
  • airflow.operators.branch_operator.BaseBranchOperatorairflow.operators.branch.BaseBranchOperator
  • airflow.operators.dummy.EmptyOperatorairflow.operators.empty.EmptyOperator
  • airflow.operators.dummy.DummyOperatorairflow.operators.empty.EmptyOperator
  • airflow.operators.dummy_operator.EmptyOperatorairflow.operators.empty.EmptyOperator
  • airflow.operators.dummy_operator.DummyOperatorairflow.operators.empty.EmptyOperator
  • airflow.operators.email_operator.EmailOperatorairflow.operators.email.EmailOperator
  • airflow.sensors.base_sensor_operator.BaseSensorOperatorairflow.sensors.base.BaseSensorOperator
  • airflow.sensors.date_time_sensor.DateTimeSensorairflow.sensors.date_time.DateTimeSensor
  • airflow.sensors.external_task_sensor.ExternalTaskMarkerairflow.sensors.external_task.ExternalTaskMarker
  • airflow.sensors.external_task_sensor.ExternalTaskSensorairflow.sensors.external_task.ExternalTaskSensor
  • airflow.sensors.external_task_sensor.ExternalTaskSensorLinkairflow.sensors.external_task.ExternalTaskSensorLink
  • airflow.sensors.time_delta_sensor.TimeDeltaSensorairflow.sensors.time_delta.TimeDeltaSensor

function

  • airflow.utils.decorators.apply_defaults
  • airflow.www.utils.get_sensitive_variables_fieldsairflow.utils.log.secrets_masker.get_sensitive_variables_fields
  • airflow.www.utils.should_hide_value_for_keyairflow.utils.log.secrets_masker.should_hide_value_for_key
  • airflow.configuration.getairflow.configuration.conf.get
  • airflow.configuration.getbooleanairflow.configuration.conf.getboolean
  • airflow.configuration.getfloatairflow.configuration.conf.getfloat
  • airflow.configuration.getintairflow.configuration.conf.getint
  • airflow.configuration.has_optionairflow.configuration.conf.has_option
  • airflow.configuration.remove_optionairflow.configuration.conf.remove_option
  • airflow.configuration.as_dictairflow.configuration.conf.as_dict
  • airflow.configuration.setairflow.configuration.conf.set
  • airflow.secrets.local_filesystem.load_connectionsairflow.secrets.local_filesystem.load_connections_dict
  • airflow.secrets.local_filesystem.get_connectionairflow.secrets.local_filesystem.load_connections_dict
  • airflow.utils.helpers.chainairflow.models.baseoperator.chain
  • airflow.utils.helpers.cross_downstreamairflow.models.baseoperator.cross_downstream

attribute

  • in airflow.utils.trigger_rule.TriggerRule
    • DUMMY
    • NONE_FAILED_OR_SKIPPED

constant / variable

  • airflow.PY\d\d

Test Plan

Copy link
Contributor

github-actions bot commented Dec 6, 2024

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+2 -0 violations, +0 -0 fixes in 1 projects; 54 projects unchanged)

apache/airflow (+2 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select ALL

+ dev/perf/dags/perf_dag_1.py:41:10: AIR302 `airflow.operators.bash_operator.BashOperator` is removed in Airflow 3.0; use `airflow.operators.bash.BashOperator` instead
+ dev/perf/dags/perf_dag_1.py:48:12: AIR302 `airflow.operators.bash_operator.BashOperator` is removed in Airflow 3.0; use `airflow.operators.bash.BashOperator` instead

Changes by rule (1 rules affected)

code total + violation - violation + fix - fix
AIR302 2 2 0 0 0

Lee-W added 7 commits December 6, 2024 16:15
…nfiguration"

* get
* getboolean
* getfloat
* getint
* has_option
* remove_option
* as_dict
* set
* `airflow.operators.bash_operator.BashOperator` → `airflow.operators.bash.BashOperator`
* `airflow.operators.branch_operator.BaseBranchOperator` → `airflow.operators.branch.BaseBranchOperator`
* `airflow.operators.dummy.EmptyOperator` → `airflow.operators.empty.EmptyOperator`
* `airflow.operators.dummy.DummyOperator` → `airflow.operators.empty.EmptyOperator`
* `airflow.operators.dummy_operator.EmptyOperator` → `airflow.operators.empty.EmptyOperator`
* `airflow.operators.dummy_operator.DummyOperator` → `airflow.operators.empty.EmptyOperator`
* `airflow.operators.email_operator.EmailOperator` → `airflow.operators.email.EmailOperator`
* `airflow.sensors.base_sensor_operator.BaseSensorOperator` → `airflow.sensors.base.BaseSensorOperator`
* `airflow.sensors.date_time_sensor.DateTimeSensor` → `airflow.sensors.date_time.DateTimeSensor`
* `airflow.sensors.external_task.ExternalTaskSensorLink` → `airflow.sensors.external_task.ExternalDagLin`
* `airflow.sensors.external_task_sensor.ExternalTaskMarker` → `airflow.sensors.external_task.ExternalTaskMarker`
* `airflow.sensors.external_task_sensor.ExternalTaskSensor` → `airflow.sensors.external_task.ExternalTaskSensor`
* `airflow.sensors.external_task_sensor.ExternalTaskSensorLink` → `airflow.sensors.external_task.ExternalTaskSensorLink`
* `airflow.sensors.time_delta_sensor.TimeDeltaSensor` → `airflow.sensors.time_delta.TimeDeltaSensor`
@Lee-W Lee-W force-pushed the extend-air-302-removed-names branch from 499d549 to 2fdd02e Compare December 6, 2024 08:16
Lee-W added 2 commits December 6, 2024 16:28
…PY39, airflow.PY310, airflow.PY311, airflow.PY312 was removed
@Lee-W Lee-W changed the title Extend AIR 302 removed names [airflow]: extend removed names (AIR302) Dec 6, 2024
@Lee-W Lee-W marked this pull request as ready for review December 6, 2024 09:16
Copy link
Member

@MichaReiser MichaReiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice

@MichaReiser MichaReiser added rule Implementing or modifying a lint rule preview Related to preview mode features labels Dec 6, 2024
@MichaReiser MichaReiser merged commit 39623f8 into astral-sh:main Dec 6, 2024
21 checks passed
@Lee-W Lee-W deleted the extend-air-302-removed-names branch December 18, 2024 00:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
preview Related to preview mode features rule Implementing or modifying a lint rule
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants