Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[airflow]: extend removed names (AIR302) #14734

Merged

Conversation

Lee-W
Copy link
Contributor

@Lee-W Lee-W commented Dec 2, 2024

Summary

Airflow 3.0 removes various deprecated functions, members, modules, and other values. They have been deprecated in 2.x, but the removal causes incompatibilities that we want to detect. This PR deprecates the following names.

  • airflow.metrics.validators.AllowListValidator
  • airflow.metrics.validators.BlockListValidator
  • airflow.utils.dates.parse_execution_date
  • airflow.utils.dates.round_time
  • airflow.utils.dates.scale_time_units
  • airflow.utils.dates.infer_time_unit
  • airflow.utils.file.TemporaryDirectory
  • airflow.utils.file.mkdirs
  • airflow.www.auth.has_access
  • airflow.api_connexion.security.requires_access
  • airflow.utils.dag_cycle_tester.test_cycle
  • airflow.utils.state.SHUTDOWN
  • airflow.utils.state.terminating_states

The full list of names we will extend apache/airflow#44556

Test Plan

A test fixture is included in the PR.

Copy link
Contributor

github-actions bot commented Dec 2, 2024

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+1 -1 violations, +0 -0 fixes in 1 projects; 54 projects unchanged)

apache/airflow (+1 -1 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select ALL

+ performance/src/performance_dags/performance_dag/performance_dag.py:244:9: AIR302 `schedule_interval` is removed in Airflow 3.0; use `schedule` instead
- performance/src/performance_dags/performance_dag/performance_dag.py:244:9: AIR302 `schedule_interval` is removed in Airflow 3.0; use schedule instead

Changes by rule (1 rules affected)

code total + violation - violation + fix - fix
AIR302 2 1 1 0 0

Comment on lines 118 to 120
["airflow", "utils", "decorators", "apply_defaults"] => {
Some((qualname.to_string(), Replacement::None))
}
Copy link
Contributor

@uranusjr uranusjr Dec 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should mention this can be safely removed (apply_defaults is now unconditionally done). It’s probably easier if we add a variant to Replacement (called Message maybe) and do

Replacement::Message(message) => {
    format!("`{deprecated}` is removed in Airflow 3.0; {message}")
}

in impl Violation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, we'll need something similar for AIR303 and AIR304 as well. or we could probably use this for those 2 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is temporarily removed, but will be added back (probably in the next PR I think?)

* airflow.metrics.validators.AllowListValidator
* airflow.metrics.validators.BlockListValidator
* airflow.utils.dates.parse_execution_date
* airflow.utils.dates.round_time
* airflow.utils.dates.scale_time_units
* airflow.utils.dates.infer_time_unit
* airflow.utils.file.TemporaryDirectory
* airflow.utils.file.mkdirs
* airflow.www.auth.has_access
* airflow.api_connexion.security.requires_access
* airflow.utils.dag_cycle_tester.test_cycle
* airflow.utils.state.SHUTDOWN
* airflow.utils.state.terminating_states
@Lee-W Lee-W force-pushed the extend-air-302-removed-names branch from d37fbad to 897ac7f Compare December 3, 2024 14:12
@Lee-W Lee-W changed the title feat(air302): extend removed names [airflow]: extend removed names (AIR302) Dec 3, 2024
@Lee-W Lee-W marked this pull request as ready for review December 3, 2024 14:26
@MichaReiser MichaReiser added rule Implementing or modifying a lint rule preview Related to preview mode features labels Dec 3, 2024
@MichaReiser MichaReiser merged commit edce559 into astral-sh:main Dec 3, 2024
21 checks passed
dcreager added a commit that referenced this pull request Dec 4, 2024
* main:
  [red-knot] Test: Hashable/Sized => A/B (#14769)
  [`flake8-type-checking`] Expands TC006 docs to better explain itself (#14749)
  [`pycodestyle`] Handle f-strings properly for `invalid-escape-sequence (W605)` (#14748)
  [red-knot] Add fuzzer to catch panics for invalid syntax (#14678)
  Check `AIR001` from builtin or providers `operators` module (#14631)
  [airflow]: extend removed names (AIR302) (#14734)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
preview Related to preview mode features rule Implementing or modifying a lint rule
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants