Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[red-knot] Infer target types for unpacked tuple assignment #13316

Merged
merged 14 commits into from
Oct 15, 2024

Conversation

dhruvmanila
Copy link
Member

@dhruvmanila dhruvmanila commented Sep 10, 2024

Summary

This PR adds support for unpacking tuple expression in an assignment statement where the target expression can be a tuple or a list (the allowed sequence targets).

The implementation introduces a new infer_assignment_target which can then be used for other targets like the ones in for loops as well. This delegates it to the infer_definition. The final implementation uses a recursive function that visits the target expression in source order and compares the variable node that corresponds to the definition. At the same time, it keeps track of where it is on the assignment value type.

The logic also accounts for the number of elements on both sides such that it matches even if there's a gap in between. For example, if there's a starred expression like (a, *b, c) = (1, 2, 3), then the type of a will be Literal[1] and the type of b will be Literal[2].

There are a couple of follow-ups that can be done:

  • Use this logic for other target positions like for loop
  • Add diagnostics for mis-match length between LHS and RHS

Test Plan

Add various test cases using the new markdown test framework.
Validate that existing test cases pass.

@dhruvmanila dhruvmanila added the red-knot Multi-file analysis & type inference label Sep 10, 2024
@dhruvmanila

This comment was marked as outdated.

@dhruvmanila dhruvmanila force-pushed the dhruv/unpacked-tuple-assignment branch from 9ea561f to c66a85d Compare September 10, 2024 22:06
Copy link
Contributor

github-actions bot commented Sep 10, 2024

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

@dhruvmanila dhruvmanila force-pushed the dhruv/unpacked-tuple-assignment branch from c66a85d to 04818bc Compare September 10, 2024 22:22
@dhruvmanila

This comment was marked as resolved.

@dhruvmanila dhruvmanila force-pushed the dhruv/unpacked-tuple-assignment branch 5 times, most recently from 925889e to c8dbb21 Compare October 14, 2024 14:38
@dhruvmanila dhruvmanila marked this pull request as ready for review October 14, 2024 16:13
Copy link
Contributor

@carljm carljm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great! A few issues, but seems quite close. Make sure for any bugs you fix in updating the PR, you also add a new test that would fail without that fix.

Comment on lines +118 to +141
TODO: Remove duplicate diagnostics. This is happening because for a sequence-like
assignment target, multiple definitions are created and the inference engine runs
on each of them which results in duplicate diagnostics.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixing this will require creating a new Salsa tracked struct for unpacking assignments to ensure we just do the unpacking once, correct?

I don't think I realized in our previous conversations that this would not just be a performance issue, but also a matter of diagnostics correctness. I think this raises the priority on a follow-up PR to create an Unpack tracked struct so we can Salsa-cache the unpacking and do it just once. (There are other unpacking cases we'll need to handle -- for loops and comprehensions -- which is why this shouldn't have an assignment-specific name.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's correct, we'd need a way to do unpacking once and cache it for future lookup. I can take this up as a follow-up at a high priority, will need to think about the required changes.

crates/red_knot_python_semantic/src/types/infer.rs Outdated Show resolved Hide resolved
crates/red_knot_python_semantic/src/types/infer.rs Outdated Show resolved Hide resolved
crates/red_knot_python_semantic/src/types/infer.rs Outdated Show resolved Hide resolved
crates/red_knot_python_semantic/src/types/infer.rs Outdated Show resolved Hide resolved
@dhruvmanila dhruvmanila force-pushed the dhruv/unpacked-tuple-assignment branch from d1e492a to b158bc5 Compare October 15, 2024 12:47
@dhruvmanila
Copy link
Member Author

@carljm Thanks for the great review, I've addressed all of the feedback except for the duplicate diagnostic issue (#13316 (comment)) which I'll fix as a follow-up.

Comment on lines 126 to 135
### Starred expression (5)

```py
# TODO: Remove 'not-iterable' diagnostic
[a, b, *c] = (1, 2, 3, 4) # error: "Object of type `None` is not iterable"
reveal_type(a) # revealed: Literal[1]
reveal_type(b) # revealed: Literal[2]
# TODO: Should be List[int] once support for assigning to starred expression is added
reveal_type(c) # revealed: @Todo
```
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new test case where the starred expression isn't at the same position as other test cases.

reveal_type(b) # revealed: LiteralString
```

### Starred expression (1)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These test cases are similar to that of the Tuple type above and it only differs such that we use strings and the correct revealed type.

Copy link
Contributor

@carljm carljm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look great! I noticed a few more things in reviewing just now, sorry I didn't catch these yesterday!

crates/red_knot_python_semantic/src/types/infer.rs Outdated Show resolved Hide resolved
crates/red_knot_python_semantic/src/types/infer.rs Outdated Show resolved Hide resolved
crates/red_knot_python_semantic/src/types/infer.rs Outdated Show resolved Hide resolved
Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice -- this is complex stuff to get right!! A couple of small points on top of Micha's and Carl's comments:

crates/red_knot_python_semantic/src/types/infer.rs Outdated Show resolved Hide resolved
@dhruvmanila dhruvmanila requested a review from carljm October 15, 2024 18:16
crates/red_knot_python_semantic/src/types/infer.rs Outdated Show resolved Hide resolved
crates/red_knot_python_semantic/src/types/infer.rs Outdated Show resolved Hide resolved
@carljm
Copy link
Contributor

carljm commented Oct 15, 2024

Given it's quite late now for @dhruvmanila , I'm going to rebase this, apply my one suggested change, and go ahead and merge, to reduce our number of outstanding PRs and thus potential conflicts!

@carljm carljm force-pushed the dhruv/unpacked-tuple-assignment branch from b905919 to efac63d Compare October 15, 2024 19:02
@carljm carljm enabled auto-merge (squash) October 15, 2024 19:03
@carljm carljm merged commit b16f665 into main Oct 15, 2024
19 checks passed
@carljm carljm deleted the dhruv/unpacked-tuple-assignment branch October 15, 2024 19:07
@dhruvmanila
Copy link
Member Author

Thanks @carljm for the reviews and merging!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
red-knot Multi-file analysis & type inference
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants