Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limited inference with union of callables and assignment to a union #18191

Open
Viicos opened this issue Nov 26, 2024 · 0 comments
Open

Limited inference with union of callables and assignment to a union #18191

Viicos opened this issue Nov 26, 2024 · 0 comments
Labels
bug mypy got something wrong topic-type-context Type context / bidirectional inference

Comments

@Viicos
Copy link
Contributor

Viicos commented Nov 26, 2024

Bug Report

Apologies if this was raised already. The only related issue I could find is #15887.

In Pydantic 2.10, we changed the definition of our field function to be:

from typing import Callable, TypeVar

_T = TypeVar('_T')

def Field(
    *,
    default_factory: Callable[[], _T] | Callable[[dict[str, Any]], _T],
) -> _T: ...

While this type checks correctly for some common use cases, we get errors when the Field function is assigned to a union of types, or when the default factory signature isn't trivial (e.g. contains overloads). Here are some examples failing:

(playground)

cvar = ContextVar[None]("session_id", default=None)


class Class:
    # error: Argument "default_factory" to "Field" has incompatible type "type[list[Any]]"; expected "Callable[[], Never] | Callable[[dict[str, Any]], Never]"  [arg-type]:
    a: list | None = Field(default_factory=list)

    # error: Argument "default_factory" to "Field" has incompatible type overloaded function; expected "Callable[[], Never] | Callable[[dict[str, Any]], Never]"  [arg-type]:
    b: None = Field(default_factory=cvar.get)  # note that `cvar.get` is overloaded. The first overload (`() -> None`) should match.


    # fine:
    fine: list = Field(default_factory=list)

Note that (at least for a), this worked as expected with the old type inference.

I first wanted to know if this can be considered a mypy limitation, or is the code really unsafe in some way? Or maybe is it because the typing spec does not provide any info regarding these use cases?

In comparison, pyright and pyre accepts the provided examples. We will currently recommend users to add a type: ignore comment and redirect to this issue. However, feel free to close if there's already an issue tracking this.

Thanks in advance

Your Environment

  • Mypy version used: 1.13
  • Mypy command-line flags: None
  • Mypy configuration options from mypy.ini (and other config files): None
  • Python version used: 3.13
@Viicos Viicos added the bug mypy got something wrong label Nov 26, 2024
@brianschubert brianschubert added the topic-type-context Type context / bidirectional inference label Nov 26, 2024
mergify bot added a commit to instructlab/instructlab that referenced this issue Dec 10, 2024
…require `pydantic<=2.9.2` (#2767)

**Issue resolved by this Pull Request:**
Resolves #2765
Resolves #2773



**Checklist:**

- [x] **Commit Message Formatting**: Commit titles and messages follow guidelines in the
  [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/#summary).
- [ ] [Changelog](https://github.com/instructlab/instructlab/blob/main/CHANGELOG.md) updated with breaking and/or notable changes for the next minor release.
- [ ] Documentation has been updated, if necessary.
- [ ] Unit tests have been added, if necessary.
- [ ] Functional tests have been added, if necessary.
- [ ] E2E Workflow tests have been added, if necessary.

In the `docling-parse` v3.0.0 release, the maintainers rearranged a lot of logic and deprecated certain syntaxes, which has resulted in our InstructLab e2e tests failing to build. (See #2765 for more details.) Since `docling-parse` is indirectly pulled in via the `instructlab-sdg` package,  the `docling-parse`/`docling` issues were resolved in `instructlab-sdg` release [v0.6.2](https://github.com/instructlab/sdg/releases/tag/v0.6.2) and I have therefore updated this PR to require `instructlab-sdg>=0.6.2`.

Also, there was another unforeseen issue that this PR fixes: We use `mypy` to execute our linting steps, but one of `mypy`'s dependencies was recently updated with breaking changes:

<img width="1316" alt="Screenshot 2024-12-09 at 5 05 51 PM" src="https://github.com/user-attachments/assets/4a14c45d-9d8c-4f42-bb50-b625d2badc29">

Doing a quick investigation, I see that `mypy` has some issues filed for this problem:

  - python/mypy#18191
  - pydantic/pydantic#10950

To workaround this dependency issue, I pinned the related dependency, `pydantic`, to `<=v2.9.2` in our `tox.ini` file. This will force our latest `mypy` to use a compatible `pydantic`. However, note that I did also pin `mypy>=1.0,<1.14`. I did this as a safety measure for when the `mypy` maintainers inevitably fix the issue in v1.14 or later.


Approved-by: nathan-weinberg

Approved-by: danmcp
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug mypy got something wrong topic-type-context Type context / bidirectional inference
Projects
None yet
Development

No branches or pull requests

2 participants