slice behavior difference in mypy 1.14+dev and 1.13

[Dr-Irv]

We test pandas-stubs with the nightly version of mypy. It reveals an issue with how a method that returns slice is annotated. With pandas-stubs, we get an error like this with the development version of mypy:

tests/test_frame.py:2401: error: Expression is of type "tuple[Index[int], slice[None, None, None]]", not "tuple[Index[int], slice[Any, Any, Any]]"  [assert-type]

In this case, the assert_type() call of assert_type(pd.IndexSlice[ind, :], tuple["pd.Index[int]", slice]) works with mypy 1.13, fails with the 1.14 dev version.

This is due to a change in typeshed where slice has become generic. I think that change in typeshed is bundled with the 1.14+dev version of mypy, not the released version 1.13.

A simpler example is below. Not sure if this is a mypy or a typeshed issue.

Bug Report

mypy 1.14+dev infers a different value for slice than 1.13

To Reproduce
Requires 2 files.

First, there is retslice.pyi:

from typing import TypeAlias, TypeVar

class Index:
    ...

_IndexSliceTuple: TypeAlias = tuple[
    Index | int | float | str | slice , ...
]

_IndexSliceUnion: TypeAlias = slice | _IndexSliceTuple

_IndexSliceUnionT = TypeVar("_IndexSliceUnionT", bound=_IndexSliceUnion)

class _IndexSlice:
    def __getitem__(self, arg: _IndexSliceUnionT) -> _IndexSliceUnionT: ...

IndexSlice: _IndexSlice

Then there is mypyslice.py :

from typing import assert_type
from retslice import IndexSlice, Index

foo = IndexSlice[Index(), :]
assert_type(foo, tuple[Index, slice])

Expected Behavior

No errors reported by mypy 1.14+dev

Actual Behavior

mypyslice.py:5: error: Expression is of type "tuple[Index, slice[None, None, None]]", not "tuple[Index, slice[Any, Any, Any]]"  [assert-type]

Your Environment

• Mypy version used: v1.14.0%2Bdev.9405bfd9205ea369c11150907764fa46c03cb1f7
• Mypy command-line flags: None
• Mypy configuration options from mypy.ini (and other config files): None
• Python version used: 3.13

May be related to #18149

[hauntsaninja]

Yeah, I think this is expected. You should use assert_type(foo, tuple[Index, slice[None, None, None]])

If you just want to check compatibility instead of the "type exact match" that assert_type does, you could do something like: bar: tuple[Index, slice] = foo

[Dr-Irv]

@hauntsaninja Well, this is inconsistent. Consider the following example:

from typing import reveal_type

class IndexSlice:
    def __getitem__(self, arg: slice) -> slice:
        return arg

foo = IndexSlice()[2:4]
reveal_type(foo)

foo2 = IndexSlice()[:]
reveal_type(foo2)

mypy 1.14.0+dev reports:

mypyslice2.py:8: note: Revealed type is "builtins.slice[Any, Any, Any]"
mypyslice2.py:11: note: Revealed type is "builtins.slice[Any, Any, Any]"

So in my original example. I would have expected the same - the type should be slice[Any, Any, Any] not slice[None, None, None]

[JelleZijlstra]

Your original example used a TypeVar.

[brianschubert]

@Dr-Irv The difference between that and the original example is that the original example uses a type variable with an upper bound of slice[Any, Any, Any] whereas that code returns slice[Any, Any, Any] exactly. It's the same distinction as in this example:

def f1(x: list) -> list: return x
def f2[T: list](x: T) -> T: return x

reveal_type(f1([1, 2, 3]))  # N: Revealed type is "builtins.list[Any]"
reveal_type(f2([1, 2, 3]))  # N: Revealed type is "builtins.list[builtins.int]"

[Dr-Irv]

OK - I see what is happening now. Let me see if I can make the pandas-stubs tests work with pyright, mypy 1.13.0 and mypy 1.14.0

[Dr-Irv]

closing as a result

[Dr-Irv]

I'm going to reopen this, because now there is a difference in pyright 1.1.389 and mypy 1.14+dev behavior. Using the original example, I changed the main python program mypyslice.py to be:

from typing import assert_type
from retslice import IndexSlice, Index

foo = IndexSlice[Index(), :]
reveal_type(foo)

pyright output:

mypyslice.py:5:13 - information: Type of "foo" is "tuple[Index, slice[_StartT@slice, _StopT@slice, _StepT@slice]]"

mypy 1.14+dev output:

mypyslice.py:5: note: Revealed type is "tuple[retslice.Index, builtins.slice[None, None, None]]"

mypy 1.13 output:

mypyslice.py:5: note: Revealed type is "tuple[retslice.Index, builtins.slice]"

From a typing perspective, I think the type of just slice is correct with pyright, because slice is generic. But I can also open an issue with pyright as well if you typing gods want to debate this.

Based on what I read in the PEP here: https://peps.python.org/pep-0696/#type-parameters-as-parameters-to-generics , I think the slice[None, None, None] result is incorrect with mypy. The default values of None should not have been instantiated into the type. But I could be misunderstanding something.

[brianschubert]

Based on what I read in the PEP here: https://peps.python.org/pep-0696/#type-parameters-as-parameters-to-generics , I think the slice[None, None, None] result is incorrect with mypy. The default values of None should not have been instantiated into the type. But I could be misunderstanding something.

If I understand what you're saying correctly, I don't think default type arguments are relevant here. (In fact, mypy does not consider them when inferring the type of slice expressions).

The reason mypy produces slice[None, None, None] is because mypy treats the slice expression : as being equivalent to None : None : None. This matches how the slice object is passed at runtime:

class Foo:
    def __getitem__[T](self, item: T) -> T: return item
foo = Foo()

>>> foo[:]
slice(None, None, None)

I think pyright's behavior may be from a lack of implementation rather than an intentional behavior (cc @erictraut for comment). It seems that pyright does not currently attempt to infer the generic type parameters for slice expressions:

class Foo:
    def __getitem__[T](self, item: T) -> T: return item
foo = Foo()


x1 = foo["a"::False]
reveal_type(x1)  # mypy:     Revealed type is "builtins.slice[builtins.str, None, builtins.bool]"
                 # pyright:  Type of "x1" is "slice[Any, Any, Any]"

x2 = foo[slice("a", None, False)]
reveal_type(x2)  # mypy:     Revealed type is "builtins.slice[builtins.str, None, builtins.bool]"
                 # pyright:  Type of "x2" is "slice[str, None, bool]"

[erictraut]

It seems that pyright does not currently attempt to infer the generic type parameters for slice expressions

Yes, this is missing functionality in pyright. The slice type only recently became generic in typeshed. I'll add support for it. Here's the tracking issue.

[Dr-Irv]

So once this is fixed in pyright, I will then face this issue with pandas-stubs. We test as follows:

• Latest version of released pyright (currently 1.1.389)
• Latest version of released mypy (currently 1.13.0)

The type checking of code needs to work with both of the above to allow a pull request to be accepted.

We also test the in-development version of mypy, (version 1.14+dev), but it is not required that things pass with that version for a PR to be approved.

So for mypy 1.13.0, and mypy 1.14+dev and the current pyright 1.1.389, using the simple example above:

class Foo:
    def __getitem__[T](self, item: T) -> T: return item
foo = Foo()


x1 = foo["a"::1]
reveal_type(x1)  # mypy: 1.14+dev    Revealed type is "builtins.slice[builtins.str, None, builtins.bool]"
                 # mypy 1.13.0   Revealed type is "builtins.slice"
                 # pyright:  Type of "x1" is "slice[Any, Any, Any]"

x2 = foo[slice("a", None, 1)]
reveal_type(x2)  # mypy:     Revealed type is "builtins.slice[builtins.str, None, builtins.bool]"
                 # mypy 1.13.0   Revealed type is "builtins.slice"                               
                 # pyright:  Type of "x2" is "slice[str, None, bool]" 

So how in the testing code can I support mypy 1.13.0 which doesn't have the updated typeshed, and 1.14+dev (and soon pyright, once the issue is fixed there) that does have the updated typeshed?

Is there a way to say "Use assert_type(x1, slice) when using mypy 1.13.0 and assert_type(x1, slice[str, None, bool] with mypy 1.14+dev and the newer version of pyright (once it is released)?

Or, equivalently, can you have code that uses assert_type() that conditionally tests whether slice is Generic or not, but does that test when type checking?

We might just have to pin down the pyright and mypy versions in our CI until mypy 1.14 is released, and then we can upgrade both.

[erictraut]

One option is to use a bool constant with a name like MYPY. You could then use --always-true MYPY in mypy and "defineConstant": {"MYPY": false } option in pyright. The test code could then use an if MYPY: conditional for different assert_type behaviors.

[brianschubert]

Some other options:

1. Check for compatibility instead of an exact type, like hauntsaninja suggested above. Something like

   foo: tuple["pd.Index[int]", slice] = pd.IndexSlice[ind, :]
   check(foo, tuple)

2. Use a slice object instead of a slice expression, and coerce it to slice[Any, Any, Any]:

   s: slice = slice(None, None, None)
   check(assert_type(pd.IndexSlice[ind, s], tuple["pd.Index[int]", slice]), tuple)

3. Similar to what Eric suggested, conditionally define an alias for the right slice type:

   # Define appropriately via --always-true/defineConstant based on mypy/pyright version
   if GENERIC_SLICE:  
       EmptySlice: TypeAlias = slice[None, None, None]
   else:
       EmptySlice: TypeAlias = slice
   
   check(assert_type(pd.IndexSlice[ind, :], tuple["pd.Index[int]", EmptySlice]), tuple)

[Dr-Irv]

Thanks @brianschubert . For now, I can use option (1) until both pyright 1.1.390 and mypy 1.14 are released.

Will close this. Thanks all for the help.