Is there a way annotate types based on class attributes?

[leandropls]

Take this snippet, for example:

from typing import Iterable, Iterator, Type, TypeVar

T = TypeVar('T')

class GenericNumberIterable(Iterable[T]):
    itemtype: Type[T]

    def __iter__(self) -> Iterator[T]:
        itemtype = self.itemtype
        yield from (itemtype(x) for x in range(10))

class FloatIterable(GenericNumberIterable, Iterable[float]):
    itemtype = float

class IntIterable(GenericNumberIterable, Iterable[int]):
    itemtype = int

An attempt with this with mypy yields:

$ mypy scratch_21.py 
scratch_21.py:11: error: Too many arguments for "object"

And further attempts to have itemtype recognized as a placeholder for a class only generates other errors, for exemple:

from typing import Iterable, Iterator, Type, List


class GenericNumberIterable(Iterable[float]):
    itemtype: Type[float]

    def __iter__(self) -> Iterator[float]:
        itemtype = self.itemtype
        yield from (itemtype(x) for x in range(10))


class FloatIterable(GenericNumberIterable, Iterable[float]):
    itemtype = float


class IntIterable(GenericNumberIterable, Iterable[int]):
    itemtype = int

x: List[int] = list(IntIterable())

Leads mypy to return:

$ mypy scratch_21.py 
scratch_21.py:19: error: Argument 1 to "list" has incompatible type "IntIterable"; expected "Iterable[int]"

[gvanrossum]

(Aside: please you mark the line where the error occurs with a comment.) If you want to parameterize a class you must use a generic parameter; your approach of using a class attribute doesn't work.

…

On Oct 12, 2017 7:10 PM, "Leandro Lima" ***@***.***> wrote: Take this snippet, for example: from typing import Iterable, Iterator, Type, TypeVar T = TypeVar('T') class GenericNumberIterable(Iterable[T]): itemtype: Type[T] def __iter__(self) -> Iterator[T]: itemtype = self.itemtype yield from (itemtype(x) for x in range(10)) class FloatIterable(GenericNumberIterable, Iterable[float]): itemtype = float class IntIterable(GenericNumberIterable, Iterable[int]): itemtype = int An attempt with this with mypy yields: $ mypy scratch_21.py scratch_21.py:11: error: Too many arguments for "object" And further attempts to have itemtype recognized as a placeholder for a class only generates other errors, for exemple: from typing import Iterable, Iterator, Type, List class GenericNumberIterable(Iterable[float]): itemtype: Type[float] def __iter__(self) -> Iterator[float]: itemtype = self.itemtype yield from (itemtype(x) for x in range(10)) class FloatIterable(GenericNumberIterable, Iterable[float]): itemtype = float class IntIterable(GenericNumberIterable, Iterable[int]): itemtype = int x: List[int] = list(IntIterable()) Leads mypy to return: $ mypy scratch_21.py scratch_21.py:19: error: Argument 1 to "list" has incompatible type "IntIterable"; expected "Iterable[int]" — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#4109>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACwrMh143kpJa4YhnMyBnGM_cAxgn-t0ks5srsahgaJpZM4P34M7> .

[leandropls]

Do you mean like this?

from typing import Iterable, Iterator, Type, TypeVar, List, Callable, Any

T = TypeVar('T', float, int)

class GenericNumberIterable(Iterable[T]):
    def __init__(self, itemtype: Type[T]) -> None:
        self.itemtype: Type[T] = itemtype

    def __iter__(self) -> Iterator[T]:
        itemtype = self.itemtype
        yield from (itemtype(x) for x in range(10))

class FloatIterable(GenericNumberIterable, Iterable[float]):
    def __init__(self) -> None:
        super().__init__(itemtype=float)

class IntIterable(GenericNumberIterable, Iterable[int]):
    def __init__(self) -> None:
        super().__init__(itemtype=int)

This one passes, though the T = TypeVar('T', float, int) doesn't really seem right, since they're duck-type compatible. I've only used it this way 'cause TypeVar doesn't allow me to leave just float (meaning any subclass of float).

[gvanrossum]

Can't you write class FloatIterable(GenericNumberIterable[float]): instead?

Honestly I'm not sure what you're trying to do, since mypy doesn't really support a numeric tower in a reasonable way. If your real-world example doesn't have to do with numbers then using float/int here is just a distraction.

And indeed the typevar declaration probably should use something like bound=float since mypy considers int a subtype of float (more or less).

[leandropls]

Yeah, it does have to do with numbers. I have numeric types that enforce contracts (test if value is valid on __new__):

• Natural, PositiveInt
• NonNegativeFloat, PositiveFloat

And I have sequence types for some of these numeric types:

• NaturalSequence (for Naturals)
• NNFSequence (for NonNegativeFloats)
• NumberSequence (for floats/ints)

The sequence types have several useful methods, allowing me to do element wise operations (like sum/multiply/divide) between sequences and also several useful class methods, facilitating the construction of these sequences (like building a sequence out of the number of elements, an initial value and a growth rate).

The implementation I'm currently using is akin to the first snippet, but I'm working towards making the code verifiable by mypy, so I'm trying to find a solution that both offer the same interface as the current implementation and is verifiable.

Based on our discussion here, this was my lastest attempt:

S = TypeVar('S')
T = TypeVar('T')

class NumberSequence(Generic[S, T], Sequence[S]):
    __slots__ = ('data',)
    data: Tuple[S, ...]
    imptype: Type[S]

    def __init__(self, more: Iterable[S]) -> None:
        imptype = cast(Type[S], self.imptype)
        self.data = tuple(imptype(x) for x in more) # error: Too many arguments for "object"

    @overload  # error: Signature of "__getitem__" incompatible with supertype "Sequence"
    def __getitem__(self, index: int) -> T:
        pass

    @overload
    def __getitem__(self, index: slice) -> S:
        pass

    def __getitem__(self, index):
        if isinstance(index, slice):
            return self.__class__(self.data[index])
        else:
            return self.data[index]

    def __len__(self) -> int:
        return len(self.data)

    def __repr__(self) -> str:
        return self.__class__.__name__ + repr(self.data)


class FloatSequence(NumberSequence[float, 'FloatSequence']):
    imptype = float

class IntSequence(NumberSequence[int, 'IntSequence']):
    imptype = int

x: List[int] = list(IntSequence((1, 2, 3)))
y: List[float] = list(FloatSequence((1, 2, 3)))

But this also generates the following errors (also marked in the code above):

scratch_21.py:14: error: Too many arguments for "object"
scratch_21.py:16: error: Signature of "__getitem__" incompatible with supertype "Sequence"

[ilevkivskyi]

There are two points that I can recommend here:

• The type variable for NumberSequence clearly should have bound=..., mypy rightfully complains that imptype(x) is not safe, since if impltype=object, then this call will fail at runtime. If you want to support both built-in numeric types and your special types, the bound may be a union.
• Be careful with __new__, there are some bugs in mypy, see e.g. Invalid signature for type object when both __init__ and __new__ #1435 and Infer attributes from __new__ #1021, do your checks in __init__ (this is almost always possible).

[leandropls]

I didn't know bound=... thanks! It does solve the first error (S = TypeVar('S', bound=float)).

I don't really know what I should bind T to.

[ilevkivskyi]

@leandropls

I don't really know what I should bind T to.

But why do you need T at all? It only appears in __getitem__, while I think its type should be:

class NumberSequence(Sequence[S]):
    ...
    @overload
    def __getitem__(self, index: int) -> S: ...
    @overload
    def __getitem__(self, index: slice) -> 'NumberSequence[S]': ...
    ...

[leandropls]

@ilevkivskyi, 'cause I don't really wanna return NumberSequence[S], but one of it subtypes, which perform the contract checking. Is there a way for me to pull out the S value at runtime to type check the elements? One of my use cases is that I can get untrusted data from the user and do:

untrusted: Iterable[Any]
try:
    mysequence = NaturalSequence(untrusted)
except ValueError:
    raise UIError('please fix the number sequence')

I'm subclassing NumberSequence[T] and defining a class attribute that points to the type each element should be converted to. This way, when I slice it, I don't wanna return an instance of NumberSequence, but one if its subtypes.

Does that make sense?

[ilevkivskyi]

I don't really wanna return NumberSequence[S], but one of it subtypes

For this you can use self-types, but they don't work currently in generic classes, see #2354.

[leandropls]

@ilevkivskyi yeah... I did try self-types with no success. Is there an workaround? Maybe with a second type variable, like above?

[leandropls]

Looks like this thread is dead already. Should I close it?

[ilevkivskyi]

Sorry, due to some reasons I have not enough time to help you here, maybe someone else can give you advice.

[leandropls]

@ilevkivskyi No problem. I've learned a lot from your advices already -- big thanks for you and @gvanrossum. I asked what to do with the issue 'cause I'm not sure how questions issues are handled when they're considered answered.

[gvanrossum]

Let's close it. If you have more questions you can still add comments to the issue, or you can create a new one. Good luck!