Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Higher-Kinded TypeVars #548

Open
tek opened this issue Mar 30, 2018 · 172 comments
Open

Higher-Kinded TypeVars #548

tek opened this issue Mar 30, 2018 · 172 comments
Labels
topic: feature Discussions about new features for Python's type annotations

Comments

@tek
Copy link

tek commented Mar 30, 2018

aka type constructors, generic TypeVars

Has there already been discussion about those?
I do a lot of FP that results in impossible situations because of this. Consider an example:

A = TypeVar('A')
B = TypeVar('B')
F = TypeVar('F')

class M(Generic[F[X], A]):
    def x(fa: F[A], f: Callable[[A], B]) -> F[B]:
        return map(f, fa)

M().x([1], str)

I haven't found a way to make this work, does anyone know a trick or is it impossible?
If not, consider the syntax as a proposal.
Reference implementations would be Haskell, Scala.
optimally, the HK's type param would be indexable as well, allowing for F[X[X, X], X[X]]


Summary of current status (by @smheidrich, 2024-02-08):

  • @JelleZijlstra has indicated interest in sponsoring a PEP, conditional on a prototype implementation in a major type checker and a well-specified draft PEP.
  • Drafting the PEP takes place in @nekitdev's fork of the peps repo. The stub PEP draft so far contains a few examples of the proposed syntax.
  • That same repo's GitHub Discussions forum forum has been designated as the place to discuss the PEP (and presumably the prototype implementation?). Some limited further discussions have taken place there.
    • If you want to be notified of new discussion threads, I think you have to set the whole repo as "watched" in GitHub?
@ilevkivskyi
Copy link
Member

I think this came up few times in other discussions, for example one use case is python/mypy#4395. But TBH this is low priority, since such use cases are quite rare.

@tek
Copy link
Author

tek commented Mar 30, 2018

damn, I searched very thoroughly but did not find this one! 😄
So, consider this my +1!

@gvanrossum
Copy link
Member

gvanrossum commented Mar 31, 2018 via email

@tek
Copy link
Author

tek commented Mar 31, 2018

are you approving the feature?

@gvanrossum
Copy link
Member

gvanrossum commented Mar 31, 2018 via email

@tek
Copy link
Author

tek commented Mar 31, 2018

awfully pragmatic. Where's your sense of adventure? 😄
anyways, I'll work on it, though it's gonna take a while to get into the project.

@landonpoch
Copy link

Similar to microsoft/TypeScript#1213

Not sure if the discussion over there provides any useful insights to the effort over here.

@rcalsaverini
Copy link

Hi @tek. I'm also very interested in this, so I'd like to ask if you had any progress with this and volunteer to help if you want.

@tek
Copy link
Author

tek commented Jan 10, 2019

@rcalsaverini sorry, I've been migrating my legacy code to haskell and am abandoning python altogether. but I wish you great success!

@rcalsaverini
Copy link

Oh, sad to hear but I see your point. Thanks.

@syastrov
Copy link

syastrov commented Oct 10, 2019

Just to add another use case (which I think relates to this issue):

Using Literal types along with overloading __new__, along with higher-kinded typevars could allow implementing a generic "nullable" ORM Field class, using a descriptor to provide access to the appropriate nullable-or-not field values. The descriptor wouldn't have to be reimplemented in subclasses.

It is one step closer to being possible due to the most recent mypy release's support for honoring the return type of __new__ (python/mypy#1020).

Note: this is basically a stripped-down version of Django's Field class:

# in stub file

from typing import Generic, Optional, TypeVar, Union, overload, Type
from typing_extensions import Literal

_T = TypeVar("_T", bound="Field")
_GT = TypeVar("_GT")

class Field(Generic[_GT]):
    # on the line after the overload: error: Type variable "_T" used with arguments
    @overload
    def __new__(cls: Type[_T], null: Literal[False] = False, *args, **kwargs) -> _T[_GT]: ...
    @overload
    def __new__(cls: Type[_T], null: Literal[True], *args, **kwargs) -> _T[Optional[_GT]]: ...
    def __get__(self, instance, owner) -> _GT: ...

class CharField(Field[str]): ...
class IntegerField(Field[int]): ...
# etc...

# in code

class User:
  f1 = CharField(null=False)
  f2 = CharField(null=True)

reveal_type(User().f1) # Expected: str
reveal_type(User().f2) # Expected: Union[str, None]

@samuelcolvin
Copy link

I wonder if this is what I need or if there's currently a work around for my (slightly simpler) case?:

I'm building an async redis client with proper type hints. I have a "Commands" class with methods for all redis commands (get, set, exists, strlen ... and hundreds more). Normally each of those methods should return a future (actually coroutine) to the result, but in pipeline mode they should all return None - the commands are added to the pipeline to be executed later.

This is easy enough to implement in python, but not so easy to type hint correctly.

Basic example:

class Redis:
    def execute(self, command) -> Coroutine[Any, Any, Union[None, str, int, float]]:
        return self.connection.execute(...)

    def get(self, *args) -> Coroutine[Any, Any, str]:
        ...
        return self.execute(command)

    def set(self, *args) -> Coroutine[Any, Any, None]:
        ...
        return self.execute(command)

    def exists(self, *args) -> Coroutine[Any, Any, bool]:
        ...
        return self.execute(command)

    # ... and many MANY more ...


class RedisPipeline(Redis):
    def execute(self, command) -> None:
        self.pipeline.append(command)

I tried numerous options to make Coroutine[Any, Any, xxx] generic, but nothing seems to work.

Is there any way around this with python 3.8 and latest mypy? If not a solution would be wonderful - as far as I can think, my only other route for proper types is a script which copy and pastes the entire class and changes the return types in code.

@gvanrossum
Copy link
Member

@samuelcolvin I don't think this question belongs in this issue. The reason for the failure (knowing nothing about Redis but going purely by the code you posted) is that in order to make this work, the base class needs to switch to an Optional result, i.e.

    def execute(self, command) -> Optional[Coroutine[Any, Any, Union[None, str, int, float]]]:

@samuelcolvin
Copy link

I get that, but I need all the public methods to definitely return a coroutine. Otherwise, if it returned an optional coroutine, it would be extremely annoying to use.

What I'm trying to do is modify the return type of many methods on the sub-classes, including "higher kind" types which are parameterised.

Hence thinking it related to this issue.

@gvanrossum
Copy link
Member

Honestly I have no idea what higher-kinded type vars are -- my eyes glaze over when I hear that kind of talk. :-)

I have one more suggestion, then you're on your own. Use a common base class that has an Optional[Coroutine[...]] return type and derive both the regular Redis class and the RedisPipeline class from it.

@samuelcolvin
Copy link

Okay, so the simple answer is that what I'm trying to do isn't possible with python types right now.

Thanks for helping - at least I can stop my search.

@gvanrossum
Copy link
Member

gvanrossum commented Apr 26, 2020 via email

@samuelcolvin
Copy link

humm, but the example above under "Basic example" I would argue IS type-safe.

All the methods which end return self.execute(...) return what execute returns - either a Coroutine or None.

Thus I don't see how this as any more "unsafe" than normal use of generics.

@jab
Copy link

jab commented Apr 29, 2020

@gvanrossum, I can relate!

I wonder if bidict provides a practical example of how this issue prevents expressing a type that you can actually imagine yourself needing.

>>> element_by_atomicnum = bidict({0: "hydrogen", 1: "helium"})
>>> reveal_type(element_by_atomicnum)  # bidict[int, str]
# So far so good, but now consider the inverse:
>>> element_by_atomicnum.inverse
bidict({"hydrogen": 0, "helium": 1})

What we want is for mypy to know this:

>>> reveal_type(element_by_atomicnum.inverse)  # bidict[str, int]

merely from a type hint that we could add to a super class. It would parameterize not just the key type and the value type, but also the self type. In other words, something like:

KT = TypeVar('KT')
VT = TypeVar('VT')

class BidirectionalMapping(Mapping[KT, VT]):
    ...
    def inverse(self) -> $SELF_TYPE[VT, KT]:
        ...

where $SELF_TYPE would of course use some actually legal syntax that allowed composing the self type with the other parameterized types.

@gvanrossum
Copy link
Member

Okay, I think that example is helpful. I recreated it somewhat simpler (skipping the inheritance from Mapping and the property decorators):

from abc import abstractmethod
from typing import *

T = TypeVar('T')
KT = TypeVar('KT')
VT = TypeVar('VT')

class BidirectionalMapping(Generic[KT, VT]):
    @abstractmethod
    def inverse(self) -> BidirectionalMapping[VT, KT]:
        ...

class bidict(BidirectionalMapping[KT, VT]):
    def __init__(self, key: KT, val: VT):
        self.key = key
        self.val = val
    def inverse(self) -> bidict[VT, KT]:
        return bidict(self.val, self.key)

b = bidict(3, "abc")
reveal_type(b)  # bidict[int, str]
reveal_type(b.inverse())  # bidict[str, int]

This passes but IIUC you want the ABC to have a more powerful type. I guess here we might want to write it as

    def inverse(self: T) -> T[VT, KT]:  # E: Type variable "T" used with arguments

Have I got that?

@jab
Copy link

jab commented Apr 30, 2020

Exactly! It should be possible to e.g. subclass bidict (without overriding inverse), and have mypy realize that calling inverse on the subclass gives an instance of the subclass (with the key and value types swapped as well).

This isn’t only hypothetically useful, it’d really be useful in practice for the various subclasses in the bidict library where this actually happens (frozenbidict, OrderedBidict, etc.).

Glad this example was helpful! Please let me know if there’s anything further I can do to help here, and (can’t help myself) thanks for creating Python, it’s such a joy to use.

@gvanrossum
Copy link
Member

Ah, so the @abstractmethod is also a red herring.

And now I finally get the connection with the comment that started this issue.

But I still don't get the connection with @samuelcolvin's RedisPipeline class. :-(

@sobolevn
Copy link
Member

sobolevn commented Apr 30, 2020

I would also say that this example is really simple, common, but not supported:

def create(klass: Type[T], value: K) -> T[K]:
     return klass(value)

We use quite a lot of similar constructs in dry-python/returns.

As a workaround I am trying to build a plugin with emulated HKT, just like in some other languages where support of it is limited. Like:

Paper on "Lightweight higher-kinded polymorphism": https://www.cl.cam.ac.uk/~jdy22/papers/lightweight-higher-kinded-polymorphism.pdf

TLDR: So, instead of writing T[K] we can emulate this by using HKT[T, K] where HKT is a basic generic instance processed by a custom mypy plugin. I am working on this plugin for already some time now, but there's still nothing to show. You can track the progress here: https://pypi.org/project/kinds/ (part of dry-python libraries)

@nekitdev
Copy link

nekitdev commented Mar 5, 2024

Hey! Yeah, sorry, I messed up sync with the upstream so had to reclone. I'll push the draft shortly.
Edit: here is the updated link.

@Badg
Copy link

Badg commented Mar 5, 2024

Awesome, thanks!

To be completely honest, theoretical typing discussions are... a bit too abstract for me at times. So I'm not 100% sure that my use case above is actually and example of HKT. But if it is, feel free to use it!

@purepani
Copy link

I think libraries like einops would also benefit greatly from this. It would be pretty useful to be able to get shape information both into and out of any sort of array manipulation function.

@finite-state-machine
Copy link

finite-state-machine commented Sep 6, 2024

I'm not sure if further motivating examples are useful, but here's a simple function, not involved in type checking, that I don't believe can be described today:

# sketch -- not carefully checked

T = TypeVar('T')
C = TypeVar('C', bound='Collection[T]')

def filter_and_count_dups(c: C[T]) -> Tuple[C[T], Counter[T]]:
    '''copy any 'Collection' w/o duplicates, counting what's omitted

    Args:
        c: any collection of elements
    Returns:
        (c_wo_dups, dup_counts), where...
        c_wo_dups: a copy of 'c', with the 2nd and subsequent
            appearances of any given element omitted
        dup_counts: counts removed duplicates
    '''
    seen: Set[T] = set()
    ret_list: List[T] = []
    ret_counter = Counter[T]()
    for elem in c:
        if elem in seen:
            ret_counter[elem] += 1
        else:
            ret_list.append(elem)
            seen.add(elem)

    ret_wo_dups = type(c)(ret_list)
    return (ret_wo_dups, ret_counter)


r1 = filter_and_count_dups((1, 2, 4, 3, 4, 4, 2, 1))
reveal_type(r1) # ≈ Tuple[Tuple[int, ...], Counter[int]]
print(repr(r1)) # ≈ ((1, 2, 4, 3), Counter({4: 2, 2: 1, 1: 1}))

r2 = filter_and_count_dups(list('alphabet'))
reveal_type(r2) # ≈ Tuple[List[str], Counter[str]]
print(repr(r2)) # ≈ (['a', 'l', 'p', 'h', 'b', 'e', 't'], Counter({'a': 1}))

@nerodono
Copy link

nerodono commented Sep 14, 2024

One more use case, consider the following code:

class Handler[I, O](Protocol):
    def __call__(self, input: I, /) -> O: ...

class Modify(Protocol):
    def modify[O](self, f: Callable[[Self], O], /) -> O: ...

# Mixing styles since idk how to properly express it in 695
M = TypeVar("M", bound="Modify")

class Ext[Inner]:
    inner: Inner

    def modify[O](self: Ext[M], f: Callable[[M], O], /) -> Ext[O]:
        return Ext(f(self.inner))

I can extend Ext, with something like:

class MyExt[Inner](Ext[Inner]):
    def another_method(self) -> MyExt[SomeOtherType]:
        return self.modify(xxx)

However, since Ext.modify is defined to return Ext[Inner], not MyExt[Inner], my type will be erased and I'll lose all extended methods. To fix this, Ext.modify needs to be defined like that:

class Ext[Inner]:
    def modify[O](self: Self[M], f: Callable[[M], O]) -> Self[O]:
        return type(self)(self.inner.modify(f))

I have encountered that limitation when writing syntactic sugar for my library for handlers. Handlers can be combined, and handlers can be specialized, like this:

class Predicate[I](Handler[I, bool], Protocol):
    ...

Here, we specialize Predicate as Handler which returns boolean value, for this specialized version we could add some methods in Ext:

class PredicateExt[P](Ext[P]):
    def and_[I](self: PredicateExt[Predicate[I]], rhs: Predicate[I]) -> PredicateExt[Predicate[I]]:
        return self.modify(lambda lhs: And(lhs, rhs))

... but there's no way to do that without triggering type-checker, unless we had HKTs

@jorenham
Copy link

I'm starting to think that by simply making types.GenericAlias generic, we can get HKT "for free".

With this, we could write some HKT function f: (T[int]) -> T[str] with T: Sequence as:

from collections.abc import Sequence
from types import GenericAlias

def f[T: Sequence](x: GenericAlias[T, int], /) -> GenericAlias[T, str]: ...

Note that type-checkers might complain about the missing type argument in T: Sequence, which I omitted for the sake of clarity


So to be a little bit more specific, the generic GenericAlias could look something like

class GenericAlias[T, *Ps]:
    @property
    def __origin__(self) -> type[T]: ...
    @property
    def __args__(self) -> tuple[*Ps]: ...
    ...  # etc

Note that because a variadic type-parameter like *Ps cannot be covariant (at the time of writing), this might not work out-of-the-box for all HKT use-cases.

Through the __class_getitem__ method of generic classes, you'd then have e.g. Spam.__class_getitem__(T) -> GenericAlias[Spam, T], which is equivalent to Spam[T].


But there's one big problem with this approach:

>>> type(list[str])
<class 'types.GenericAlias'>
>>> class Spam[T]: ...
... 
>>> type(Spam[str])
<class 'typing._GenericAlias'>

There are two generic aliases!
... but this is probably not the right place to be discussing this, so I'll keep my rant short :)

@joaoe
Copy link

joaoe commented Nov 8, 2024

Hi.
I see many people with somewhat complex examples which are hard to read.
Some of the proposed syntaxes I see are convoluted and hard to work, and will require constantly peeking at the documentation and copy-paste the same code snippets repeatedly.

Here's the code example I have from my project

T_num = TypeVar("T_num", float, int, str)
T_seq = TypeVar("T_seq", list, tuple)

def my_fn(seq: T_seq[T_num]) -> T_seq[T_num]: ....

This obviously does not work. mypy requires list, tuple to be parametrized in the T_seq declaration, and mypy does not recognize T_seq as being a generic.

I'd suggest the syntax for the feature to be something as simple as

T_seq = TypeVar[T_num]("T_seq", list[T_num], tuple[T_num])

Cheers.

@jorenham
Copy link

jorenham commented Nov 8, 2024

I see many people with somewhat complex examples which are hard to read.
Some of the proposed syntaxes I see are convoluted and hard to work, and will require constantly peeking at the documentation and copy-paste the same code snippets repeatedly.

Can you be more specific?

I'd suggest the syntax for the feature to be something as simple as

T_seq = TypeVar[T_num]("T_seq", list[T_num], tuple[T_num])

Since Python 3.12 the new PEP 695 generic syntax is preferred over TypeVar

@samvv
Copy link

samvv commented Nov 28, 2024

Someone pointed me to this issue. I just crashed hard onto this one today. I am writing generic tree algorithms. I have: IntervalTree inherits from AVLTree inherits from BinaryTree. What I want:

T = TypeVar('T')

# What every node should have in common
class BinaryNode(Protocol[T]): ...

# Represents the actual node that is used by e.g. AVLTree
N = TypeVar('N', bound=BinaryNode)

# Parameterized on the actual node that is used and the value of it
class BinaryTree(Generic[N[T], T]):
   def rotate_left(self, node: N[T]) -> None: ...
   def add(self, value: T) -> None: ...

However, N[T] isn't valid in MyPy. No matter what I try (I even tried to just use plain functions), I can't seem to eliminate the 'higher kindness' of this structure. There is no way to extract T out of N.

If anyone knows workarounds for this pattern I'd be happy to learn them.

@Matt-Ord
Copy link

Matt-Ord commented Nov 28, 2024

Someone pointed me to this issue. I just crashed hard onto this one today. I am writing generic tree algorithms. I have: IntervalTree inherits from AVLTree inherits from BinaryTree. What I want:

T = TypeVar('T')

# What every node should have in common
class BinaryNode(Protocol[T]): ...

# Represents the actual node that is used by e.g. AVLTree
N = TypeVar('N', bound=BinaryNode)

# Parameterized on the actual node that is used and the value of it
class BinaryTree(Generic[N[T], T]):
   def rotate_left(self, node: N[T]) -> None: ...
   def add(self, value: T) -> None: ...

However, N[T] isn't valid in MyPy. No matter what I try (I even tried to just use plain functions), I can't seem to eliminate the 'higher kindness' of this structure. There is no way to extract T out of N.

If anyone knows workarounds for this pattern I'd be happy to learn them.

I've been able to work around this issue using default type parameters in python 3.13. The idea is to have both a node and a T param like so

# What every node should have in common
class BinaryNode[T]: ...



# Parameterized on the actual node that is used and the value of it
class BinaryTree[T, N: BinaryNode[Any] = BinaryNode[T]]:
   def rotate_left[_T, _N: BinaryNode[Any] = BinaryNode[_T]](self:BinaryTree[_T, _N], node: _N) -> None: ...
   def add(self, value: T) -> None: ...

Note that this is subtlety broken, as the bound on BinaryNode[T] is
not necessarily upheld, but it seems to work ok in practise

For a working example (applied to a similar array type):
https://github.com/Matt-Ord/slate/blob/main/slate/array/_array.py

@thomas-mckay
Copy link

thomas-mckay commented Dec 3, 2024

I think I may have a case that might be related to this:

class Language(Enum):
    ...

LanguageT = TypeVar('LanguageT', bound=Language)


class Params(Generic[LanguageT]):
    target_language: LanguageT

ParamsT = TypeVar('ParamsT', bound=Params)


class Tasks(Generic[LanguageT, ParamsT]):  # I'd like it to be Generic[LanguageT, ParamsT[LanguageT]]
    source_language: LanguageT
    params: list[ParamsT]  # I'd like it to be list[ParamsT[LanguageT]]

I then have several providers which all have their own Language Enum class (because each provider supports a different set of languages). Each provider also has their own params classes.

class ProviderALanguage(Language):
    ...

class ProviderAParams(Params[ProviderALanguage]):
    ...


class ProviderBLanguage(Language):
    ...

class ProviderBParams(Params[ProviderBLanguage]):
    ...

Ideally I'd need the type-checker to catch bad cases of inheritance as well as wrongful usage:

class ProviderATasks(Tasks[ProviderALanguage, ProviderBParams]):
     ...


tasks = Tasks[ProviderALanguage, ProviderBParams](...)

Currently I don't see a way to do this, but I think this proposed feature would help. It's also possible I can't see the obvious, as I've been looking at this so long...

@jorenham
Copy link

jorenham commented Dec 3, 2024

I think I may have a case that might be related to this:

class Language(Enum):
    ...

LanguageT = TypeVar('LanguageT', bound=Language)


class Params(Generic[LanguageT]):
    target_language: LanguageT

ParamsT = TypeVar('ParamsT', bound=Params)


class Tasks(Generic[LanguageT, ParamsT]):  # I'd like it to be Generic[LanguageT, ParamsT[LanguageT]]
    source_language: LanguageT
    params: list[ParamsT]  # I'd like it to be list[ParamsT[LanguageT]]

The best you could currently do is (using Python 3.12 syntax, but this is also possible with TypeVar):

class Language(Enum): ...

class Params[LanguageT: Language]:
    target_language: LanguageT

class Tasks[LanguageT: Language]:
    source_language: LanguageT
    params: list[Params[LanguageT]]

edit:

alternatively, you could also parametrize the entire params list like

class Language(Enum): ...

class Params[LanguageT: Language]:
    target_language: LanguageT

class Tasks2[LanguageT: Language, ParamsListT: list[Params[Language]]]:
    source_language: LanguageT
    params: ParamsListT

this Tasks2 is strictly more general than the Tasks in my earlier example, which you could "get back" by doing something like

type Tasks[LanguageT: Language] = Tasks[LanguageT, list[Params[LanguageT]]]

@thomas-mckay
Copy link

thomas-mckay commented Dec 4, 2024

The best you could currently do is (using Python 3.12 syntax, but this is also possible with TypeVar):

class Language(Enum): ...

class Params[LanguageT: Language]:
    target_language: LanguageT

class Tasks[LanguageT: Language]:
    source_language: LanguageT
    params: list[Params[LanguageT]]

I thought of that, but it doesn't work for me as I really need Tasks to be a Generic of ParamsT. I'm using Generics in the context of Pydantic and basically every single class in my code snippets, save for Language, is a pydantic.BaseModel. I left it out of the example out of simplicity but maybe I shouldn't have. Even without the Pydantic constraint, I lose code-completion on Tasks.params[0] (each implementation has their own attributes).

edit:

alternatively, you could also parametrize the entire params list like

class Language(Enum): ...

class Params[LanguageT: Language]:
    target_language: LanguageT

class Tasks2[LanguageT: Language, ParamsListT: list[Params[Language]]]:
    source_language: LanguageT
    params: ParamsListT

this Tasks2 is strictly more general than the Tasks in my earlier example, which you could "get back" by doing something like

type Tasks[LanguageT: Language] = Tasks[LanguageT, list[Params[LanguageT]]]

This feels like a more convoluted way of writing the first example (maybe I'm wrong, though). The type-checker (and the code completion tools) still assume Tasks.params[0] is strictly Params. And I get the same problem with pydantic.

t = Tasks[LangAImpl]()
t.params.append(ParamsAImpl())
t.params[0].  # any ParamsAImpl-specific attributes are not shown by code-completion tools

Thanks for trying to help. I'm trying to have it all, I know. I should have been more clear in that I can't get rid of the ParamsT generic-ity at the Tasks level and I'm trying to enforce a rule on the parametrization of ParamsT at the Tasks level (which is why I thought this proposed feature might be the solution). Otherwise, your examples would all work.

@jorenham
Copy link

jorenham commented Dec 4, 2024

This feels like a more convoluted way of writing the first example (maybe I'm wrong, though). The type-checker (and the code completion tools) still assume Tasks.params[0] is strictly Params. And I get the same problem with pydantic.

Exactly; I'm not trying to say that it's a solution, but that without HKT, it's the best we can currently do 🤷🏻

@joaoe
Copy link

joaoe commented Dec 4, 2024

Since Python 3.12 the new PEP 695 generic syntax is preferred over TypeVar

Ok then, here's some extra syntax.

def my_fn[T_elm: (int, float, str), T_seq: list[T_elm] | tuple[T_elm]](
    seq: T_seq[T_elm],
) -> T_seq[T_elm]:
    return seq

which nowadays fails

error: Name "T_elm" is not defined  [name-defined]
error: Type variable "T_seq" used with arguments  [valid-type]
error: Type variable "T_seq" used with arguments  [valid-type]

But this syntax is actually covered in PEP 695

# The following generates no compiler error, but a type checker
# should generate an error because an upper bound type must be concrete,
# and ``Sequence[S]`` is generic. Future extensions to the type system may
# eliminate this limitation.
class ClassA[S, T: Sequence[S]]: ...

# The following generates no compiler error, because the bound for ``S``
# is lazily evaluated. However, type checkers should generate an error.
class ClassB[S: Sequence[T], T]: ...

So it's syntax that has been suggested but not yet supported or precised.

Or we could allow the syntax I showed above, where TypeVar can be subscripted like a generic.

T_elm = TypeVar("T_elm", float, int, str)
T_seq = TypeVar[T_elm]("T_seq", list[T_elm], tuple[T_elm])

@jorenham
Copy link

jorenham commented Dec 4, 2024

Since Python 3.12 the new PEP 695 generic syntax is preferred over TypeVar

Ok then, here's some extra syntax.

def my_fn[T_elm: (int, float, str), T_seq: list[T_elm] | tuple[T_elm]](
    seq: T_seq[T_elm],
) -> T_seq[T_elm]:
    return seq

which nowadays fails

error: Name "T_elm" is not defined  [name-defined]
error: Type variable "T_seq" used with arguments  [valid-type]
error: Type variable "T_seq" used with arguments  [valid-type]

But this syntax is actually covered in PEP 695

# The following generates no compiler error, but a type checker
# should generate an error because an upper bound type must be concrete,
# and ``Sequence[S]`` is generic. Future extensions to the type system may
# eliminate this limitation.
class ClassA[S, T: Sequence[S]]: ...

# The following generates no compiler error, because the bound for ``S``
# is lazily evaluated. However, type checkers should generate an error.
class ClassB[S: Sequence[T], T]: ...

So it's syntax that has been suggested but not yet supported or precised.

I like this 👍🏻

Or we could allow the syntax I showed above, where TypeVar can be subscripted like a generic.

T_elm = TypeVar("T_elm", float, int, str)
T_seq = TypeVar[T_elm]("T_seq", list[T_elm], tuple[T_elm])

This is confusing when doing "meta-typing" of TypeVar, where you'd annotate TypeVar("T", bound=A) as e.g. T: TypeVar[A]. Even though this is all hypothetical, implementing such "meta-typing" in e.g. typeshed could potentially solve many open issues, one of them being Higher Kinded Typing itself, as I explained in #548 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: feature Discussions about new features for Python's type annotations
Projects
None yet
Development

No branches or pull requests