PEP 724: Stricter TypeGuard

.github/CODEOWNERS

@@ -601,6 +601,7 @@ pep-0719.rst  @Yhg1s
 pep-0720.rst  @FFY00
 pep-0721.rst  @encukou
 pep-0722.rst  @pfmoore
+pep-0725.rst  TBD
 # ...
 # pep-0754.txt
 # ...

pep-0725.rst

@@ -0,0 +1,236 @@
+PEP: 725
+Title: Stricter Type Guards
+Author: Rich Chiodo <rchiodo at microsoft.com>, Eric Traut <erictr at microsoft.com>, Erik De Bonte <erikd at microsoft.com>
+Sponsor: <real name of sponsor>
+PEP-Delegate: <PEP delegate's real name>
+Discussions-To: https://mail.python.org/archives/list/[email protected]/thread/EMUD2D424OI53DCWQ4H5L6SJD2IXBHUL/
+Status: Draft
+Type: Standards Track
+Topic: Typing
+Content-Type: text/x-rst
+Created: 28-Jul-2023
+Python-Version: 3.10
+Post-History: 
+Resolution: 
+
+
+Abstract
+========
+
+:pep:`647` introduced the concept of ``TypeGuard`` functions which return true
+if their input parameter matches their target type. For example, a function that
+returns ``TypeGuard[str]`` is assumed to return ``true`` if and only if it's
+input parameter is a ``str``. This allows type checkers to narrow types in this
+positive case.
+
+This PEP further refines :pep:`647` by allowing type checkers to also narrow types
+when a ``TypeGuard`` function returns false.
+
+Motivation
+==========
+
+``TypeGuards`` are used throughout Python libraries to allow a type checker to
+narrow the type of something when the ``TypeGuard`` returns true.
+
+However, in the ``else`` clause, :pep:`647` didn't prescribe what the type might
+be:
+
+.. code-block:: python
+
+    def is_str(val: str | int) -> TypeGuard[str]:
+        return isinstance(val, str)
+
+    def func(val: str | int):
+        if is_str(val):
+            # Type checkers can assume val is a 'str' in this branch
+        else:
+            # Type here is not narrowed. It is still 'str | int' 
+
+
+This PEP proposes that when the type argument of the ``TypeGuard`` is a subtype
+of the type of the first input parameter, then the false return case can be
+further narrowed. 
+
+This changes the example above like so:
+
+.. code-block:: python
+
+    def is_str(val: str | int) -> TypeGuard[str]:
+        return isinstance(val, str)
+
+    def func(val: str | int):
+        if is_str(val):
+            # Type checkers can assume val is a 'str' in this branch
+        else:
+            # Type checkers can assume val is an 'int' in this branch
+
+Since the ``TypeGuard`` type (or output type) is a subtype of the first input
+parameter type, a type checker can determine that the only possible type in the
+``else`` is the other type in the ``Union``. In this example, it is safe to
+assume that if ``is_str`` returns false, then type of the ``val`` argument is an
+``int``.
+
+
+Specification
+=============
+
+This PEP requires no new changes to the language. It is merely modifying the
+definition of ``TypeGuard`` for type checkers. Runtimes are already behaving
+in this way.
+
+
+Existing ``TypeGuard`` usage may change though, as described in 
+`Backwards Compatibility`_.
+
+Unsafe Narrowing
+--------------------
+
+There are cases where this further type narrowing is not possible though. Here's
+an example:
+
+.. code-block:: python
+
+    def is_str_list(val: list[int | str]) -> TypeGuard[list[str]]
+        return all(isinstance(x, str) for x in val)
+
+    def func(val: list[int | str]):
+        if is_str_list(val):
+            # Type checker assumes list[str] here
+        else:
+            # Type checker cannot assume list[int] here
+
+Since ``list`` is invariant, it doesn't have any subtypes. ``list[str]`` is not
+a subtype of ``list[str | int]``. This means type checkers cannot narrow the
+type to ``list[int]`` in the false case.
+
+Type checkers should not assume any narrowing in the false case when the
+``TypeGuard`` type argument is not a subtype of the first input parameter type. 
+
+However, narrowing in the true case is still possible. In the example above, the
+type checker can assume the list is a ``list[str]`` if the ``TypeGuard``
+function returns true.
+
+User error
+--------------------------
+
+The new ``else`` case for a ``TypeGuard`` can be setup incorrectly. Here's an
+example:
+
+.. code-block:: python
+
+    def is_positive_int(val: int | str) -> TypeGuard[int]:
+        return isinstance(val, int) and val > 0
+
+    def func(val: int | str):
+        if is_positive_int(val):
+            # Type checker assumes int here
+        else:
+            # Type checker assumes str here
+
+A type checker will assume for the else case that the value is ``str``. This is
+a change in behavior from :pep:`647` but as that pep stated, there are many ways
+a determined or uninformed developer can subvert type safety.
+
+A better way to handle this example would be something like so:
+
+.. code-block:: python
+
+    PosInt = NewType('PosInt', int)
+
+    def is_positive_int(val: PosInt | int | str) -> TypeGuard[PosInt]:
+        return isinstance(val, int) and val > 0
+
+    def func(val: int | str):
+        if is_positive_int(val):
+            # Type checker assumes PosInt here
+        else:
+            # Type checker assumes str | int here
+
+
+Backwards Compatibility
+=======================
+
+For preexisting code this PEP should require no changes.
+
+However, some use cases such as the one below can be simplified:
+
+.. code-block:: python
+
+    class A():
+        pass
+    class B():
+        pass
+
+    def is_A(x: A | B) -> TypeGuard[A]:
+        return is_instance(x, A)
+
+
+    def is_B(x: A | B) -> TypeGuard[B]:
+        return is_instance(x, B)
+
+
+    def test(x: A | B):
+        if is_A(x):
+            # Do stuff assuming x is an 'A'
+            return
+        assert is_B(x)
+
+        # Do stuff assuming x is a 'B'
+        return
+
+
+With this proposed change, the code above continues to work but could be
+simplified by removing the assertion that x is of type B in the negative case:
+
+.. code-block:: python
+
+    class A():
+        pass
+    class B():
+        pass
+
+    def is_A(x: A | B) -> TypeGuard[A]:
+        return is_instance(x, A)
+
+
+    def test(x: A | B):
+        if is_A(x):
+            # Do stuff assuming x is an 'A'
+            return
+
+        # Do stuff assuming x is a 'B'
+        return
+
+
+How to Teach This
+=================
+
+The belief is that new users will assume this is how ``TypeGuard`` works in the
+first place. Meaning this change should make ``TypeGuard`` easier to teach.
+
+
+Reference Implementation
+========================
+
+A reference `implementation <https://github.com/microsoft/pyright/commit/9a5af798d726bd0612cebee7223676c39cf0b9b0>`__ of this idea exists in Pyright.
+
+
+Rejected Ideas
+==============
+
+Originally a new ``StrictTypeGuard`` construct was proposed. A
+``StrictTypeGuard`` would be similar to to a ``TypeGuard`` except it would
+explicitly state that output type was a subtype of the input type. Type checkers
+would validate that the output type was a subtype of the input type.
+
+See this comment: `StrictTypeGuard proposal <https://github.com/python/typing/discussions/1013#discussioncomment-1966238>`__
+
+This was rejected because for most cases it's not necessary. Most people assume
+the negative case for ``TypeGuard`` anyway, so why not just change the
+specification to match their assumptions?
+
+Copyright
+=========
+
+This document is placed in the public domain or under the CC0-1.0-Universal
+license, whichever is more permissive.