opt-in exception handling specification

[remdragon]

This is a feature request.

I'd like to see opt-in exception specification system in mypy.

Knowing what Exceptions a function can throw is one of the biggest sources of unexpected crashes in the wild for me.

Not everyone would want this, so it would have to be explicitly enabled with a flag in mypy.

what I'm proposing is that if mypy is run with the hypothetical --report-uncaught-exceptions the following scenarios would be caught:

###foo1.py:
def foo ( n: int ) -> int:
    return 1 // n
###
mypy: foo1.py:2: warning: uncaught ZeroDivisionError exception ( wrap in a try block or declare @raises(ZeroDivisionError)

###foo2.py
from typing import raises
@raises(ZeroDivisionError)
def foo ( n: int ) -> int:
    return 1 // n
foo ( 1 )
###
mypy: foo2.py:5: warning: uncaught ZeroDivisionError exception from foo2.foo()

maybe have a --report-uncaught-exceptions2 which can catch exception generalizations:

###foo3.py
from typing import raises
@raises(Exception)
def foo ( n: int ) -> int:
   return 1 // n
###
mypy: foo3.py:4: warning: ZeroDivisionError caught by more generic @raises(Exception)

At runtime, @raises() would be a pass-through

I'm open to suggestions on a better way to do this. What I've presented is just my first thought on how this could be handled.

I would be willing to attempt to create a PR for this if it's an acceptable feature, but I would need some guidance as to what parts of mypy would need hacking.

I'm not the only one that wants something like this:
#6091

I realize this is a big project as it would require adding @raises to a bunch of functions in typing. However, since this proposal is an opt-in only feature, those changes can be added over time.

Another challenge I can foresee is that pypy may not raise the same exceptions as cpython in certain cases.

Also (although this may not be in the scope of mypy), some products like pyinstaller modify the exception types that are thrown.

[ilevkivskyi]

Just to note: the amount of work for this needed in typeshed is enormous. I think there was a discussion on the tracker about some alternative approaches recently. I think @JukkaL should remember the details.

[JukkaL]

We've discussed a related approach in #6936. It would be opt-in and exceptions from libraries would not be checked by default. I don't think full coverage of all exceptions is practical, since I doubt anybody will have the time or patience to annotate exceptions raised by hundreds of library modules. Also, exceptions such as IndexError would be very painful to handle explicitly, so we'd probably need a concept of unchecked exceptions anyway. If coverage is very incomplete, exception checking may not be very useful.

We'd probably need an accepted PEP before we could start annotating exceptions in typeshed, since it's used by multiple tools and exception annotations would be a big change. There would be questions about which exceptions should be annotated (IndexError?), etc.

If you still want to pursue this, I'd suggest reading all comments in #6936 first. If you are not convinced by them, you can argue why that approach is worse than what you are proposing. We should first have a rough consensus before starting an implementation, since this is a very complex topic, and even if we'll eventually agree on an approach, it could be quite different from what has been proposed so far.

[DrJackilD]

@JukkaL actually, the solution described in #6936 is much different than just possible exception hinting. It dictates a different style of programming, which is unpythonic, as @gvanrossum mentioned at the end of the thread. But what if developers have the mechanism to leave optional hints for functions mentioned that they are able to throw an exception during execution? For example, that could be achieved simply with # type: ... comments in the function. A little example:

def foo (n: int) -> int:  # type: raises ZeroDivisionError
    return 1 // n

That will give us the ability to check that somewhere in the client code we possibly could get this error

def driver_code(inut: int):
    return foo(input)

# mypy warning
mypy: foo2.py:5: warning: uncaught ZeroDivisionError exception from driver_code.foo()

and if we'd like to ignore it, we could simply add something like # type: unwrap commend, which leaves this line out of checks.

def driver_code(inut: int):
    result = foo(input)  # type: unwrap
    return result

That could help developers to achieve more fault-tolerant code and don't have a large overhead with a special return type or syntax-related things in Python itself

[DrJackilD]

As an alternative, maybe we could extend NoReturn with the possibility to add exception types? Then we could use it for typing like this:

def divide(a: int, b: int) -> int | NoReturn[ZeroDivisionError]:
    return a // b


def main():
    res = divide(5, 4)
    total = 5 + res  # mypy will complain here because NoReturn type can't be using with integer
    print(total)

That looks like a very soft extension for the existing ecosystem. And with Python 3.10 new union style, it looks very cute :) Of course, that's not a question for Mypy then, but for typing module.

UPD: Actually, I just realize, that NoReturn isn't a type but a type alias for Union[None]. I honestly don't know how this affects the whole idea of using this type alias to indicate a possible exception, but maybe that's not a problem?

[hauntsaninja]

Duplicate of #1773 (either way, mypy is extremely unlikely to do this)