Stop saying mypy is beta software?

[msullivan]

We're tagged as alpha in pypi and in our readme we say "Mypy is alpha software, but it has already been used in production for well over a year at Dropbox, and it has an extensive test suite."

While mypy certainly has its flaws I, I think it is pretty well established that it is usable in a serious way, and in fact we pretty seriously recommend it as such.

EDIT (by Ethan): We now say we are beta, should we say we are production ready?

[ilevkivskyi]

I think we can call it beta. Some people define beta when software is feature-complete w.r.t. subsequent major release, but IIUC this is rather a tradition than a specification.

[gvanrossum]

We're being very self-deprecating here. I would not hesitate calling mypy production quality, i.e. beyond beta.

[emmatyping]

I agree at minimum that mypy is beta quality, if not better. When adding mypy typing to a project recently, a friend was surprised mypy hasn't hit 1.0 yet! I do think we should explain that backwards incompatible changes may still happen, but we try not to do so?

[Michael0x2a]

It'd be nice if we could align the announcement that mypy is now production-quality with a somewhat impressive release. Maybe we should just silently switch over to calling ourselves "beta" for now and wait one or two releases before announcing that mypy is production-quality/that we're releasing v1.0? We could perhaps combine that announcement with the "we have a new semantic analyzer" release or something.

This is only a suggestion though -- I don't really have a strong opinion on this apart from agreeing that we're definitely past the alpha stage.

[JukkaL]

@Michael0x2a I like your idea!

[wasilukm]

Hi, could you please consider also switching to semantic versioning?

[emmatyping]

@wasilukm AIUI the plan is to switch to semver at 1.0. So if we go ahead with @Michael0x2a's proposal that will be in a couple of releases.

With regards to Michael's plan, my only concern is that the new semantic analyzer should probably be battle tested for a couple of releases, as I worry we'd release 1.0 then have several bugs crop up, which would weaken the stability message of a 1.0.

[JukkaL]

Applying semantic versioning is not trivial, since we'd need to document what the public API of mypy is. The command-line interface is well documented, but it's not clear which APIs plugins can safely depend on.

Even with semantic versioning, there would likely be no guarantees about new non-major version of mypy not generating extra errors, since many bug fixes, features and stub updates focus on turning false negatives into errors (and these changes may sometimes also cause false positives).

[gvanrossum]

Let’s at least call it beta everywhere.

I agree that the definition of semver will always be problematic.

So as an alternative to semver we could number releases by date, i.e. calver.

[emmatyping]

In #6962 we changed to a beta label. The new semantic analyzer is out, so I suppose in the next release we could consider marking mypy as stable. I have a few concerns/thoughts around this. First, I personally feel like waiting another release to let the new semantic analyzer percolate would be better. I don't care strongly about it though.

Once we do decide to become stable I think we should either use CalVer or do a major version bump with semantic versioning, because I don't think we will be able to make sure typeshed updates don't break things.

I'm interested in the thoughts of others!

[ilevkivskyi]

@ethanhs I think it is a bit premature, we need to stabilize plugin API (and therefore a bunch of other internals). Plus I would like to see recursive types, shape types, and variadic generics (three most requested features) have a basic support. I think we can aim at releasing something like "1.0" in time for PyCon 2020.

[emmatyping]

@ilevkivskyi ok that makes a lot of sense. I'm particularly eager for a 1.0, so I'm more than happy to wait

[JukkaL]

I don't think that recursive types, shape types and variadic generics are necessary for a 1.0 release -- they can be introduced in later feature releases. There will probably always be missing features that a lot of users would like to have. I agree that the plugin API should be more stable and maybe better documented. There may also be some high-priority issues that we should fix before 1.0. For example, if the fix could break backwards compatibility in non-trivial ways, we might want to do it before 1.0. I'm thinking of changes to type inference, in particular, but there may be other things. We should at least review all high-priority issues.

[JukkaL]

I'd like to make the 1.0 release in the not-too-distance future. Here are things that would be nice to have by then:

• Make --local-partial-types enabled the default (or maybe always). This is necessary to make daemon and non-daemon modes behave in a similar way. (PR: Enable local partial types by default #10169)
• Support following imports in dmypy. (Daemon should respect the rule that stubs are normal even with follow-imports=skip #5566 is related) (DONE)
• Type narrowing with := (PEP 572: Type not narrowed correctly in assignment #7316). (DONE)
• Better Windows testing (Test running dmypy compiled on Windows before a release #7034).
• --disallow-untyped-calls and Any callee (--disallow-untyped-calls allows calling values with Any type #5968).

I'd like to make some release process related changes as well:

• Introduce a deprecation period before removing features (6 months, for example).
• Announce major planned plugin API related changes at least one release ahead of making the change, so that plugin authors have time to prepare and give us feedback on the planned change.
• Make a bugfix release if there is any significant regression. Currently we sometimes wait for the next major release even if the regression is fairly significant.
• Increase the major version number if there is a major backward compatibility break (or a big new feature, perhaps).

In the long term, it would be nice have periodic long term support releases, but this can happen later.

[ilevkivskyi]

A good target may be PyCon 2020. I think we should fix two clusterbugs before:

• Fix all the checkmember and related inheritance issues, see Refactor all member access to go through checkmember.py (meta issue) #7724
• Fix type inference logic: leaking type variables, overusing type context, generic decorators not working etc, see my long comment in Generic inference regression from #5699 #5738

In general, I think we should dedicate some significant time (like a month) to bug fixing before doing this. In particular, it would be good to fix some historical stuff like not giving an error on empty bodies (well, this particular one I am going to fix myself soon, this is just an example).

I would like mypy 1.0 to be in the state where one can safely assume that if mypy rejects/accepts something it is actually wrong/correct code, not a bug in mypy. The problem is that many newcomers can't easily distinguish mypy bugs/quirks from genuine type errors, so they will have hard time building (and constantly re-building) mental model for how mypy works. In current state it is pretty easy for person to start using mypy and immediately hit a bug.

[JukkaL]

Those are also good suggestions. The main question is time -- it's unclear when we'll be able to devote enough time to fix all the big issues.

I'd be prepared to release 1.0 even if there are various type inference bugs. I believe that only a small fraction of issues cause significant trouble to a significant fraction of users, and it would be sufficient to fix (most of) them. Identifier those issues is clearly non-trivial.

Another minor issue that would be nice to fix: generate an error in variable annotation is used in an unchecked function (#3948).

[TH3CHARLie]

Recently, there are more and more issues on recursive types like #8320 and #6730. Reporting errors instead of crashes is a reasonable solution for these issues, at least for now. However, adding more and more error handling code requires a detailed analysis of each bug and makes the codebase fragile.

As there are some discussions about recursive types, I'd like to know about the current status of the progress on recursive types. It would be nice to have them. Is anyone working on the topic? Is there any available design for implementing recursive types? @JukkaL @msullivan @ilevkivskyi

Updates: I've found some links in #731 and therefore some PRs by @ilevkivskyi: #7885 #7366 #7330. I'll see what I can do following the existing work.

[ilevkivskyi]

Is anyone working on the topic?

I don't think so.

[JukkaL]

Is there any available design for implementing recursive types?

I started documenting the design in #731 (comment).

[TH3CHARLie]

I started documenting the design in #731 (comment).

Thanks! That's super useful! I'll see if there's anything I can to help push recursive types forward

[dmtucker]

FWIW, I would advocate for semver at 1.0.0+.

Even with semantic versioning, there would likely be no guarantees about new non-major version of mypy not generating extra errors, since many bug fixes, features and stub updates focus on turning false negatives into errors (and these changes may sometimes also cause false positives).

This feels a little like "sacrificing good in the name of perfect". Semver adherence is always best-effort IME, and whether a new check is truly new (i.e. good for a minor release) or is a fixed version of an existing check (i.e. good for a patch release) is a case-by-case judgment call.

Also, people can use pinning to get the same effect that calver would likely have.

[srittau]

While Semver is great for libraries and other things that provide an API, it really doesn't make much sense for type checkers (or other applications). In fact, it is unclear, how Semver would apply to mypy as Semver is defined in terms APIs. mypy is likely to improve checks with every non-bugfix release, which has the potential to break some code.

[dmtucker]

I was under the impression mypy's API was its CLI (plus mypy.api.run):

mypy/mypy/api.py

Lines 3 to 4 in e4b4959

	Since mypy still changes, the API was kept utterly simple and non-intrusive.
	It just mimics command line activation without starting a new interpreter.

New checks in non-bugfix releases sounds appropriate to me... Given X.Y.Z:

• Bump X for breaking changes (i.e. things that may cause mypy integration to stop working / not even run).
• Bump Y for new checks and other new features (i.e. things that won't break integration, but might clash with existing code).
• Bump Z for bug fixes (i.e. things that downstream should always take).

As another data point, I believe this is what flake8 & co. does:

Flake8 uses ranges for mccabe, pyflakes, and pycodestyle because each of those projects tend to add new checks in minor releases. It has been an implicit design goal of Flake8’s to make the list of error codes stable in its own minor releases. That way if you install something from the 2.5 series today, you will not find new checks in the same series in a month from now when you install it again.

[jab]

While Semver is great for libraries and other things that provide an API

Not even. I think the reality is more complicated. :) There's some good discussion in python-trio/trio#1182 (among other places).

[ulidtko]

Let's not forget that mypy has a plugin API.

Here's one example of changes in that API causing breakage downstream: jspahrsummers/adt#21

Such breakage is fine, it is to be expected given the "beta status" of mypy — which is also clearly communicated through 0.nnn versions. For mypy 1.x.y, the expectation is different. Is the plugin API ready for stabilization?

Further, let me postulate that the impact of changes in mypy output (brought via changes in typechecking logic, e.g. additional checks) is no worse than the impact of breaking the plugin API. Said another way: the people to become upset with semver-disregarding updates of mypy will more likely be plugin authors than the end-users. (Feel free to dispute; I'm still not a heavy user of mypy, and this isn't a particularly well-educated guess by far).

That is to say: please adhere to SemVer, it helps other engineers get through issues faster.

[hauntsaninja]

With our current versioning scheme, and our aspirations of a regular release schedule, we seem to be headed for a 1.0 at some point in the foreseeable future (I guess we could have 0.1000, but that would only cause confusion). We could also change our versioning scheme, for instance making releases like 0.920.1 to free up 0.925 for a non patch release, or switching to calver.

I don't really care about a "beta" label or pre-1.0 version number; mypy is great and I use it every day. But since this issue has been used to talk about things we'd like to see fixed in mypy, here's my list: #11201

[ilevkivskyi]

Superseded by #13685