gh-128041: Add a terminate_workers method to ProcessPoolExecutor #128043
Conversation
Provides a way to forcefully stop all the workers in the pool Typically this would be used as a last effort to stop all workers if unable to shutdown / join in the expected way
|
All commit authors signed the Contributor License Agreement. |
csm10495
force-pushed
the
terminate_workers
branch
from
ae41bf7 to
61c9b14
Compare
Misc/NEWS.d/next/Library/2024-12-17-18-53-21.gh-issue-128041.W96kAr.rst
Outdated
Show resolved
Hide resolved
|
Thanks @picnixz, I think I resolved all of your comments, please check again when you can. |
|
Hey @gpshead mind taking a look? Ref https://discuss.python.org/t/cancel-running-work-in-processpoolexecutor/58605/2 |
| @@ -855,3 +856,31 @@ def shutdown(self, wait=True, *, cancel_futures=False): | |||
| self._executor_manager_thread_wakeup = None | |||
|
|
|||
| shutdown.__doc__ = _base.Executor.shutdown.__doc__ | |||
|
|
|||
| def terminate_workers(self, signal=signal.SIGTERM): | |||
There was a problem hiding this comment.
Actually, can we perhaps name it kill? (namely, you kill the pool's workers with the given signal) or is kill already taken (possibly for something else in the future)?
There was a problem hiding this comment.
I'm not sure. I thought this was explicit. I'm open for either or.
There was a problem hiding this comment.
Ok, so after some research:
Process.terminate-- mimicsSIGTERMProcess.kill-- sendsSIGKILLto the process.
Many other interfaces use terminate for sending SIGTERM and kill for sending SIGKILL. Now, we also have multiprocessing.Pool.terminate, so maybe we could mimic it by just naming it terminate(signal=...)? It would be slightly inconsistent with the others terminate methods since this one would be able to send a SIGKILL as well.
So it's up to you (or up to Gregory)
There was a problem hiding this comment.
I could definitely be down for terminate to match multiprocessing.Pool. (Though i'd still like to be able to change the signal sent so it would be a slight api difference). I'll wait to see what Gregory thinks to either change it or leave as is.
Co-authored-by: Bénédikt Tran <[email protected]>
Co-authored-by: Bénédikt Tran <[email protected]>
Provides a way to forcefully stop all the workers in the pool
Typically this would be used as a last effort to stop all workers if unable to shutdown / join in the expected way.
terminate_workerstoProcessPoolExecutor#128041