Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow comm message to be processed during cell execution #65

Open
dsblank opened this issue Oct 10, 2015 · 4 comments
Open

Allow comm message to be processed during cell execution #65

dsblank opened this issue Oct 10, 2015 · 4 comments
Milestone

Comments

@dsblank
Copy link
Contributor

dsblank commented Oct 10, 2015

The use case: a Python function getMouse() that blocks waiting for the user to click on an SVG image that has been wired up to a trait to pass the x,y value back to Python. We would like getMouse() to return that x,y.

The implementation we are thinking of would allow comm messages to be processed during the cell execution. I think this is general to any comm_msg, but our goal is just the mouse widget. Here is a little version of the idea:

from ipywidgets import widgets
from IPython.display import display
i = widgets.IntText()
display(i)
i.value = 78
while i.value == 78:
    pass

We would like to be able to change the value in the displayed IntText (for example) and have the loop stop when the detected value changes. We understand that jupyter is not designed to do this, but we would like to replicate a pre-existing API in the notebook, getMouse().

Is it possible, while the cell is executing, to process any queued comm messages? Thinking of something like:

while i.value == 78:
    kernel = get_ipython_kernel()
    kernel.process_comm_msgs()

Any help or alternative ideas welcomed!

@minrk
Copy link
Member

minrk commented Oct 12, 2015

I think this can be handled by processing messages in a thread while running code in another. What complicates things is that comm messages come on the same queue as execution, but I think we can deal with that if we are careful. I've been designing the threaded IO in my head for a bit, because I think it would solve several problems with flushing stdout as well.

@dsblank
Copy link
Contributor Author

dsblank commented Oct 12, 2015

@minrk Thanks for the time on this issue. BTW, when we implemented our backend in C#, we execute the cell code in the background:

https://bitbucket.org/ipre/calico/src/master/Source/Calico/ZMQServer.cs?fileviewer=file-view-default#ZMQServer.cs-964

I thought that that was the way it was supposed to work. I don't remember any adverse effects, but I think we made sure that the kernel acted as if it were blocking (didn't accept another cell execution message, for example). I do remember testing parallel cell execution, which worked, if you keep the output cells associated with the execution. Although, that can be a confusing UI. It might be nice to have a magic that would allow cell code to run in background, and not block. No need to write specific code to have it run in a thread if it is already running in a thread.

@AlexTugarev
Copy link

@minrk I'm glad, I found this issue, because I also need to handle comm messages in background. I'm about to prototype a kernel based on IPythonKernel which processes comm messages in another thread. My plan is to use two queues for comm and other messages, which would be polled in different threads. Please share your ideas on the treaded IO, maybe it's helpful.

Also I'd like to create a PR once it passes existing tests, and it would be great to receive feedback on this.

@oeway
Copy link

oeway commented May 16, 2020

I recently encountered the same issue, in the cell we run something like await prompt("x=?") and the front end popup a dialog to get user input, once done, it sends a comm message to the backend. However, because the cell is running, the comm message cannot go through, it ended up with a dead state that we can only send an interrupt execution command to end it.

@AlexTugarev do you have a chance to work on a solution? Two queues for comm and other messages sounds like a good idea.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants