Send custom messages from JS to Python

[domoritz]

https://anywidget.dev/en/jupyter-widgets-the-good-parts/#2-custom-messages explains how to send custom messages to the frontend but is there an equivalent to send data to the backend? I know traitlets that are synchronized can be used for this but I wonder whether there is a way to send messages that I don't need to synchronize back to the frontend.

[manzt]

Yes! You can use the DOMWidgetModel.send method in the frontend to send messages from JS to Python that don't need synchronization. This works for any Jupyter Widget, but you can try it out easily with anywidget:

import anywidget

class CustomMessageWidget(anywidget.AnyWidget):
    _esm = """
    export function render(view) {
        setInterval(() => {
            view.model.send({ foo: "bar" });
        }, 2000);
    }
    """
        
widget = CustomMessageWidget()

@widget.on_msg
def do_something(instance: CustomMessageWidget, data: dict, buffers: list):
    print(f"{instance=}, {data=}, {buffers=}")
    
widget

Screen.Recording.2023-02-05.at.11.03.35.PM.mov

It's worth noting that you should be able to send not only JSON-serializable data, but also binary data from the client to Python (that's what the buffers should be populated with in a custom message hanlder), but there is currently a bug preventing this. I plan to implement support in anywidget in the meantime.

[manzt]

Also worth noting that I haven't found a nice pattern in this framework to block/wait for a response from the front end for different messages. I.e.,

class MyWidget(anywidget.AnyWidget):
    _esm = """
    export function render(view) {
        let api = /* ... */ 
        view.model.on("msg:custom", async msg => {
          if (msg.type === "export-png") {
            let bytes = await api.exportPng();
            view.model.send({ type: "export-png", data: new DataView(bytes.buffer) });
          }
        })
    }
    """
    def save_png(self):
        self.send({ type: "export-png" })
        # somehow wait for response ...

• Question: How to wait (in a blocking way) for messages from widget frontend? jupyter-widgets/ipywidgets#3039

If we found some of general pattern that makes this RPC-like stuff work nicely, I'd be more than happy to add something to anywidget's core.

[domoritz]

Thank you! That's perfect.

I actually thought about RPC/blocking messages as well as I will need it in my app. However, I want to go the other way around and request something from the Python side. I came up with a hack where messages get an ID and the frontend would keep a map of promises that it resolves when the response with a particular id arrives. It feels dirty, though, so if you have a cleaner solution, I would be the first to adopt it.

Anyway, closing this issue since you provided the solution. One thing I am not getting yet is how to listen to messages in the class rather than a specific instance (@widget.on_msg only works when you already have an instance of the widget).

[manzt]

However, I want to go the other way around and request something from the Python side. I came up with a hack where messages get an ID and the frontend would keep a map of promises that it resolves when the response with a particular id arrives.

Ha, yes I've come up with something very similar for this! Maybe worth adding a short blog post in the docs, but nothing official.

One thing I am not getting yet is how to listen to messages in the class rather than a specific instance (@widget.on_msg only works when you already have an instance of the widget).

You should be able to add an on_msg handler in the constructor of your widget.

class MyWidget(anywidget.AnyWidget):
    _esm = " .... "

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.on_msg(self._handle_custom_msg)

    def _handle_custom_msg(self, data, buffers):
        ...

[domoritz]

I think it's

...
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.on_msg(self._handle_msg)

    def _handle_msg(self, message):
        print(f"{message=}")

[manzt]

Ah! Actually, _handle_msg is implemented on the base class (ipywidgets.Widget._handle_msg), so this example breaks all the other functionality implemented by the widget. Sorry about that. We should use a different name for the custom message handler:

import anywidget

class Button(anywidget.AnyWidget):
    _esm = """
    export function render(view) {
      let btn = Object.assign(document.createElement("button"), { innerText: "Click me" });
      btn.addEventListener("click", () => { view.model.send("ping") });
      view.el.appendChild(btn);
    }
    """

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.on_msg(self._handle_custom_msg)

    def _handle_custom_msg(self, data, buffers):
        print(f"{data=}, {buffers=}")

Button()

(I'm editing the snippet above so that others do not accidentally make this mistake).

[domoritz]

Ahh, good catch. Thanks!