-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Workers crash when queue is empty #2
Comments
I can't seem to replicate. From what I see until now, the only way this can happen is if the queues used by RedisQueue internally might be tampered with external data. At some point, I had something in mind to check and move those kind of invalid values to some "bad-data-queue"; I'll try to do it now. |
That may actually be what's happening. I'm using your library in conjuction with jimbojsb/workman, which creates manages/thread workers: https://github.com/jimbojsb/workman I would have assumed each individual worker would grab its queue item, and leave the others alone. It seems once the queue is "done" all the individual works start crashing out, with the above error. When I run this single-threaded, I never get the error. Only happens with the multi-threaded approach. |
Yes, indeed problems might occur if one would start a worker by forking it. So you can start several workers and then just queue tasks. |
That is what I do. I have a separate worker script, that is designed to work just as single worker, and a queue loader that works as queue loader. Obviously you can't run the queue loader as a fork, or things will be stepping on each other. ;) |
After queue is completed, workers commit suicide:
Actually, can't exactly pin-point the exact instance this occurs. It used to happen all the time after the queue finished processing... but now it's not erroring. hrmm.
The text was updated successfully, but these errors were encountered: