Handling dynamic load with MRSK

[nickchomey]

The primary reason for MRSK appears to be to enable easily deploying containers to considerably lower-cost owned or rented bare metal servers, low-cost vps etc...

However, these options typically don't have much capability for any degree of autoscaling (be it scheduled, load-driven, etc...), which many applications surely benefit from having. Perhaps it's a moot point when using MRSK, given the significant cost savings available from moving off-cloud - just maintain significant excess capacity so as to give you time to absorb traffic spikes and manually add extra capacity as needed (which can be done in minutes with most VPS providers).

Still, there's surely a place for auto-scaling but MRSK does not and will not have the capability to do any sort of kubernetes-like cloud auto-scaling. In fact, that's perhaps the entire point of MRSK - be able to deploy containers to a cluster without the complexity of kubernetes!

So, I'm curious what approaches people might have to achieve some degree of dynamic/auto scaling with MRSK.

My first thought is to simply (as the readme largely suggests) maintain a low-cost baseline and then link to one or more EC2/VM instances and use the cloud provider's autoscaling tools to scale the capacity. That seems relatively straight forward for something like scheduled scaling, which you would set based on a known pattern.

But it's not clear to me how it would work for demand-driven scaling to respond to unexpected traffic spikes. How would we tell MRSK/Traefik to saturate the low-cost baseline hardware (perhaps to a preset threshold, say 90% CPU, for a particular duration) before sending traffic to the high-cost, flexible cloud hardware (which would then scale itself to your defined limit/infinity) to accommodate demand.

Anyone have suggestions or other ideas?

[gregmsanderson]

I just came across MRSK and was wondering about this too. However as far as I can see there is no on-server daemon/process that could monitor such things to know when to do it. I wondered about whether containers got automatically restarted upon a failed healthcheck and that appears to have this same issue: there is no control plane/equivalent to monitor/handle that either.

Maybe there is a place for a "simpler" k8s that is basically MRSK plus restarting/auto-scaling 🤔. However ... with each addition, you add complexity ... and back to recreating k8s again.

[nickchomey]

Yeah, that's the crux of it - just use kubernetes if we need that sort of automated functionality...

But, I like the idea of some sort of health check daemon that maybe could send email notifications so that you can inspect and manually respond accordingly. Surely some tooling already exists for such a thing?

[gregmsanderson]

Maybe the HEALTHCHECK that Dockerfile already supports (https://docs.docker.com/engine/reference/builder/#healthcheck) could be used to do that? A container can detect if it's unhealthy and so can take action. Maybe email you there?

Personally I'd want it to automatically restart, as inevitably it will happen at 3am. Rather than need manual intervention via email. I guess if the container was set to restart always then the healthcheck could kill it on a fail ... and then it would be restarted. Hmm ... It's trying to add orchestration without there actually being any.

[christianbpedersen]

My first thought when seeing mrsk was that if you're moving to your own bare metal servers, autoscaling is probably a mute point, with the idea being they so much cheaper than the equivalent EC2 instances, that you just let them all hum, all the time. And then use something like raindrops or equivalent to figure out your actual utilization and alert you when your assumption of max needed capacity was wrong.

But I guess if you really really have to, and you were in AWS despite what the man said, you could construct an EC2 AMI to self register with something that updates the deploy.yml file and kicks off a deploy when the file changes, and have mrsk setup on a tiny ass instance running alongside for controlling the damn thing. That is probably the most low tech solution for hardware scaling I can think of that would fit the spirit of mrsk.

Your milage may vary.

[benedikt-roth]

Another option would be to go hybrid and host some of your workloads on a cloud provider. Integrate it using things like PrivateLink to share resources - if needed (comes with some overhead and tradeoffs of course.) Once you are able to do proper capacity planning for the workload, you can go and move it onto your bare metal servers.

Heavier tenants can get their own tenant in the cloud if scalability is a concern. But then again, DHH recently had a nice podcast where they were talking about having Twitter as a client which was waay to big and they were glad when they left. Depends on your customer strategy.

To me it is all about capacity planning. The cloud is great to figure that out. Once you know, go away from the cloud.