Container.d filling up /var causing disk pressure and evictions

[kristofre]

Hi!

I installed k3s on a VM where I have a mount point /app that has 30GB. That worked out fine for the binaries, by specifying INSTALL_K3S_BIN_DIR.

However, containerd is filling up /var causing disk pressure on the node and eventually eviction of pods.

I attempted to configure containerd using the /var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl and set in there the root and state parameters to a subfolder of /app. After a restart of k3s that does not seem to have changed anything.

Anybody have any experience with this and/or can tell me where to start looking?

Thanks!

[brandond]

You might look at the --data-dir flag, as opposed to trying to tweak containerd's configuration directly.

[kristofre]

Thanks @brandond . Apologies for the beginner question, but where do I set that flag? Thanks in advance!

[brandond]

Changing the data-dir should be done before initial startup, since it determines paths for things that are created during initial cluster startup. While you could possibly manually move things around, that's not something I would recommend a beginner attempt. I would probably uninstall and reinstall k3s, passing the --data-dir flag to the install script.

[kristofre]

Thanks @brandond, reinstall did the trick.