self-hosted agents not removing after job finishes

[BenH-Puregym]

I have a self-hosted agent setup as per the documentation here.
I've added run once to the script as per documentation here however sometimes it removes the agent and sometimes it doesn't which results in lots of random offline agents sitting there waiting for manual removal.

The whole point of adding this run once is so that when the job finishes it removes the agent.

[astlock]

The documentation says that the agent will be stopped, not removed. Some script from the outside should handle the removal from the pool. We do it like this.

${AZP_HOME}/run.sh --once
shutdown

function shutdown()
{
  cd ${AZP_HOME}
  echo "$(date) ${AZP_HOME}/config.sh remove"
  ${AZP_HOME}/svc.sh uninstall
  ${AZP_HOME}/config.sh remove \
      --unattended \
      --token "${TOKEN}" \
      --auth pat
  echo "$(date): ${AZP_HOME}/config.sh removed"
  exit 0
}

[BenH-Puregym]

I don't see where it says it will be just be stopped? it says it will "spin down gracefully":
Agents in this mode will accept only one job and then spin down gracefully (useful for running in [Docker](https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/docker?view=azure-devops) on a service like Azure Container Instances).
they should be removed every time and like I said they sometimes remove but sometimes they don't remove which proves it does remove the agents just not always.

I have a cron job setup externally which targets the devops API and removes them but would be nice if it just removed it like it should without any additional script.

worth noting I'm mostly just seeing this with windows agents, linux ones seem to work fine.

[github-actions]

This issue has had no activity in 180 days. Please comment if it is not actually stale

[BenH-Puregym]

yes still an issue

[kmbisset89]

Still an issue

[github-actions]

This issue has had no activity in 180 days. Please comment if it is not actually stale

[Nickmman]

Still an issue

[ruisantox]

I do not have the problem running in Linux but I have the same problem running in windows agent. Version 3.230

Can we have some feedback ? thank you

btw if the job is cancelled i am 60% sure that will not exit gracefully

[cariad-chris]

Same issue even with the shutdown function.
Using SPN auth in my case. Logs say successfully removed but shows as offline in the pool