How to make Agents use Proxy (for everything)

[Nightreaver]

Hello,

I successfully built the agent-image in azure and it seems to deploy successfully in our machine scale set.
But I have one issue, that is that no regular network traffic is allowed, only communication to the internet can go through a proxy server.
I cannot even install the extension because the agent cannot download the extension from azure without having the proxy setup.

My question is, how can I setup proxy settings for these agents properly?

What I have tried:
Created a script and added that script as late step in the ubuntu2004.json from packer.

#!/bin/bash -e

echo 'http_proxy="http://http-proxy.ournet:8080/"
https_proxy="http://http-proxy.ournet:8080/"
ftp_proxy="http://http-proxy.ournet:8080/"
no_proxy="127.0.0.1,localhost"
HTTP_PROXY="http://http-proxy.ournet:8080/"
HTTPS_PROXY="http://http-proxy.ournet:8080/"
FTP_PROXY="http://http-proxy.ournet:8080/"
NO_PROXY="127.0.0.1,localhost"
VSTS_AGENT_INPUT_PROXYURL="http://http-proxy.ournet:8080"
' | sudo tee -a /etc/environment

As you can see, I add these ENV-VARS to /etc/environment but I don't if they ever get used because it seems even with these changes I cannot connect anywhere.

Any help on how to setup the agent so connections are possible is much appreciated.