🤖 Use SageMaker Foundation models in VPC mode (no internet connectivity)

[bagg3rs]

Idea Description
🛡️ Use AWS SageMaker AI foundation models in VPC mode with no internet connectivity to avoid unintended traffic ingress/egress for sensitive data workloads.

Why Should We Do This
✨ GenerativeAI and LLMs are coming, we should be prepared for the barrage of requests and especially prepared when dealing with our most sensitive data.

🚢 We also don't want to miss the boat and find we have nothing to catalogue as all the data is in another system where these tools are.

• SageMaker JumpStart doesn’t download the models from a public model zoo, it can be used in fully locked-down environments as well where there is no internet access
• Because the network access can be limited and scoped down for SageMaker JumpStart models, this helps teams improve the security posture of the environment
• Due to the VPC boundaries, access to the endpoint can also be limited via subnets and security groups, which adds an extra layer of security
• We can also understand cost and usage let that feed into future tooling work.

Definition of Done
🙋 Can ask questions to a model which sits on its own without skynet connection.

What skills do we need?
AWS, Terraform, SageMaker

[julialawrence]

Awesome idea!

[jacobwoffenden]

[murdo-moj]

if you need a hand, I can help with this