Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need to be able to push data from ADF into a RDBMS in the workspace #205

Open
4 tasks
TonyWildish-BH opened this issue Nov 26, 2024 · 0 comments
Open
4 tasks
Labels
EPIC - SQL push to workspace Pushing data into an SQL DB in a workspace, instead of just raw CSV MVP Things that need to be considered for the MVP release

Comments

@TonyWildish-BH
Copy link
Collaborator

TonyWildish-BH commented Nov 26, 2024

We have the base+ADF workspace that Benedict created for us at the beginning of the project. This gives us linkage to the ADF shared service, which is how the data team are able to push CSV files into the shared storage.

We need an extension to this capability, so we can push data into a MySQL or AzureSQL database in the workspace as well. It's enough to have MySQL, we can copy the solution for AzureSQL later.

This probably means we need to extend the MySQL template, since these services may be deployed/destroyed multiple times throughout the lifetime of the workspace. The CSV solution can rely on the existence of shared storage in the workspace, which is why it could be created at the same time as the workspace.

Alternatively, maybe a DB connection can be provisioned with the workspace, and the MySQL DB can be plugged into it when they are started up?

So we need investigation to propose a solution, and then an implementation.

Acceptance criteria:

  • Solution proposed
  • PoC implemented
  • Documentation completed
  • Deployment to MVP
@TonyWildish-BH TonyWildish-BH added MVP Things that need to be considered for the MVP release EPIC - SQL push to workspace Pushing data into an SQL DB in a workspace, instead of just raw CSV labels Nov 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
EPIC - SQL push to workspace Pushing data into an SQL DB in a workspace, instead of just raw CSV MVP Things that need to be considered for the MVP release
Projects
None yet
Development

No branches or pull requests

1 participant