Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add an example of CI/CD - one that builds Argo Workflows, using Argo Workflows #8591

Open
alexec opened this issue May 3, 2022 · 19 comments · May be fixed by #13895
Open

Add an example of CI/CD - one that builds Argo Workflows, using Argo Workflows #8591

alexec opened this issue May 3, 2022 · 19 comments · May be fixed by #13895
Labels
area/docs Incorrect, missing, or mistakes in docs hacktoberfest type/feature Feature request

Comments

@alexec
Copy link
Contributor

alexec commented May 3, 2022 •
edited
Loading

Users want a more detailed example of doing CI and CD. We should have one that builds Argo Workflows (so it is Argo all the way down):

CI:

  • Use webhook to start build (i.e. WorkflowEventBinding).
  • Build the CLI.
  • Build images using Buildkit (see template).
  • Run test and collect test report.
  • Run coverage and collect report.
  • Deploy to a cluster.
  • Run basic E2E tests and collect report.

CD:

  • Tag and push tag.
  • Update deployment manifests using kustomize edit set image.
  • Commit deployment manifests.
  • Start Argo CD sync step (should this use a plugin?).

Pipline:

  • Combine CI and CD jobs with an approval step.
@alexec alexec added type/feature Feature request good first issue Good for newcomers area/docs Incorrect, missing, or mistakes in docs labels May 3, 2022
@KirtiGhugtyal6
Copy link

Can I work on this issue?

@Yh010
Copy link
Contributor

Yh010 commented Aug 13, 2022

I would like to work on this issue. I am completely new to Kubernetes, could you pls guide me on how to get this issue solved?

@terrytangyuan
Copy link
Member

Removing assignee as there hasn't been any progress. Feel free to update here if you are still working on it.

@terrytangyuan
Copy link
Member

I would like to work on this issue. I am completely new to Kubernetes, could you pls guide me on how to get this issue solved?

You'll need to be familiar with Kubernetes.

@surbhiahuja
Copy link

surbhiahuja commented Sep 5, 2022 •
edited
Loading

@terrytangyuan , I have worked on the initial draft of writing examples and use cases of ARGO CD, Please assign this to me and I will raise a PR ASAP.

@surbhiahuja
Copy link

@alexec , I have added a few detailed examples for ARGO CI-CD, Please review and share your inputs. #9596

@surbhiahuja
Copy link

@terrytangyuan , Please look into PR raised and suggest if any changes are required #9596

@zren11 zren11 mentioned this issue Oct 30, 2022
Closed
@mahi072
Copy link

mahi072 commented Nov 16, 2022

I would like to work on this issue, if it is still open.

@caelan-io
Copy link
Member

@StevenYuanyuan - are you working on this issue already?

@mahi072 will take on if you no longer plan to.

@zren11
Copy link

zren11 commented Nov 29, 2022

Hi @caelan-io, @mahi072 can take on it

@Sajiyah-Salat
Copy link

Hello I have done somework in kubernetes and I know ci/cd but completely new to argo cd. can someone guide me for the next steps. and if required provide me some materials please.

@Sajiyah-Salat
Copy link

/assign

@caelan-io
Copy link
Member

Thanks, @Sajiyah-Salat - please feel free to take this one on! Let me know if have any questions.

Also, if anyone is looking for 80% of a solution to this issue, please refer to this Argo CI example repo built by @tico24 :
https://github.com/pipekit/argo-workflows-ci-example

If you want a simple example to get started with CI using Argo Workflows for your own personal or work codebase, check out that repo. It uses a GitOps approach to pulling a repo, building a container, and then deploying a simple website to an Argo CD app.

Important note:

In order to make this a semi-usable example, we have cut a number of security corners. Please don't just blindly run this in production.

@TheSwarnim
Copy link
Contributor

Hi all, if no one is picking this issue I wanted to pick this task

@aakankshabhende
Copy link

@caelan-io @alexec I would like to work on this issue. Could you please assign me this?

@caelan-io
Copy link
Member

Thanks @aakankshabhende. Assigned you. 🙌

@agilgur5
Copy link

Removing assignee as there has been no progress.

Please feel free to submit a PR without being assigned.

@wesleyscholl
Copy link

wesleyscholl commented Oct 15, 2024 •
edited
Loading

@agilgur5 @alexec I started working on this:

CI

  • WorkflowEventBinding
  • WorkflowTemplate - Build the CLI.
  • WorkflowTemplate - Build images using Buildkit - Pushes image to Docker Hub
  • WorkflowTemplate - Run unit tests and collect test report.
  • WorkflowTemplate - Run coverage and collect report.
  • WorkflowTemplate - Deploy to a cluster.
  • WorkflowTemplate - Run basic E2E tests and collect report.
  • CI Documentation

CD

  • Tag and push tag.
  • Update deployment manifests using kustomize edit set image.
  • Commit deployment manifests.
  • Start Argo CD sync step.
  • CD Documentation

Pipeline

  • Combine CI and CD jobs with an approval step.
  • Pipeline Documentation

wesleyscholl added a commit to wesleyscholl/argo-workflows that referenced this issue Nov 6, 2024
@wesleyscholl
feat: Add CI/CD example for building Argo Workflows using Argo Workfl…
b4b5b0d 
…ows. Closes argoproj#8591
wesleyscholl added a commit to wesleyscholl/argo-workflows that referenced this issue Nov 7, 2024
@wesleyscholl
docs: Add CI/CD example for building Argo Workflows. Closes argoproj#…
4791ca2 
…8591

This commit introduces a detailed example for implementing CI/CD pipelines to build Argo Workflows using Argo Workflows:

- **CI**: Includes steps for initializing builds via webhook, building the CLI, using Buildkit for image builds, running tests, collecting coverage reports, deploying to a cluster, and executing basic E2E tests.
- **CD**: Covers tagging, updating manifests with kustomize, committing updates, and initiating an Argo CD sync.
- **Pipeline**: Integrates CI and CD jobs with an approval step.

Signed-off-by: Wesley Scholl <[email protected]>
Signed-off-by: Wesley Scholl <[email protected]>
wesleyscholl added a commit to wesleyscholl/argo-workflows that referenced this issue Nov 7, 2024
@wesleyscholl
docs: Add CI/CD example for building Argo Workflows. Closes argoproj#…
9230135 
…8591

Signed-off-by: Wesley Scholl <[email protected]>
wesleyscholl added a commit to wesleyscholl/argo-workflows that referenced this issue Nov 7, 2024
@wesleyscholl
docs: Add CI/CD example for building Argo Workflows. Closes argoproj#…
9752d58 
…8591

This commit introduces a detailed example for implementing CI/CD pipelines to build Argo Workflows using Argo Workflows:

- **CI**: Includes steps for initializing builds via webhook, building the CLI, using Buildkit for image builds, running tests, collecting coverage reports, deploying to a cluster, and executing basic E2E tests.
- **CD**: Covers tagging, updating manifests with kustomize, committing updates, and initiating an Argo CD sync.
- **Pipeline**: Integrates CI and CD jobs with an approval step.

Signed-off-by: Wesley Scholl <[email protected]>
@wesleyscholl wesleyscholl mentioned this issue Nov 7, 2024
Closed
2 tasks
wesleyscholl added a commit to wesleyscholl/argo-workflows that referenced this issue Nov 12, 2024
@wesleyscholl
docs: Add CI/CD example for building Argo Workflows using Argo Workfl…
ae71cdf 
…ows - Fixes argoproj#8591

Signed-off-by: Wesley Scholl <[email protected]>
wesleyscholl added a commit to wesleyscholl/argo-workflows that referenced this issue Nov 12, 2024
@wesleyscholl
docs: Add CI/CD example for building Argo Workflows using Argo Workfl…
1d59d3d 
…ows - Fixes argoproj#8591

Signed-off-by: Wesley Scholl <[email protected]>
@wesleyscholl wesleyscholl linked a pull request Nov 12, 2024 that will close this issue
Open
2 tasks
@wesleyscholl
Copy link

@agilgur5 @alexec I started working on this:

CI

  • WorkflowEventBinding
  • WorkflowTemplate - Build the CLI.
  • WorkflowTemplate - Build images using Buildkit - Pushes image to Docker Hub
  • WorkflowTemplate - Run unit tests and collect test report.
  • WorkflowTemplate - Run coverage and collect report.
  • WorkflowTemplate - Deploy to a cluster.
  • WorkflowTemplate - Run basic E2E tests and collect report.
  • CI Documentation

CD

  • Tag and push tag.
  • Update deployment manifests using kustomize edit set image.
  • Commit deployment manifests.
  • Start Argo CD sync step.
  • CD Documentation

Pipeline

  • Combine CI and CD jobs with an approval step.
  • Pipeline Documentation

PR #13895 - Completed the CI/CD example workflow.

wesleyscholl added a commit to wesleyscholl/argo-workflows that referenced this issue Nov 13, 2024
@wesleyscholl
docs: Add apiVersion and WorkflowTemplate kind to CI/CD example - Fixes
a7d9290 
argoproj#8591

Signed-off-by: Wesley Scholl <[email protected]>
wesleyscholl added a commit to wesleyscholl/argo-workflows that referenced this issue Nov 13, 2024
@wesleyscholl
docs: Add apiVersion and WorkflowTemplate kind to CI/CD example workf…
ad104ae 
…low - Fixes argoproj#8591

Signed-off-by: Wesley Scholl <[email protected]>
wesleyscholl added a commit to wesleyscholl/argo-workflows that referenced this issue Dec 5, 2024
@wesleyscholl
docs(examples): Add comprehensive test matrix and parallel execution …
dd9a795 
…support - Fixes argoproj#8591

- Added test matrix to cover all E2E test cases.
- Introduced a DAG for parallel test execution to optimize runtime.
- Updated workflow steps for improved parallelism.
- Refactored code to use generic templates, reducing duplication and overall code length.
- Enhanced code readability by adding detailed comments.
- Added build step for the controller.

Signed-off-by: Wesley Scholl <[email protected]>
wesleyscholl added a commit to wesleyscholl/argo-workflows that referenced this issue Dec 5, 2024
@wesleyscholl
docs(examples): Add CI/CD example for building Argo Workflows - Fixes a…
fdfc511 
…rgoproj#8591

Signed-off-by: Wesley Scholl <[email protected]>
@MasonM MasonM removed the good first issue Good for newcomers label Dec 15, 2024
wesleyscholl added a commit to wesleyscholl/argo-workflows that referenced this issue Dec 18, 2024
@wesleyscholl
docs(examples): Update test matrix software versions and improve step…
86f2f4b 
… configuration - Fixes argoproj#8591

- Updated test matrix to reflect latest software versions.
- Added detailed comments to improve workflow maintainability.
- Set  for all steps to ensure consistent file paths.
- Adjusted resource requests and limits (CPU/memory) to align with step requirements and runtime optimization.
- Refactored step/task commands to conform to WorkflowTemplate format for modularity.
- Combined unit tests and test coverage into a single step for efficiency.
- Improved readability and reduced redundancy by centralizing template logic.

Signed-off-by: Wesley Scholl <[email protected]>
wesleyscholl added a commit to wesleyscholl/argo-workflows that referenced this issue Dec 18, 2024
@wesleyscholl
docs(examples): Update test matrix software versions and improve step…
49c1578 
… configuration - Fixes argoproj#8591

- Updated test matrix to reflect latest software versions.
- Added detailed comments to improve workflow maintainability.
- Set  for all steps to ensure consistent file paths.
- Adjusted resource requests and limits (CPU/memory) to align with step requirements and runtime optimization.
- Refactored step/task commands to conform to WorkflowTemplate format for modularity.
- Combined unit tests and test coverage into a single step for efficiency.
- Improved readability and reduced redundancy by centralizing template logic.

Signed-off-by: Wesley Scholl <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/docs Incorrect, missing, or mistakes in docs hacktoberfest type/feature Feature request
Projects
None yet
Milestone
No milestone