GitHub Code Sync (alpha)

[tnolet]

Introduction

updates

• 23-06-22: local dependencies are now automatically resolved. See the update post below

Managing your Browser checks' code (Playwright or Puppeteer) through the Checkly UI is a good start for many users, but it's hard to scale as you create more and more browser checks. We noticed the following problems:

1. Users copy & pasted checks from git to Checkly as local debugging is easier and quicker.
2. Checks are hard to keep in sync with updated deploys: a new screen or UI update broke a check before it was (manually) updated.
3. Code should be versioned so you can use your regular git history and PR-review processes.

TL;DR — How do I get started?

We can enable this feature on your account with a feature flag. Please send an email to tim [at] checklyhq.com with your accountId which you can find on your account settings page

After that, you can find the GitHub Code Sync option in the "Change Settings" flyout on the Browser check edit screen.

What is GitHub Code Sync?

This Alpha version of GitHub Code Sync enables two use cases based on the same idea: you link a single .js file inside a repository on GitHub (private or public) to a Browser check and Checkly syncs it for you.

The Alpha supports two use cases:

1. Sync on push

Requirements

• You have installed the Checkly GitHub app

1. Create a Browser check or edit an existing one.
2. Click the "Change Settings" button.
3. Link a file on specific branch, for instance main
4. On each push or merge to main the contents of the file are injected into the check.

2. Sync on Vercel Deployment done

This option is "Vercel deployment aware" and only persists file contents to your check on a successful deployment from Vercel to your Production environment. This enables you to lifecycle your checks' code with your application code.

Requirements

• You are using our Vercel integration to trigger checks on deploy
• You have installed the Checkly GitHub app

1. Create a Browser check or edit an existing one.
2. Click the "Change Settings" button.
3. Link a file in a repo.
4. On a Preview deployment, the contents of the file are injected to run on a deployment triggered check against your Preview environment and then discarded.
5. On a Production deployment, the contents of the file are injected to run on a deployment triggered check against your Production environment. If the deployment check is successful, the file contents are persisted so your check code is now in sync with your version control.

Examples

We are dog fooding this solution right now on our own marketing site at https://github.com/checkly/checklyhq.com/tree/main/_checkly

What's next?

This Alpha version is us shipping very early to gather as much feedback and insights as early as possible. We are actively working on expanding the use cases and functionality, while not waiting to "ship a Rolls Royce" that is unvalidated. Next topics are

• dependencies / snippets
• clearer UI and workflow integration
• other CVS systems like GitLab.

Discussion

What feedback or suggestions do you have about GitHub Code Sync (alpha)?

Public Roadmap Issue

We are tracking the delivery of this feature here checkly/public-roadmap#213

[pondorasti]

After trying this out for a bit it makes me really excited to see what's to come next for this feature 🤗.

"Sync on Vercel Deployment done" is such a great addition as it ties app changes with e2e changes 🤝. No need to create maintenance windows anymore when there's a breaking change in the codebase. Absolutely love it 💯.

Here's some things that are still missing for my workflow:

• integration with groups - instead of manually setting the path of each test, I would love to do it once for a folder and be done with it
• snippets - they are must for scaling repetitive actions (ex: login flow) to multiple tests

[tnolet]

@pondorasti thanks for the feedback already. Somehow GH did not send me a notification!

- on snippets: (local) dependency management is 100% coming. We are looking at what the best way is. Maybe interesting is that you can already require() snippets, we just made that feature public today (it already worked under the hood). See https://www.checklyhq.com/docs/browser-checks/partials-code-snippets/

- on groups: That is a good suggestion. Tnx.

[pondorasti]

@tnolet I've ran into an issue with code sync where I cannot select the branch I'm looking for. It looks like it only fetches the first ~20 and no way of searching. I did a bit of sorcery and inserted a custom HTML select option and got it to work, but not optimal. Due to my magic branch name label also broke 😅.

[tnolet]

@pondorasti this is something we will fix before the GA launch. Thanks for reporting

[pondorasti]

@tnolet are there any plans to also support Typescript in the future? While this can be achieved through transpiling, it also reveals another one of my questions. Have there been any considerations for TS native runtime, something along the lines of Deno?

[tnolet]

@pondorasti no direct plans on the roadmap right now, but it will become a thing later. The Deno team is actually a Checkly user even!

[tnolet]

Update on Local dependencies

We just enabled the ability to resolve local dependencies. This means:

• You do not require snippets, just require() your own files in your repo.
• You can start using shared configuration, setup routines etc. for all your browser checks.

This is done automatically. No GH action or CLI's needed.

I created a demo repo based on the Vercel Commerce template to showcase this.

• it has a central checkly.config.js
• it has a __checks__ folder with the checks and some helpers to DRY up shared routines.

Known limitations

• We are using rollup under the hood to parse your dependency chain to one JS bundle. You will see that bundle in the Checkly UI which might be confusing. We will use maps later to clear that up
• You cannot use any NPM package. We will add error handling to alert you when you might include a package we do not support.