Skip to content

Commit

Permalink
Additional details on invisible environment variables. (vercel#1665)
Browse files Browse the repository at this point in the history
  • Loading branch information
nathanhammond authored Aug 9, 2022
1 parent 2df99d6 commit b1ede0d
Show file tree
Hide file tree
Showing 2 changed files with 103 additions and 8 deletions.
16 changes: 8 additions & 8 deletions docs/pages/blog/turbo-1-4-0.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Update today by running `npm install turbo@latest`.

## Automatic environment variable inclusion

To help ensure correct caching across environments, Turborepo will now automatically infer and include public environment variables when calculating cache keys for apps built with Astro, Create React App, Gatsby, Next.js, Nuxt, SvelteKit, Vite, Vue, and more. You can safely remove framework-specific public environment variables from `turbo.json` if you manually declared them.
To help ensure correct caching across environments Turborepo will now automatically infer and include public environment variables when calculating cache keys for apps built with Astro, Create React App, Gatsby, Next.js, Nuxt, SvelteKit, Vite, Vue, and more. You can safely remove framework-specific public environment variables from `turbo.json` if you manually declared them.

```diff filename="turbo.json"
{
Expand All @@ -36,17 +36,17 @@ To help ensure correct caching across environments, Turborepo will now automatic
- // Include build time public inlined environment variables that
- // are different in development and production, so that
- // Turborepo does not use the same cached build
- // accross environments
- // across environments
- "$NEXT_PUBLIC_EXAMPLE_ENV_VAR"
]
}
}
}
```

Note that this automatic detection and inclusion only works if Turborepo successfully infers the framework your apps are built with. Additionally, environment variables will only be part of the cache key in the workspaces where that framework is used. In other words, environment variables inferred for Next.js apps, will only apply to the cache key for workspaces inferred as Next.js apps. Other workspaces in the monorepo will not be impacted.
Note that this automatic detection and inclusion only works if Turborepo successfully infers the framework your apps are built with. Additionally, the environment variables will only be included in the cache key for tasks in workspaces where that framework is used. In other words, environment variables inferred for Next.js apps, will only be included in the cache key for workspaces detected as Next.js apps. Tasks in other workspaces in the monorepo will not be impacted.

For example, consider a monorepo with three workspaces: a Next.js project, a Create React App project, and a Typescirpt package. Each has a `build` script, and both apps depend on the Typescript project. Let's say that this Turborepo has a standard `turbo.json` pipeline that builds them all in order:
For example, consider a monorepo with three workspaces: a Next.js project, a Create React App project, and a TypeScript package. Each has a `build` script, and both apps depend on the TypeScript project. Let's say that this Turborepo has a standard `turbo.json` pipeline that builds them all in order:

```jsonc filename="turbo.json"
{
Expand All @@ -66,11 +66,11 @@ This improvement in hash specificity by framework is a significant step toward o

## `eslint-config-turbo`

We've also created a new ESLint plugin for further in-editor assistance and conformance to help ensure your Turborepo cache can be correctly shared across every environment. While our new hashing algorithm should cover most situations with most frameworks, this ESLint config will provide feedback for teams using other build time inlined environment variables that are not framework-prefixed, but impact build outputs (i.e. caching), and teams using their own in-house framework that we cannot infer automatically.
We've also created a new ESLint config for further in-editor assistance to help ensure your Turborepo cache can be correctly shared across every environment. While our new hashing algorithm should cover most situations with most frameworks, this ESLint config will provide in-editor feedback for teams using other build time inlined environment variables that are not framework-prefixed but impact build outputs (i.e. caching), and teams using their own in-house frameworks that we cannot detect automatically.

To get started, extend from `eslint-config-turbo` in your root [`eslintrc`](https://eslint.org/docs/latest/user-guide/configuring/configuration-files#configuration-file-formats) file:

```jsx
```jsonc
{
// Automatically flag env vars missing from turbo.json
"extends": ["next/core-web-vitals", "turbo"],
Expand All @@ -79,10 +79,10 @@ To get started, extend from `eslint-config-turbo` in your root [`eslintrc`](http

If you prefer more control over the rules - use can install and configure the `eslint-plugin-turbo` _plugin_ directly by first adding it to plugins and then configuring the desired rules:

```jsx
```jsonc
{
"extends": ["next/core-web-vitals"],
"plugins": ["turbo"]
"plugins": ["turbo"],
"rules": {
// Automatically flag env vars missing from turbo.json
"turbo/no-undeclared-env-vars": "error"
Expand Down
95 changes: 95 additions & 0 deletions docs/pages/docs/core-concepts/caching.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,101 @@ Luckily, you can control `turbo`'s cache fingerprinting (a.k.a. hashing) behavio
Pro Tip: In most monorepos, you don't often use environment variables in shared packages, but mostly only in applications. Thus, to get higher cache hit rates, you should likely only include environment variables in the app-specific tasks where they are used/inlined.
</Callout>

### Automatic environment variable inclusion

To help ensure correct caching across environments Turborepo 1.4+ will automatically infer and include public environment variables when calculating cache keys for apps built with detected frameworks. You can safely omit framework-specific public environment variables from `turbo.json`:

```diff filename="turbo.json"
{
"pipeline": {
"build": {
"dependsOn": [
"^build"
- "$NEXT_PUBLIC_EXAMPLE_ENV_VAR"
]
}
}
}
```

Note that this automatic detection and inclusion only works if Turborepo successfully infers the framework your apps are built with. The supported frameworks and the environment variables that Turborepo will detect and include in the cache keys:

- Astro: `PUBLIC_*`
- Blitz: `NEXT_PUBLIC_*`
- Create React App: `REACT_APP_*`
- Gatsby: `GATSBY_*`
- Next.js: `NEXT_PUBLIC_*`
- Nuxt.js: `NUXT_ENV_*`
- RedwoodJS: `REDWOOD_ENV_*`
- Sanity Studio: `SANITY_STUDIO_*`
- Solid: `VITE_*`
- SvelteKit: `VITE_*`
- Vite: `VITE_*`
- Vue: `VUE_APP_*`

The environment variables will only be included in the cache key for tasks in workspaces where that framework is used. In other words, environment variables inferred for Next.js apps will only be included in the cache key for workspaces detected as Next.js apps. Tasks in other workspaces in the monorepo will not be impacted.

For example, consider a monorepo with three workspaces: a Next.js project, a Create React App project, and a TypeScript package. Each has a `build` script, and both apps depend on the TypeScript project. Let's say that this Turborepo has a standard `turbo.json` pipeline that builds them all in order:

```jsonc filename="turbo.json"
{
"pipeline": {
"build": {
"dependsOn": [
"^build",
]
}
}
}
```

As of 1.4, when you run `turbo run build`, Turborepo will not consider any build time environment variables relevant when building the TypeScript package. However, when building the Next.js app, Turborepo will infer that environment variables starting with `NEXT_PUBLIC_` could alter the output of the `.next` folder and should thus be included when calculating the hash. Similarly, when calculating the hash of the Create React App's `build` script, all build time environment variables starting with `REACT_APP_PUBLIC_` will be included.

### `eslint-config-turbo`

To help detect additional places where unseen dependencies can be creeping into your builds, you can use `eslint-config-turbo` for further in-editor assistance to help ensure your Turborepo cache can be correctly shared across every environment. While automatic environment variable inclusion should cover most situations with most frameworks, this ESLint config will provide just-in-time feedback for teams using other build time inlined environment variables that are not framework-prefixed but impact build outputs. This will also help support teams using their own in-house frameworks that we cannot detect automatically.

To get started, extend from `eslint-config-turbo` in your root [`eslintrc`](https://eslint.org/docs/latest/user-guide/configuring/configuration-files#configuration-file-formats) file:

```jsonc
{
// Automatically flag env vars missing from turbo.json
"extends": ["next/core-web-vitals", "turbo"],
}
```

If you prefer more control over the rules - use can install and configure the `eslint-plugin-turbo` _plugin_ directly by first adding it to plugins and then configuring the desired rules:

```jsonc
{
"extends": ["next/core-web-vitals"],
"plugins": ["turbo"],
"rules": {
// Automatically flag env vars missing from turbo.json
"turbo/no-undeclared-env-vars": "error"
}
}
```

The plugin will warn you if you are using non-framework-related environment variables in your code that have not been declared in your `turbo.json`.

### Invisible Environment Variables

Since Turborepo runs _before_ your tasks, it is possible for your tasks to create or mutate environment variables after `turbo` has already calculated the hash for a particular task. For example, consider this `package.json`:

```json
{
"scripts": {
"build": "NEXT_PUBLIC_GA_ID=UA-00000000-0 next build",
"test": "node -r dotenv/config test.js"
}
}
```

`turbo`, having calculated a task hash prior to invoking the `build` script, will be unable to discover the `NEXT_PUBLIC_GA_ID=UA-00000000-0` environment variable and thus unable to partition the cache based upon that, or any environment variable configured by `dotenv`.

Be careful to ensure that all of your environment variables are configured prior to invoking `turbo`!

## Force overwrite cache

Conversely, if you want to force `turbo` to re-execute a previously cached task, add the `--force` flag:
Expand Down

0 comments on commit b1ede0d

Please sign in to comment.