breaking: use webpack builtin cache

[JLHwung]

Please Read the CONTRIBUTING Guidelines
In particular the portion on Commit Message Formatting

Please check if the PR fulfills these requirements

• Tests for the changes have been added (for bug fixes / features)
• Docs have been added / updated (for bug fixes / features)

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)

• Bugfix
• Feature
• Code style update (formatting, local variables)
• Refactoring (no functional changes, no api changes)
• Build related changes
• CI related changes
• Other... Please describe: Use webpack builtin cache

What is the current behavior? (You can also link to an open issue here)
The babel-loader uses per-input filesystem cache.

What is the new behavior?
The babel-loader option cacheCompression will be removed, users can configure the webpack option `cache.
babel-loader will use the builtin webpack cache.

The builtin webpack cache offers many options and users will be able to compose their own cache plugin by tapping the compiler cache hooks.

Does this PR introduce a breaking change?

• Yes
• No

If this PR contains a breaking change, please describe the following...

• Impact: Big
• Migration path for existing applications:
  If you are using the babel-loader cacheDirectory or cacheCompression option, set cacheDirectory to true and enable the webpack persistent cache via

// webpack.config.js
{
  cache: {
    type: "filesystem",
    compression: "gzip"
  }
}

The default webpack fs cache location is ./node_modules/.cache/webpack. You can change the cache directory via the webpack option cache.cacheDirectory. See the official docs for more webpack cache options.

• Github Issue(s) this is regarding:
  Closes Use LMDB cache instead of regular FS cache to speed up cache hits #932, as LMDB cache can be implemented via a third-party webpack cache plugin
  Closes Cache files take up too much disk space #978 and closes Prune babel-loader's cache so it does not grow forever #634, as you can instruct webpack to store unused filesystem cache up to one-month via cache.maxAge
  Closes Expose cache implementation #831, as you can implement your own webpack cache plugin
  Closes Can u provide write and read options to override default behavior #816, as redis cache can be implemented via a third-party webpack cache plugin
  Closes [feature request] Split cache files into subfolders to reduce number of files in a single folder #558, as webpack save multiple files into a single pack when the cache type is filesystem

Other information:

This is an early WIP, obviously most cache tests are currently failing due to config errors and incorrect filesystem structures. The updated cache test serves as a proof-of-concept that this approach works with the memory cache (The default of webpack 5 in development mode). Note that the memory cache will be purged when the compiler shutdown, so for those who enabled the babel-loader cache, they should enable the webpack filesystem cache.

When the filesystem cache is enabled, users can simply disable the Babel cache in most circumstances, such as 1) babel-loader is the only webpack loader for the js/ts inputs or 2) every js/ts loader is cachebale (the webpack defaults for all loaders unless the loader explicitly called this.cacheable(false)). The transformed output will be cached by webpack into the filesystem even without babel-loader cache. So in the proof-of-concept test we have to plug in an uncacheable loader for babel-loader cache to be hit again.

Alternative Solutions:

A cacheProvider: "webpack" | "babel-loader" option to switch between webpack cache and our own cache. But the new option will further complicates the cache config, and the babel-loader cache does not offer very much functionalities than the webpack builtin cache.

I would like to get feedback from team members, @liuxingbaoyu and @nicolo-ribaudo Do you think this approach is a good direction or we should keep our own cache? Do you think we should ship this in v10 or defer the changes to v11?

[JLHwung]

The ./cache is now unused, we can remove this file and the only dependency find-up after tests are fixed.

[JLHwung]

The cache log is updated from "writing result to cache file" to current wordings to reflect the fact that the loader is unaware of the cache implementation: it may be written to the filesystem, it may be written to a database or it is retained in the memory.

[liuxingbaoyu]

Do you think this approach is a good direction or we should keep our own cache? Do you think we should ship this in v10 or defer the changes to v11?

This looks good to me. Both v10 and v11 are ok for me and the migration steps seem to be simple.

[JLHwung]

This PR is ready for review. The options docs are also updated.

[liuxingbaoyu]

Can we still utilize webpack cache system when webpack options disable cache?🤔

[JLHwung]

Can we still utilize webpack cache system when webpack options disable cache?🤔

In most case, no. When webpack cache is disabled, the builtin cache is doing nothing. The cache facade provided by webpack is essentially an event emitter. In this case, the cache action event is triggered but there is no cache event handler to actually do the cache job.

If this PR gets merged, I would also add to the release note that to enable the babel-loader cache, you will have to turn on the webpack cache. But for most cases, the webpack cache is good enough and you don't have to enable the babel-loader cache at all, which is also our default.

[steverep]

Will this break compatibility with Rspack?