Concurrent Compilation Error

[tleef]

Caching the webpack compiler, introduced in PR #109, causes an error when compiling multiple files concurrently.

This issue is not related to gulp however I ran into it when using gulp to compile my project. I am working with a mono-repo where I have a services directory which contains several microservices. each microservice has its own src directory which contains an index.js.

I use the following code to compile each micro service individually

const gulp = require('gulp')
const compiler = require('webpack')
const webpack = require('webpack-stream')

...

gulp.task('webpack', function () {
  const tasks = getDirectories(`./services`).map((dir) => {
    return gulp.src(join(dir, 'src/index.js'))
      .pipe(webpack(require('./webpack.config.js'), compiler))
      .pipe(gulp.dest(join(dir, 'dist/')))
  })

  return merge(tasks)
})

webpack-stream 5.0.0 works as expected and produces a compiled version of each microservice which is stored in a dist directory alongside its respective src directory.

webpack-stream 5.1.0 introduced caching and reuses the same webpack instance for each compilation task which results in the following error:

ConcurrentCompilationError: You ran Webpack twice. Each instance only supports a single concurrent compilation at a time.

[Linux-Dream]

i found this is may be the compiler cache problem. when options change, webstream will empty the cache object. but when run parallel gulp task, all the task will share the same compiler instance.
the problem code show here(webpack-stream/index.js):
line 147: var compiler = cache.compiler || webpack(config);
line 148: cache.compiler = compiler;

so,i think the compiler may not cache by closure..should cache on the function.

[bobbybol]

I'm having the exact same issue when going from 5.0.0 to 5.1.0 - hoping for a fix or instructions on how to have multiple compilations.

[danieldiekmeier]

I came across the same error and fixed it by downgrading to 5.0, but it would be nice to have the cache working correctly/reliably.

[WillsB3]

I'm also seeing this issue when using webpack-stream in combination with gulp.parallel.

[swwind]

The same problem. When I using gulp.series instead of gulp.parallel, it generates the same result for different tasks.

Solved by cloning my options for each task.

[Renkas]

Any update on a fix?

[shama]

Patches are welcome.

[mightydes]

+1

[An-Haijun]

I came across the same error and fixed it by downgrading to 5.0, but it would be nice to have the cache working correctly/reliably.

@danieldiekmeier Thanks, it helped me

[Drafteed]

#207 increased rebuild time up to 52x (!) for my project. See this.