Would it be possible to add a simple and easy to use degree of parallelism flag?
ie: I want to run 3 specs at the same time: --parallelism=3
or
I want to run all specs at the same time: --parallelism=all, where all = number of cores - 2

This will make it much easier for Cypress to utilise all resources on whatever system it's running on.
Without adding the extra complexity of having to refactor all your specs into different directories and adding the groups proposed.

But I still think it's important to have both options avilable to fit all use cases

Hey @Toxicable, thank you for submitting feedback on our proposal! We really want to get this right.

The proposal above as it is currently is missing a bit of context in terms of the --specs flag, which probably should have been included. You will be able to pass this flag to define the specs to run in the cypress run command - this will be a pretty key flag to use in some use cases.

Could you give us some more information on your use case? What does your project look like that you want to run tests on? How are you currently choosing which files are included in a run?

@jennifer-shehane I don't see the usefulness of the --specs flag at scale.
We have hundrends of spec files and listing off each of them, either manually or with scripts, would be horrible.
Another issue is that our CI systems have between 48 - 128 CPU cores which would also be tedious to manage the number of cypress run commands to be run and we'd probably underutilizing the resources available.

We currently point Cypress at a single directory and expect that it'll run all spec files there.
It would be amazing if we could simply tell Cypress to run all the files in that directory in parallel using all cores available with a single Cypress run commands.

@Toxicable Cores here isn't really the problem, memory is likely the bottleneck. How much memory do your machines have?

Also are they linux machines? Browsers behave differently when they are out a focus, and only a single browser can be in focus... but in linux with the use of xvfb they will behave as if they have focus (which is what we do today).

Regardless though - I believe our implementation will work flawlessly for you. It's up to you to determine how many machines you want to run in parallel - and you can easily do this with a node script or even a bash command.

const _ = require('lodash')
const os = require('os')
const cypress = require('cypress')

// kick off a bunch of cypress runs
const runs = _.times(os.cpus.length / 2, (i) => {
  return cypress.run({ record: true, parallel: true }) // this is it
})

Promise.all(runs)
.then((resultsArray) => {
  const totalFailed = _.map(resultsArray, "totalFailed")

  process.exit(totalFailed)
})

This kicks off a bunch of cypress runs based on half of your cpu cores. You may have other problems trying to run this all on a single machine. For instance your server or database would have to support multiple sesssions at the same time. If you do any kind of seeding you will have to take into account.

That's why we generally suggest parallelizing at the OS level. For instance, you could take your beefy machine and cut it into a dozen docker containers running all at the same time. It would work the same way. In most CI providers, that's how it works - they are all isolated from each other.

The --parallel flag automatically load balances the specs, you don't have to specify anything.

The grouping as @jennifer-shehane suggested is only if you want to chunk your specs by a particular group and then manage / parallelize them differently. If you have a single group, then you don't do anything.

EDIT: I just realized that spawning a bunch of cypress runs together will clobber things like screenshotsFolder and videosFolder. You could pass a configuration option using the (i) index to create those dynamically.

Also, I think there may be an issue because we don't create dynamic user data dir profiles for chrome or electron. There's already an issue open with that, and that would enable chrome to be opened multiple times and prevent session clashing with cookies or localstorage. That's a really simple, easy fix.

@brian-mann They are Linux servers with equivilent amounts of RAM, between 128GB - 512GB I believe.
Could you elaborate on the RAM part:
What part of Cypress casues it to uses large amounts of RAM?
Chrome instances? Electron Instances? test length?

I must have misunderstood part of how the parallel flag works.
With your example is there some sort of global variable /singleton that hands off specs to the workers?

If that is all the config needed, aside from app level stuff such as database config as you mentioned then I think you're right about it fitting out use case then.

Yeah I think I was so concerned about covering the complicated edge cases in the proposal that I glanced over the standard use cases. I've updated it to hopefully be clearer that:

• the --group flag and name are optional (only use it if you need it!)
• Cypress will automatically parallelize all specs in your project when run with the --parallel flag if all run within a homogeneous environment (aka, all run in same OS, version, browser - everyones standard use of cypress run --record today)

@Toxicable its not Cypress that eats up the memory - it is the browser. I would allocate at least 4gb per "instance". Sounds like you have plenty though, so it should be fine. Once we create new profiles for each cypress run I think this will all "just work".

Disregard my previous comment about needing to generate new screenshotsFolder and videosFolder - @jennifer-shehane pointed out that all you have to do is turn off trashAssetsBeforeRuns in the config. That would prevent race conditions where one run accidentally blows away assets from another current run.