Count of total and failed tests is off

[josdejong]

I just started using the this Vitest extension in VS Code, and noticed that the total count of tests and successful tests is off. For example in the following screenshot, there is 1 test failing out of 430 tests, but the totals of the plugin suggests 4 failing tests out of 533 which is quite misleading.

Here is a minimal example demonstrating the issue:

import { test, describe, expect } from 'vitest'

function add(a, b) {
  return a + b
}

describe('describe A', () => {
  describe('describe B', () => {
    test('test 1', () => {
      expect(add(2, 2)).toBe(4)
    })

    test('test 2', () => {
      expect(add(3, 3)).toBe(42) // This test will fail
    })
  })
})

This example contains two tests inside 2 nested describe statements, where 1 of the tests fails. Running this with the test plugin I get:

Which suggests there are 4 failing tests out of 5 (but it is 1 failing test out of 2)

On the command line, the counts are correct:

 RERUN  rerun all tests x5

 ❯ index.test.js (2)
   ❯ describe A (2)
     ❯ describe B (2)
       ✓ test 1
       × test 2

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Failed Tests 1 ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯

...

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯
 Test Files  1 failed (1)
      Tests  1 failed | 1 passed (2)
   Start at  09:40:25
   Duration  10ms


 FAIL  Tests failed. Watching for file changes...
       press h to show help, press q to quit

It looks like the describe(...) wrappers are also counted as a test too, and I don't think that should be the case.

I guess this is a bug?

[sheremet-va]

This is how vscode's built-in Test API works. It shows the tree up to the root if there is a failing test so it's easier to find if you have tests collapsed. I think it is quite useful and intended.

[josdejong]

Thanks! Here you go: https://github.com/josdejong/vitest-vscode-count-issue

[sheremet-va]

Thank you! I fixed the error in #419

[josdejong]

Awesome! Thanks Vladimir.

[josdejong]

I can confirm that the latest version of the plugin now shows the right numbers 🎉

[josdejong]

There is one thing that still could be improved: whilst the tests are running, the old total (including test suites) is shown. When the tests are finished, the right total is shown.