Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unit Tests Are Executed During Integration Test Runs #36817

Open
rogercoll opened this issue Dec 13, 2024 · 0 comments
Open

Unit Tests Are Executed During Integration Test Runs #36817

rogercoll opened this issue Dec 13, 2024 · 0 comments
Labels
ci-cd CI, CD, testing, build issues

Comments

@rogercoll
Copy link
Contributor

Component(s)

No response

Describe the issue you're reporting

While modifying the CI to include sudoers execution for integration tests, I noticed that we are using non-exclusive Go build tags to differentiate between unit and integration tests.

The CI runs the following make targets for each commit:

  • make gotest GROUP=
  • make gointegration GROUP=...

If we run the equivalent locally and just targeting one component for a reduced output:

Unit test

$ make --no-print-directory -C receiver/filestatsreceiver test
/home/hello/opentelemetry-collector-contrib/.tools/gotestsum --rerun-fails=1 --format testname --packages="./..." -- -race -count=1 -timeout 600s -parallel 4 --tags=""
PASS Test_Config_Validate/valid (0.00s)
PASS Test_Config_Validate/missing_include_pattern (0.00s)
PASS Test_Config_Validate (0.00s)
PASS TestComponentFactoryType (0.00s)
PASS TestComponentConfigStruct (0.00s)
PASS internal/metadata.TestMetricsBuilderConfig/default (0.00s)
PASS TestComponentLifecycle/metrics-shutdown (0.00s)
PASS TestComponentLifecycle/metrics-lifecycle (0.00s)
PASS TestComponentLifecycle (0.00s)
PASS Test_Scrape (0.00s)
PASS Test_Scrape_All (0.00s)
PASS internal/metadata.TestMetricsBuilderConfig/all_set (0.00s)
PASS internal/metadata.TestMetricsBuilderConfig/none_set (0.00s)
PASS internal/metadata.TestMetricsBuilderConfig (0.00s)
PASS internal/metadata.TestResourceAttributesConfig/default (0.00s)
PASS internal/metadata.TestResourceAttributesConfig/all_set (0.00s)
PASS internal/metadata.TestResourceAttributesConfig/none_set (0.00s)
PASS internal/metadata.TestResourceAttributesConfig (0.00s)
PASS internal/metadata.TestMetricsBuilder/default (0.00s)
PASS internal/metadata.TestMetricsBuilder/all_set (0.00s)
PASS internal/metadata.TestMetricsBuilder/none_set (0.00s)
PASS internal/metadata.TestMetricsBuilder/filter_set_include (0.00s)
PASS internal/metadata.TestMetricsBuilder/filter_set_exclude (0.00s)
PASS internal/metadata.TestMetricsBuilder (0.01s)
PASS internal/metadata.TestResourceBuilder/default (0.00s)
PASS internal/metadata.TestResourceBuilder/all_set (0.00s)
PASS internal/metadata.TestResourceBuilder/none_set (0.00s)
PASS internal/metadata.TestResourceBuilder (0.00s)
PASS .
PASS internal/metadata

DONE 28 tests in 1.671s

Integration test

$ make --no-print-directory -C receiver/filestatsreceiver mod-integration-test
running go integration test ./... in /home/hello/opentelemetry-collector-contrib/receiver/filestatsreceiver
/home/hello/opentelemetry-collector-contrib/.tools/gotestsum --rerun-fails=1 --format testname --packages="./..." -- -race -timeout 360s -parallel 4 -tags=integration,""
PASS Test_Config_Validate/valid (0.00s)
PASS Test_Config_Validate/missing_include_pattern (0.00s)
PASS Test_Config_Validate (0.00s)
PASS TestComponentFactoryType (0.00s)
PASS TestComponentConfigStruct (0.00s)
PASS TestComponentLifecycle/metrics-shutdown (0.00s)
PASS TestComponentLifecycle/metrics-lifecycle (0.00s)
PASS TestComponentLifecycle (0.00s)
PASS Test_Integration (3.00s)
PASS Test_Scrape (0.00s)
PASS Test_Scrape_All (0.00s)
PASS . (cached)
PASS internal/metadata.TestMetricsBuilderConfig/default (0.00s)
PASS internal/metadata.TestMetricsBuilderConfig/all_set (0.00s)
PASS internal/metadata.TestMetricsBuilderConfig/none_set (0.00s)
PASS internal/metadata.TestMetricsBuilderConfig (0.01s)
PASS internal/metadata.TestResourceAttributesConfig/default (0.00s)
PASS internal/metadata.TestResourceAttributesConfig/all_set (0.00s)
PASS internal/metadata.TestResourceAttributesConfig/none_set (0.00s)
PASS internal/metadata.TestResourceAttributesConfig (0.00s)
PASS internal/metadata.TestMetricsBuilder/default (0.00s)
PASS internal/metadata.TestMetricsBuilder/all_set (0.00s)
PASS internal/metadata.TestMetricsBuilder/none_set (0.00s)
PASS internal/metadata.TestMetricsBuilder/filter_set_include (0.00s)
PASS internal/metadata.TestMetricsBuilder/filter_set_exclude (0.00s)
PASS internal/metadata.TestMetricsBuilder (0.01s)
PASS internal/metadata.TestResourceBuilder/default (0.00s)
PASS internal/metadata.TestResourceBuilder/all_set (0.00s)
PASS internal/metadata.TestResourceBuilder/none_set (0.00s)
PASS internal/metadata.TestResourceBuilder (0.00s)
PASS internal/metadata (cached)

DONE 29 tests in 0.188s

The total number of executed tests just differs by one, this is because unit test files are also included in the "integrations" execution. Files without any build tags (default files) will also be included when using the Go's -tags= flag.

Do we want to keep running unit tests among the integration ones? Could we remove the "unit" test execution from the CI if they are already run?

If all the unit test files are modified to include a negative tag (//go:build !integration), only the actual integration tests are run with the corresponding make target:

 make --no-print-directory -C receiver/filestatsreceiver mod-integration-test
running go integration test ./... in /home/hello/opentelemetry-collector-contrib/receiver/filestatsreceiver
/home/hello/opentelemetry-collector-contrib/.tools/gotestsum --rerun-fails=1 --format testname --packages="./..." -- -race -timeout 360s -parallel 4 -tags=integration,""
PASS Test_Integration (3.00s)
PASS . (cached)
EMPTY internal/metadata

DONE 1 tests in 0.185s
@rogercoll rogercoll added the needs triage New item requiring triage label Dec 13, 2024
@atoulme atoulme added ci-cd CI, CD, testing, build issues and removed needs triage New item requiring triage labels Dec 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-cd CI, CD, testing, build issues
Projects
None yet
Development

No branches or pull requests

2 participants