Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parameterized test may drop some parameters silently #99

Open
andrey-kuznetsov opened this issue Aug 14, 2024 · 0 comments
Open

Parameterized test may drop some parameters silently #99

andrey-kuznetsov opened this issue Aug 14, 2024 · 0 comments

Comments

@andrey-kuznetsov
Copy link

Describe the bug
Suite runner uses literal parameter value to create test name:
https://github.com/ozontech/allure-go/blob/master/pkg/framework/runner/suite_runner.go#L230-L234
Also, test name is truncated to 150 characters, and this truncated string is used as a key in "name->method" mapping. Once one have long parameter strings or structures with long common prefix, they surprisingly lose tests for some of the parameters.

To Reproduce

import (
	"github.com/ozontech/allure-go/pkg/framework/provider"
	"github.com/ozontech/allure-go/pkg/framework/suite"
	"github.com/stretchr/testify/assert"
	"strings"
	"testing"
)

type SomeSuite struct {
	suite.Suite
	ParamSomething []string
}

var count int

func (s *SomeSuite) BeforeAll(t provider.T) {
	count = 0
	commonPrefix := strings.Repeat("*", 150)
	s.ParamSomething = []string{commonPrefix + "a", commonPrefix + "b"}
}

func (s *SomeSuite) TableTestSomething(t provider.T, param string) {
	count++
}

func TestIt(t *testing.T) {
	count = 0
	suite.RunSuite(t, new(SomeSuite))
	assert.Equal(t, 2, count)
}

Additional context
Name length limitation in order to fix unrelated TempDir issue looks fragile. Please consider UUID, parameter value hash or even require Stringer conformance from a parameter. I tried UUID -- it worked fine for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant