GenerateTestingPlatformEntryPoint has no effect on class library project

[drieseng]

Describe the bug

We currently do not use Microsoft.Testing.Platform, and as such our test projects are defined as class library instead of executable.
Since we've upgraded to the .NET 9.0 SDK, CA1515 is reported for our test classes. The root cause of this is that an AutoGeneratedProgram class is generated automatically which causes CA1515 to believe it's dealing with an executable/application.

We tried to instruct MSTest (or the Microsoft.Testing.Platform¨) not to emit this class by setting the GenerateTestingPlatformEntryPoint MSBuild property to false, but this has no effect at all.

Steps To Reproduce

1. Clone https://github.com/drieseng/do_not_emit_AutoGeneratedProgram.git
2. Open the UnitTests solution.
3. Compile the solution.
4. Open the UnitTests assembly in ildasm.

Expected behavior

There's no AutoGeneratedProgram class.

Actual behavior

There's a AutoGeneratedProgram class.

[Youssef1313]

That looks like VSTest. Could you try <GenerateProgramFile>false</GenerateProgramFile> instead? You may also need to explicitly set <OutputType>Library</OutputType>

[Evangelink]

@Youssef1313 is right, you haven't opt-in the new testing platform so GenerateTestingPlatformEntryPoint has no effect. I am not sure that disabling the main of VSTest would not cause issue with running tests.

@drieseng Just to confirm, are you seeing the CA1515 on the test classes?

[Youssef1313]

It the analyzer's fault I think. This would be a duplicate of dotnet/roslyn-analyzers#7192

[drieseng]

@Youssef1313, setting the GenerateProgramFile MSBuild property only makes things worse.
If I do that, the following compile error is produced:

CS5001: Program does not contain a static 'Main' method suitable for an entry point

I've updated my test repo accordingly.

@Evangelink, I'm indeed seeing that. I've updated my test repo to enable the analyzers. If you remove the #pragma warning disable CA1515 from the Class1 class, you'll see that CA1515 is reported.

CA1515 could indeed be updated to ignore test classes, but if the AutoGeneratedProgram was not emitted then CA1515 would not have to be modified for my case.

[Youssef1313]

Yes, it's actually an analyzer's fault IMO, as I mentioned in the previous comment. Sorry for confusing you.

I don't see an issue on our side. We could implement a suppressor for the analyzer if really necessary, but I'm viewing this more to be an analyzer bug. My personal opinion is to go for <NoWarn>$(NoWarn);CA1515</NoWarn> in your test projects temporarily until discussions on what the analyzer behavior should be are settled. @Evangelink Do you agree?

[Youssef1313]

Note that I think the current behavior regarding generating an entry point has been like this for so long.

https://github.com/microsoft/vstest/blob/7d34b30433259fb914aaaf276fde663a47b6ef2f/src/package/Microsoft.NET.Test.Sdk/netcoreapp/Microsoft.NET.Test.Sdk.targets#L19-L26

This code has been there for at least 6 years

[drieseng]

@Youssef1313, why don't you consider it an issue that the AutoGeneratedProgram class is emitted even if GenerateTestingPlatformEntryPoint is set to false?

That class looks like this:

using System;
[Microsoft.VisualStudio.TestPlatform.TestSDKAutoGeneratedCode]
class AutoGeneratedProgram {static void Main(string[] args){}}

Who is responsible for emitting this class, and how can I disable this?

[Youssef1313]

Because AutoGeneratedProgram is unrelated to GenerateTestingPlatformEntryPoint.

AutoGeneratedProgram is generated by VSTest, while TestingPlatform part of the name of GenerateTestingPlatformEntryPoint is referring to Microsoft.Testing.Platform, which is basically an alternative to VSTest. See Microsoft.Testing.Platform overview docs.

The way to disable AutoGeneratedProgram in VSTest is <GenerateProgramFile>false</GenerateProgramFile>. But disabling will cause a lot of trouble and doesn't look like how things were designed to work.

[drieseng]

@Youssef1313, thanks for the clear feedback. One more question: how does one opt into Microsoft.Testing.Platform?

[Youssef1313]

See MSTest Runner overview for instructions. However, you will still be building as executable so it won't fix anything for CA1515, but it's good to migrate for other reasons, including performance.

[Youssef1313]

@drieseng I'm curious though, as the analyzer is disabled by default. Do you have something like <AnalysisLevel>latest-All</AnalysisLevel> that enables this rule?

[drieseng]

@Youssef1313, indeed.