General issue (No source code was seen and extracted to C:\***\DB-codeql)

[FaiqueAli]

No source code was seen and extracted to C:*\DB-codeql**

Hi all,

I am using codeql cli and trying to create a database of my msvc CPP project inside a docker container and getting the error No source code was seen and extracted to C:\***\DB-codeql.
The project is being compiled normally without codeql but with the codeql cli I am getting this error.
I am using this command to create a database codeql database create --ram 4096 -J-Xmx1G -vvvv --language=cpp -c path-to-file -s path-to-source

the code was being compiled previously using mingw but with msvc I am getting this issue.
the sourceLocationPrefix is also set to my project path and there is no much information available in build-tracer.log file as shown below

Initializing tracer. Initialising tags... CodeQL CLI version 2.11.3 Initializing tracer. Initialising tags... Allocated ID F7C238AA000000E0_0000000000000001 (parent ) ==== Candidate to intercept: C:\codeql\tools\win64\runner.exe (700) ==== Executing the following tracer actions: Tracer actions: pre_invocations(0) post_invocations(0) trace_languages(1): [cpp] Initializing tracer. Initialising tags... ID set to F7C238AA000000E0_0000000000000001 (parent root) Allocated ID BB65D210000002BC_0000000000000001 (parent F7C238AA000000E0_0000000000000001) ==== Candidate to intercept: C:\Windows\System32\cmd.exe (1028) ==== Executing the following tracer actions: Tracer actions: pre_invocations(0) post_invocations(0) trace_languages(1): [cpp] Initializing tracer. Initialising tags... ID set to BB65D210000002BC_0000000000000001 (parent F7C238AA000000E0_0000000000000001) Allocated ID 96C4BE6E00000404_0000000000000001 (parent BB65D210000002BC_0000000000000001) ==== Candidate to intercept: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe (8) ==== Executing the following tracer actions: Tracer actions: pre_invocations(0) post_invocations(0) trace_languages(1): [cpp] Initializing tracer. Initialising tags... ID set to 96C4BE6E00000404_0000000000000001 (parent BB65D210000002BC_0000000000000001)

looking for the suggestion
Thanks :)

[alexet]

Are you running codeql inside the docker container or does the build launch the docker conatiner? If it is the latter then it won't work. CodeQL works by tracing the build and finding all compiler invocations. However it cannot trace across system boundaries (across networks or into vms and containers). If your code is built inside a container then you must run codeql inside the container also.

[FaiqueAli]

Hi,
Thank you for your reply,

As I said it was getting compiled previously with other compiler settings (minGW) but now not.
Yes I am compiling my code inside a container that works but now trying to compile using codeql inside a container that doesnt work.

[alexet]

What is your build command. Your wriote path-to-file as the build command above but what file are you pointing to?

[FaiqueAli]

I have a shell script (xxx.ps1) file that downloads all the required dependencies before a build command executes and then it runs a build commad.
my build command is

cmd.exe /c ' "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Auxiliary\\Build\\vcvars64.bat " & cmake -G "Ninja" -S C:\\xxx C:\\xxx\\build -DCMAKE_BUILD_TYPE=Debug -DCMAKE_PREFIX_PATH="C:/Qt/5.15.10/msvc2019_64/bin" & cmake --build C:\\xxx\\build '

[FaiqueAli]

#10132
Hi @aibaars ,
Could you please have a look into this issue this is related to my previous issue but this time we have changed the compiler and getting the same issue.
Maybe I am missing something
Thanks :)

[aibaars]

@FaiqueAli Is that the entire build-tracer.log ? If not, could you attach it (as a zip file) to the issue? Make sure to check that it does not contain sensitive information though.

[FaiqueAli]

Yes this is the entire text I found inside build-tracer.log

[aibaars]

Yes this is the entire text I found inside build-tracer.log

That is a very short log. It looks like CodeQL is only intercepting cmd.exe and powershell.exe and no build commands at all. Not even cmake.exe. Are you sure any source code is built by the provided build command?

It looks like powershell.exe is either not running any further commands, or perhaps it somehow breaks the CodeQL tracer. Could you try to rewrite your ps1 script to a bat or cmd script instead?

[FaiqueAli]

Yes, I am sure, I used this command to compile/build the project.
sure, let me try to change the extension of the build script and let you know.

[FaiqueAli]

I am sorry for not updating with the issue as there were major changes in the project.
I have changed the file type to .bat but still getting the same issue.
Since we moved now to C# instead of c++, the code was compiled but at finalizing step I get this issue.
Please have a look at the attached file.
buildtracer.txt

[aibaars]

@FaiqueAli The file you attached appears to be empty.

[FaiqueAli]

ahh my bad, here we go..
buildtracer.txt

[aibaars]

I see there is some build command being run. However, CodeQL seems to be configured to look for C# code instead of C++. That's a bit strange. Just to double check you are running codeql database create -l cpp ...., right?

[T 15:43:39 1964] ==== Candidate to intercept: C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\MSBuild\Current\Bin\amd64\MSBuild.exe (1732) ====
[T 15:43:39 1964] Executing the following tracer actions:
[T 15:43:39 1964] Tracer actions:
[T 15:43:39 1964] pre_invocations(0)
[T 15:43:39 1964] post_invocations(0)
[T 15:43:39 1964] trace_languages(1): [csharp]

[FaiqueAli]

No, this time I am compiling a C# code not cpp.

[aibaars]

Ah yes indeed, I got confused. So things look good, msbuild is being run, the CodeQL tracer "sees" it, but somehow CodeQL does not start analysing the C# code.

If I recall correctly, you are running the builds in a windows docker container, right?

[aibaars]

Looking at the github/codeql repository, this commit looks relevant for analysing with msbuild on Windows 914b968 . That commit is not in CodeQL 2.10, could you try upgrading to 2.11 or 2.12?

[FaiqueAli]

Ah yes indeed, I got confused. So things look good, msbuild is being run, the CodeQL tracer "sees" it, but somehow CodeQL does not start analysing the C# code.

If I recall correctly, you are running the builds in a windows docker container, right?

You you are right, I am using a docker container to compile and execute codeql.

[FaiqueAli]

Looking at the github/codeql repository, this commit looks relevant for analysing with msbuild on Windows 914b968 . That commit is not in CodeQL 2.10, could you try upgrading to 2.11 or 2.12?

Ok, I will do that but it will take time, and I will let you inform.
Thanks :)