Run CodeQL analysis on a particular sub-directory in monorepo

[amalay]

Hi Guys, I have monorepo and it contains multiple services written on multiple languages and separated by sub-directory. I want to run codeQL analysis on sub-directory level only. And it is working perfectly for JavaScript, TypeScript, Python sub-directories using paths/paths-ignore in codeql-config.yml file as mentioned in GitHub doc. But the same is not working for go language sub-directory. For go lang, it is start scanning all the repository's folders instead of specific go folder. Is there any limitation with go lang? or something different need to be done for go lang?

Below is my workflow:

name: Lib - Scanning CodeQL

on:
  workflow_call:
    inputs:
      WORKING_DIRECTORY:
        required: true
        type: string
        description: "Working directory of the project"
      SCRIPTING_LANGUAGES:
        required: true
        type: string
        description: "Scripting languages used in the project"

jobs:
  codeql_analysis:
    name: CodeQL
    runs-on: ubuntu-latest

    defaults:
      run:
        working-directory: ${{ inputs.WORKING_DIRECTORY }}

    permissions:
      actions: read
      contents: read
      security-events: write

    strategy:
      fail-fast: false
      
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
        
      - name: Initialize CodeQL
        uses: github/codeql-action/init@v2
        with:          
          config-file: ${{ inputs.WORKING_DIRECTORY }}/codeql-config.yml
          languages: ${{ inputs.SCRIPTING_LANGUAGES }}          
          
      # Autobuild attempts to build any compiled languages (C/C++, C#, or Java). If this step fails, then you should remove it and run the build manually.
      - name: Autobuild
        id: autobuild
        uses: github/codeql-action/autobuild@v2
        continue-on-error: true

      # If the Autobuild fails above, then build it manually.
      - if: steps.autobuild.outcome == 'failure'
        name: Manual Build
        run: |
          make bootstrap
          make release      

      - name: Perform CodeQL Analysis
        uses: github/codeql-action/analyze@v2

My codeql config file:

name: 'CodeQL Config File'

paths:
  - go-hello-world-preview-app/app

paths-ignore:
  - brightskies-preview-app
  - brightskies-reporting-app
  - mdr-ds-preview-app
  - mdr-ds-static-web-app
  - mdr-preview-app
  - sirion
  - soc-preview-app
  - '**/*.yaml'
  - '**/*.yml'
  - '*.yaml'
  - '*.yml'

[smowton]

Yes: for Golang you need to set the environment variable CODEQL_EXTRACTOR_GO_BUILD_TRACING=on to override the default index-everything behaviour, and then replace the autobuild step with a manual build command that go builds only the .go files you want to be indexed.

[amalay]

Hi @smowton, I noticed that after setting CODEQL_EXTRACTOR_GO_BUILD_TRACING=on, Extracting step is not showing. So not sure it is analyzing correctly. See the screenshot for more details:

Thanks

[smowton]

Your yaml file should look something like this: https://github.com/chenrui333/actionlint/blob/a8724183a3c0c394f93750446b18b79d12463d33/.github/workflows/codeql.yaml#L16

Note that user has set the env variable globally, retains the codeql-action/init step, replaces the autobuild step (for go; you can omit it completely if Go is your only language of interest), then runs a build, then finally the codeql-action/analyze step.

The No rule to make target 'bootstrap'. error is not to do with CodeQL -- this is an ordinary Actions step and is trying to run make bootstrap in the root directory of your repository. What happens if you do that in a fresh checkout at the command-line? Do you need to make -C into some subdirectory?

[amalay]

Yes. It is identical to the above link. See my below yaml file

name: Lib - Scanning CodeQL

on:
  workflow_call:
    inputs:
      WORKING_DIRECTORY:
        required: true
        type: string
        description: "Working directory of the project"
      SCRIPTING_LANGUAGES:
        required: true
        type: string
        description: "Scripting languages used in the project"

env:      
  CODEQL_EXTRACTOR_GO_BUILD_TRACING: on  

jobs:
  codeql_analysis:
    name: CodeQL
    runs-on: ubuntu-latest

    defaults:
      run:
        working-directory: ${{ inputs.WORKING_DIRECTORY }}

    permissions:
      actions: read
      contents: read
      security-events: write

    strategy:
      fail-fast: false
      
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
        
      - name: Initialize CodeQL
        uses: github/codeql-action/init@v2
        with:          
          config-file: ${{ inputs.WORKING_DIRECTORY }}/codeql-config.yml
          languages: ${{ inputs.SCRIPTING_LANGUAGES }}
          # debug: true          
          
      # Autobuild attempts to build any compiled languages (C/C++, C#, or Java). If this step fails, then you should remove it and run the build manually.      
      - if: ${{ inputs.SCRIPTING_LANGUAGES != 'go' }}
        name: Autobuild
        id: autobuild
        uses: github/codeql-action/autobuild@v2
                     
      - if: ${{ inputs.SCRIPTING_LANGUAGES == 'go' }}
        name: Setup Go
        uses: actions/setup-go@v3
        with:
          go-version: '>=1.17.0'
      
      - if: ${{ inputs.SCRIPTING_LANGUAGES == 'go' }}
        name: Build Go  
        run: go build main.go

      - name: Perform CodeQL Analysis
        uses: github/codeql-action/analyze@v2

[amalay]

Hi @smowton, After comparing mine GitHub action result with the GitHub action result taken from the link you shared (https://github.com/rhysd/actionlint/runs/7368584815?check_suite_focus=true), I noticed that at both places, "Extracting go" steps is not coming/displaying while Performing CodeQL Analysis. Though it is showing the code analysis report.

If it is a correct behavior then it is good.

[smowton]

This looks good to me -- that "extracting go" phase is trawling for all Go in your repository. With CODEQL_EXTRACTOR_GO_BUILD_TRACING set, we instead follow your custom build process and see what Go files it touches, so there is no one step labelled "Extracting".