Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't retrieve outputs from SecHub GitHub Action #3481

Closed
adobryn opened this issue Oct 2, 2024 · 10 comments · Fixed by #3734
Closed

Can't retrieve outputs from SecHub GitHub Action #3481

adobryn opened this issue Oct 2, 2024 · 10 comments · Fixed by #3734
Assignees
Labels
bug Something isn't working github-action

Comments

@adobryn
Copy link

adobryn commented Oct 2, 2024

Situation

I have a workflow that uses SecHub scan action outputs and sends a message to a Teams channel. It worked fine, but since last Friday, all output values have been null without any code changes from my side:

      - id: sechub_scan
        uses: mercedes-benz/sechub/github-actions/scan@master
        with:
          url: https://sechub.url/
          api-token: ${{ API_TOKEN }}
          user: ${{ USER }}
          project-name: 'project'
          version: 'latest'
          scan-types: codeScan, secretScan
          report-formats: 'html'
          fail-job-with-findings: true
          
      - name: Check SecHub scan traffic light
        run: |
          echo "DEBUG: Scan traffic light: ${{ steps.sechub_scan.outputs.scan-trafficlight }}"
          echo "DEBUG: Scan summary: ${{ steps.sechub_scan.outputs.scan-readable-summary }}"

Further investigation

Debugging showed that the SecHub action creates the outputs:

##[debug]Set output scan-trafficlight = GREEN
##[debug]Set output scan-findings-count = 0
##[debug]Set output scan-findings-high = 0
##[debug]Set output scan-findings-medium = 0
##[debug]Set output scan-findings-low = 0
##[debug]Set output scan-readable-summary = SecHub reported traffic light color GREEN without findings

but they can't be retrieved in the next action :

##[debug]', steps.sechub_scan.outputs.scan-trafficlight, steps.sechub_scan.outputs.scan-readable-summary)
##[debug]Evaluating format:
##[debug]..Evaluating String:
##[debug]..=> 'echo "DEBUG: Scan traffic light: {0}"
##[debug]echo "DEBUG: Scan summary: {1}"
##[debug]'
##[debug]..Evaluating Index:
##[debug]....Evaluating Index:
##[debug]......Evaluating Index:
##[debug]........Evaluating steps:
##[debug]........=> Object
##[debug]........Evaluating String:
##[debug]........=> 'sechub_scan'
##[debug]......=> Object
##[debug]......Evaluating String:
##[debug]......=> 'outputs'
##[debug]....=> Object
##[debug]....Evaluating String:
##[debug]....=> 'scan-trafficlight'
##[debug]..=> null
##[debug]..Evaluating Index:
##[debug]....Evaluating Index:
##[debug]......Evaluating Index:
##[debug]........Evaluating steps:
##[debug]........=> Object
##[debug]........Evaluating String:
##[debug]........=> 'sechub_scan'
##[debug]......=> Object
##[debug]......Evaluating String:
##[debug]......=> 'outputs'
##[debug]....=> Object
##[debug]....Evaluating String:
##[debug]....=> 'scan-readable-summary'
##[debug]..=> null
##[debug]=> 'echo "DEBUG: Scan traffic light: "
##[debug]echo "DEBUG: Scan summary: "
##[debug]'
##[debug]Result: 'echo "DEBUG: Scan traffic light: "
##[debug]echo "DEBUG: Scan summary: "

I also tried waiting 20 seconds to ensure the action was completed and used if: always(), but it didn't help.

Do you have any suggestions on how to resolve this? Thank you for your help!

upd: with previous version of client 1.7.0 everything works as expected

@haerter-tss
Copy link
Member

Thank you for bringing this to our attention!
We will look into this issue and let you know once we know more.

@haerter-tss
Copy link
Member

@adobryn We had a bug in our Github Action, the scan was started in a wrong directory. The issue was fixed and a new version of the Action has been released. Please rerun your scans and check if everything is now working for you.

@haerter-tss haerter-tss assigned haerter-tss and unassigned sven-dmlr Oct 10, 2024
@haerter-tss haerter-tss added the bug Something isn't working label Oct 10, 2024
@haerter-tss
Copy link
Member

Apparently the fix of the Github Action did not help with this bug

@sven-dmlr sven-dmlr self-assigned this Oct 16, 2024
@sven-dmlr
Copy link
Member

Could reproduce the behavior.

  • mercedes-benz/sechub/github-actions/scan@master

    • empty variables with 'latest' / SecHub client 1.8.1
    • empty variables with SecHub client 1.7.0
  • Action code from Mo 10. Jun 16:48:23 CEST 2024 (last release)

    • works as expected for 'latest' / SecHub client 1.8.1
    • works as expected for SecHub client 1.7.0

Bottom line: The latest action produces the problem.

@sven-dmlr
Copy link
Member

Maybe the thread here is helpful: actions/toolkit#1218

@sven-dmlr sven-dmlr assigned hamidonos and unassigned sven-dmlr Oct 17, 2024
sven-dmlr added a commit that referenced this issue Oct 28, 2024
sven-dmlr added a commit that referenced this issue Oct 28, 2024
also print content of action.yml
sven-dmlr added a commit that referenced this issue Oct 28, 2024
sven-dmlr added a commit that referenced this issue Oct 28, 2024
sven-dmlr added a commit that referenced this issue Oct 28, 2024
sven-dmlr added a commit that referenced this issue Oct 28, 2024
sven-dmlr added a commit that referenced this issue Oct 28, 2024
sven-dmlr added a commit that referenced this issue Oct 30, 2024
but with recent core library
@de-jcup de-jcup self-assigned this Nov 27, 2024
@de-jcup
Copy link
Member

de-jcup commented Dec 12, 2024

Analyze

Test workflow

Created a SecHub github workflow like this:

name: Test (de-jcup) - SecHub code scan

on:
  # enable manual triggering of workflow
  workflow_dispatch:

jobs:
  sechub-scan:
    runs-on: [testrunner-amd64-linux]

    steps:
    - name: Check out this repo
      id: repo_checkout
      uses: actions/checkout@v3
 
   - name: Setup Node.js
      uses: actions/setup-node@0a44ba7841725637a19e28fa30b79a866c81b0a6
      with:
        node-version: 22
   
    - name: Show NPM information about core actions
      id: show_npm_info_actions_core
      run: npm info @actions/core
        
    - name: SecHub scan
      id: sechub_scan
      uses: mercedes-benz/sechub/github-actions/scan@master
      with:
        project-name: ${{ vars.SECHUB_PROJECT }}
        url: https://mysechub-server.example.org
        user: ${{ secrets.SECHUB_USERID }}
        api-token: ${{ secrets.SECHUB_APITOKEN }}
        fail-job-with-findings: false
        
    - name: Print outputs of action
      id: print_outputs
      if: always()
      run: |
        # echo "scan-trafficlight:  '${{ steps.sechub_scan.outputs.scan-trafficlight }}'"
        # echo "scan-findings-count '${{ steps.sechub_scan.outputs.scan-findings-count }}'"
        # echo "scan-findings-high   '${{ steps.sechub_scan.outputs.scan-findings-high }}'"
        # echo "scan-findings-medium '${{ steps.sechub_scan.outputs.scan-findings-medium }}'"
        # echo "scan-findings-low    '${{ steps.sechub_scan.outputs.scan-findings-low }}'"
        # echo "readable-summary    '${{ steps.sechub_scan.outputs.scan-readable-summary }}'"
        echo "greeting=Hello World" >> "$GITHUB_OUTPUT"  # Test Output
        echo "additional=Additoinal info" >> "$GITHUB_OUTPUT"  # Test Output 2
    
    - name: Print greeting output by step access
      id: test_output_access_to_print_outputs_by_other_step
      if: always()
      run: |
        # echo "greeting:  '${{ steps.print_outputs.outputs.greeting }}'"
        # echo "greeting:  '${{ steps.print_outputs.outputs.additional }}'"
    - name: Show all github output files
      id: output_file
      if: always()
      run: |
        echo "GITHUB_OUTPUT=$GITHUB_OUTPUT"
        DIRECTORY=$(dirname -- "$GITHUB_OUTPUT")

        # Check if the directory exists
        if [ ! -d "$DIRECTORY" ]; then
          echo "Directory does not exist."
          exit 1
        fi

        # Iterate through each file in the directory
        for FILE in "$DIRECTORY"/*; do
          # Check if the file name starts with "set_output"
          if [[ $(basename "$FILE") == set_output* ]]; then
            echo "----------------------------------------------------------------------------------------------------"
            echo "Contents of $FILE:"
            echo "----------------------------------------------------------------------------------------------------"
            cat "$FILE"
            echo # Add a newline for better readability
          fi
        done
        
    - name: Print Context Information
      if: always()
      env:
        CONTEXT: ${{ toJson(steps) }}
      run: echo "$CONTEXT"

Results

Core action version

@actions/[email protected] | MIT | deps: 2 | versions: 28
Actions core lib
https://github.com/actions/toolkit/tree/main/packages/core

keywords: github, actions, core

dist
.tarball: https://registry.npmjs.org/@actions/core/-/core-1.11.1.tgz
.shasum: ae683aac5112438021588030efb53b1adb86f172
.integrity: sha512-hXJCSrkwfA46Vd9Z3q4cpEpHB1rL5NG04+/rbqW9d3+CSvtB1tYe8UTpAlixa1vj0m/ULglfEK2UKxMGxCxv5A==
.unpackedSize: 90.9 kB

"Funny" remark: core 1.11.1 cannot be found as a release tag at GitHub toolkit tags I was forced to download the tarball and inspect the output...

The found javascript content

/**
 * Sets the value of an output.
 *
 * @param     name     name of the output to set
 * @param     value    value to store. Non-string values will be converted to a string via JSON.stringify
 */
// eslint-disable-next-line @typescript-eslint/no-explicit-any
function setOutput(name, value) {
    const filePath = process.env['GITHUB_OUTPUT'] || '';
    if (filePath) {
        return (0, file_command_1.issueFileCommand)('OUTPUT', (0, file_command_1.prepareKeyValueMessage)(name, value));
    }
    process.stdout.write(os.EOL);
    (0, command_1.issueCommand)('set-output', { name }, (0, utils_1.toCommandValue)(value));
}
exports.setOutput = setOutput;

was similar to https://github.com/actions/toolkit/blob/main/packages/core/src/core.ts#L192 :

export function setOutput(name: string, value: any): void {
  const filePath = process.env['GITHUB_OUTPUT'] || ''
  if (filePath) {
    return issueFileCommand('OUTPUT', prepareKeyValueMessage(name, value))
  }

  process.stdout.write(os.EOL)
  issueCommand('set-output', {name}, toCommandValue(value))
}

Content of set_report* files on runner

github-output-test-workflow1

The green area is from node-version, the blue one from the test step inside the workflow (The output content is as expected).

The red marked area is from SecHub scan action which uses the core.setOutput(...) method.

Outputs of step access and context print

As shown in next figure, the context output (green) contains the parts which were
directly written to the environment file.
The blue area shows, it is accessible from another step.

github-output-test-workflow2

Final result

It looks like using the core.setOutput(...) method (which we currently use in SecHub github action) writes the information in a way to the file that is not readable from outside.

As a solution we try to create a wrapper here which does no longer use this method, but instead writes to the file directly.

de-jcup added a commit that referenced this issue Dec 13, 2024
- introduced output helper
- storing now outputs directly to
  file by own helper method
  instead of using core.setOutput(..)
@de-jcup
Copy link
Member

de-jcup commented Dec 13, 2024

Writing directly to the file with correct content did NOT solve the problem.
The outputs were still empty.

Reading https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/

To avoid untrusted logged data to use save-stateand set-output workflow commands without the intention of the workflow author we have introduced a new set of environment files to manage state and output.

Important here IMO: "without the intention of the workflow author"

I tried out to directly call

 issueCommand('set-output', {name}, toCommandValue(value))

by the wrapper method, but inside the logs I found also the deprecation warning. And the output of the context did still not contain the wanted outputs from sechub scan action.

@sven-dmlr
Copy link
Member

sven-dmlr commented Dec 13, 2024

'set-output' command is deprecated.
This alternative approach was also not successful:
It writes directly in $GITHUB_OUTPUT file as recommended by Github.

const filePath = process.env[`GITHUB_OUTPUT`];
if (!filePath) {
  throw new Error(`Empty environment variable GITHUB_OUTPUT`);
 }
if (!fs.existsSync(filePath)) {
  throw new Error(`No access to file ${filePath}`);
}
fs.appendFileSync(filePath, `${field}=${valuestring}${os.EOL}`);

(See branch feature-3481-make-output-work-again)

sven-dmlr added a commit that referenced this issue Dec 16, 2024
sven-dmlr added a commit that referenced this issue Dec 16, 2024
sven-dmlr added a commit that referenced this issue Dec 16, 2024
sven-dmlr added a commit that referenced this issue Dec 16, 2024
@de-jcup
Copy link
Member

de-jcup commented Dec 16, 2024

IMO this is a bug in GitHub and I created actions/toolkit#1906

Because we do not know when/if this will be fixed, we decided to switch to environment variables instead!

For example: scan-readable-summary will become SECHUB_OUTPUT_SCAN_READABLE_SUMMARY
and can be access by ${env.SECHUB_OUTPUT_SCAN_READABLE_SUMMARY} inside other steps (after the sechub scan action step)

de-jcup added a commit that referenced this issue Dec 16, 2024
- directly writing to the file did not work
- directly using set-output command did not work
- using SECHUB_OUTPUT* environment values as alternative
- changed documentation
- wrote tests
de-jcup added a commit that referenced this issue Dec 17, 2024
@de-jcup
Copy link
Member

de-jcup commented Dec 17, 2024

Remark: The changes (environment variables instead of outputs) will be available when the GitHub action has been released.

@sven-dmlr sven-dmlr added this to the Github action 2.3.0 milestone Dec 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment