MacOS 15, fail to run unit tests with webkit

[cuongtv51]

Description

i have unit tests, when tests setup, it start a webview, then tests run by request to that webview, it local run success, but on CI it alway fail with Error: Process completed with exit code 65.

Platforms affected

• Azure DevOps
• GitHub Actions - Standard Runners
• GitHub Actions - Larger Runners

Runner images affected

• Ubuntu 20.04
• Ubuntu 22.04
• Ubuntu 24.04
• macOS 12
• macOS 13
• macOS 13 Arm64
• macOS 14
• macOS 14 Arm64
• macOS 15
• macOS 15 Arm64
• Windows Server 2019
• Windows Server 2022

Image version and build link

macos-15

Is it regression?

webkit in unit tests

Expected behavior

success all tests

Actual behavior

always fail with: Error: Process completed with exit code 65.

Repro steps

on local machine: ./run_tests.sh run success.
my ci file:
`
name: Test

on:
workflow_dispatch:
push:
branches: [ master ]
pull_request:
branches: [ master ]

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
build:
name: Build and test
runs-on: macos-15
steps:
- uses: actions/checkout@v4
- name: List available Xcode versions
run: ls /Applications | grep Xcode
- name: Set up Xcode version
run: sudo xcode-select -s /Applications/Xcode_16.1.0.app/Contents/Developer
- name: Show current version of Xcode
run: xcodebuild -version
- name: Run tests
run: ./run_tests.sh
`

[aartis17]

Hi @cuongtv51,
We will look into the issue and keep you posted with updates.
Thank you.

[aartis17]

Hi @cuongtv51 ,
The workflow which you have shared, after executing it, I am not encountering any error.
The Error which you have mentioned: Process completed with exit code 65. This error often arises due to permission issues in macOS CI environments. To resolve this, ensure that all files and scripts have the necessary permissions. Specifically, check the permissions for run_tests.sh and make sure it's marked as executable by adding the chmod +x command. If you still face issue, please provide necessary files such as exact workflow files, .sh files and error screenshots. Thank you

Please see below snips:

name: Webkit Test
on: push
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  build:
    strategy:
      fail-fast: false
      matrix:
        os: [ 'macos-15-xlarge', 'macos-15-large']
    runs-on: ${{ matrix.os }}

    steps:
    - name: Checkout repository
      uses: actions/checkout@v4

    - name: List available Xcode versions
      run: ls /Applications | grep Xcode
      
    - name: Set up Xcode version
      run: sudo xcode-select -s /Applications/Xcode_16.1.0.app/Contents/Developer
      
    - name: Show current version of Xcode
      run: xcodebuild -version

    - name: Make run_tests.sh Executable 
      run: chmod +x .github/workflows/run_tests.sh
      
    - name: Run tests
      run: .github/workflows/run_tests.sh