[WebGPU] Support crosscompiling on windows platform #22777
Conversation
Compile onnxruntime with --use_webgpu triggers dawn's compilation. Dawn build dxc from sources in third_party folders. However, dxc has poor support for crosscompilation (e.g. build arm64 on x64 platform). See details: microsoft/DirectXShaderCompiler#7000 This PR support crosscompilation through: - Build llvm-tblgen and clang-tblgen in configuration step with host cpu arch. - Use the built tool do build dxc and dawn - Copy dxil.dll from correct target arch folder (e.g. target is arm64 and copy from arm64/ folder instead of x64 folder always) Using local patch to host these changes for Dawn.
|
|
||
| +# Disable CMAKE_CROSSCOMPING because we don't have full support | ||
| +if(CMAKE_CROSSCOMPILING) | ||
| + set(CMAKE_CROSSCOMPILING OFF) |
There was a problem hiding this comment.
But we are doing cross-compiling? Usually the var is set to ON when the CMAKE_SYSTEM_NAME variable has been set manually. And usually we only read from this var, we do not set it. I worry a lot of other cmake functions/modules may rely on this var.
There was a problem hiding this comment.
I think we can remove this. Let me have a try locally.
| + file(MAKE_DIRECTORY ${CMAKE_BINARY_DIR}/native-tools) | ||
| + endif() | ||
| + | ||
| + if (NOT CLANG_TBLGEN OR NOT LLVM_TBLGEN) |
There was a problem hiding this comment.
This part is complicated. I think we may ask the user to provide a prebuilt binary instead. Like what we do now for protoc.exe .
There was a problem hiding this comment.
When we build onnxruntime on Windows, we also pass in a lot of other cmake parameters like "-A Win32 --toolset 14.40". These flags cannot be handled here.
There was a problem hiding this comment.
This part is complicated. I think we may ask the user to provide a prebuilt binary instead. Like what we do now for protoc.exe
It would be ideal that user could provided the pre-baked tool for this.
When we build onnxruntime on Windows, we also pass in a lot of other cmake parameters like "-A Win32 --toolset 14.40". These flags cannot be handled here.
My understanding is that in crosscompilation situation, the llvm-talgen and clang-tblgen are built in default behaviour? (Because we don't have extra params to define the behaviour)
There was a problem hiding this comment.
(Sry for late reply due to some OOO)
By thinking it again, one thing I need to point out is that provide a prebuilt binary for user is not that straightforward. I tried the released llvm package contains llvm-tblgen and clang-tblgen tools and that doesn't work with dxc compilation.
And considering it is used by dawn project and dawn set some flags for the self-built dxc, it is a bit complicated for them to provided the proper tool.
Another thing is that the code here is a quick workable way. Would it be possible to replace these execute_process calls with custom_target + custom command and proper deps relationships(I'm not so familiar with CMake)?
There was a problem hiding this comment.
the released llvm package contains llvm-tblgen and clang-tblgen tools and that doesn't work with dxc compilation.
No, it would not work. We need the special DXC one.
I was just thinking, if we could achieve the goal without patching dawn, I'd prefer not doing that.
There was a problem hiding this comment.
I noticed that with cmake flag CMAKE_CROSSCOMPILING, onnxruntime generated NATIVE folder in build-dir and configs with host-cpu-os info (e.g. Generated visual studio project files with x64 arch while target platform is arm64 by setting -A flag). Several third_party repo works but dawn doesn't. Do you think it is possible that we could rely on this?
Another thoughts is that we might host dxc as a third_party repo and compile tools from it when cross-compilation is required.
|
fix is working for me |
|
not merged yet because we are discussing if we can use prebuilt binaries. |
Compile onnxruntime with --use_webgpu triggers dawn's compilation. Dawn build dxc from sources in third_party folders. However, dxc has poor support for crosscompilation (e.g. build arm64 on x64 platform).
See details: microsoft/DirectXShaderCompiler#7000
This PR support crosscompilation through:
Using local patch to host these changes for Dawn.
Description
Motivation and Context