Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Allow setting a command prefix for the sysimage build command #870

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

sloede
Copy link
Collaborator

@sloede sloede commented Oct 20, 2023

This PR adds a new, optional sysimage_build_prefix kwarg to create_{sysimage,app,library}. It aims to allow one to prepend the build command called when creating a sysimage with a user-specified command prefix.

The main motivation is to be able to call PackageCompiler.jl from within an MPI-started process. This is sometimes required on supercomputers, where calling MPI functions without having started a process via mpirun/srun/aprun etc. is disabled.

cc @lchristm

@codecov
Copy link

codecov bot commented Oct 20, 2023

Codecov Report

Merging #870 (38c58d8) into master (846292c) will decrease coverage by 0.68%.
Report is 4 commits behind head on master.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #870      +/-   ##
==========================================
- Coverage   84.94%   84.26%   -0.68%     
==========================================
  Files           3        3              
  Lines         797      801       +4     
==========================================
- Hits          677      675       -2     
- Misses        120      126       +6     
Files Coverage Δ
src/PackageCompiler.jl 92.57% <100.00%> (-0.87%) ⬇️

It is not necessary to provide a prefix in the build phase but it may be
necessary when the script given by `precompile_execution_file` is executed,
e.g. if some package requires the script to be run in an MPI context
via `mpirun`.
Thus, this reverts commit d3b7edc.
This change adds some flexibility to how precompile execution files
are executed by PackageCompiler to generate precompile statements.

In some cases, for example when one wants to use precompile statements
generated from a script that uses a package that uses MPI and requires
that the Julia process is started via `mpiexec` or similar,
one currently has to execute the script manually and use Julia's
`--trace-compile` option to generate precompile statements and pass
them manually to PackageCompiler afterwards.

The new `precompile_execution_prefix` keyword argument introduced by
this commit allows one to skip manual generation of precompile
statements in such cases by passing the necessary prefix to
PackageCompiler.
@lchristm lchristm force-pushed the msl/allow-sysimage-build-command-prefix branch from 22fa0de to 38c58d8 Compare October 26, 2023 15:54
@lchristm
Copy link

@sloede The prefix is only needed for executing scripts passed via precompile_execution_file and not for building the system image afterwards. I changed the PR branch accordingly but I think it would be reasonable to change the PR title as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants