Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set JULIA_PROJECT when running the subprocess in ensurecompiled() #995

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

DilumAluthge
Copy link
Member

@DilumAluthge DilumAluthge commented Oct 29, 2024

Suppose that you have two different projects, where one project contains the packages that will go into the sysimage, and the other project is a "build project".

As an example, consider the examples/MyLib folder in this repo:

  • The examples/MyLib/Project.toml is the "main project" that contains the packages that will go into the sysimage.
  • The examples/MyLib/build/Project.toml is the "build project". The packages in the "build project" will not go into the sysimage. In this example, the "build project" contains Libdl and PackageCompiler, but neither of those will go into the sysimage.

When the ensurecompiled() function is run, we want to make sure that it precompiles the "main project", not the "build project". That's what this PR does.

@DilumAluthge DilumAluthge changed the title WIP: Set JULIA_PROJECT when running the subprocess in `ensurecompiled() WIP: Set JULIA_PROJECT when running the subprocess in ensurecompiled() Oct 29, 2024
@DilumAluthge DilumAluthge changed the title WIP: Set JULIA_PROJECT when running the subprocess in ensurecompiled() Set JULIA_PROJECT when running the subprocess in ensurecompiled() Oct 29, 2024
@DilumAluthge DilumAluthge force-pushed the dpa/ensure-precompiled branch from de20427 to 2ee4534 Compare October 29, 2024 22:36
@DilumAluthge DilumAluthge marked this pull request as ready for review October 29, 2024 22:36
@DilumAluthge DilumAluthge force-pushed the dpa/ensure-precompiled branch from aa7ffc9 to b38c01c Compare October 29, 2024 22:39
Copy link

codecov bot commented Oct 29, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.75%. Comparing base (530b392) to head (6b59a30).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #995      +/-   ##
==========================================
+ Coverage   87.73%   87.75%   +0.01%     
==========================================
  Files           3        3              
  Lines         799      800       +1     
==========================================
+ Hits          701      702       +1     
  Misses         98       98              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.


🚨 Try these New Features:

@KristofferC
Copy link
Member

I would have think this is already the case from the existing code? Do you have something like an MWE where it isn't?

julia> Base.active_project()
"/home/kc/JuliaPkgs/PackageCompiler.jl/examples/MyLib/build/Project.toml"

julia> project = ".."
".."

julia> run(addenv(`$(Base.julia_cmd()) -e '@show Base.load_path()'`, "JULIA_LOAD_PATH" => "$project:@stdlib"));
Base.load_path() = ["/home/kc/JuliaPkgs/PackageCompiler.jl/examples/MyLib/Project.toml", "/home/kc/.julia/juliaup/julia-1.10.6+0.x64.linux.gnu/share/julia/stdlib/v1.10"]

@DilumAluthge
Copy link
Member Author

Good question. I haven't been able to come up with a MWE. I'm encountering some kind of problem (that I thought was JULIA_PROJECT-related) when I try to move the build-mylib test into the PackageCompiler test suite (instead of having it as a separate CI job). That's what I've been trying in #991.

@DilumAluthge DilumAluthge marked this pull request as draft November 21, 2024 17:23
@DilumAluthge DilumAluthge removed the request for review from KristofferC November 21, 2024 17:23
@DilumAluthge DilumAluthge force-pushed the dpa/ensure-precompiled branch from b38c01c to 1f98bca Compare November 23, 2024 18:08
@DilumAluthge DilumAluthge force-pushed the dpa/ensure-precompiled branch from 1f98bca to 6b59a30 Compare November 24, 2024 07:36
@DilumAluthge
Copy link
Member Author

DilumAluthge commented Nov 25, 2024

Okay, so I do have an example where things don't seem to work unless I have this JULIA_PROJECT change.

In #991, I'm working on moving the build-mylib test into the regular test suite (and thus get rid of the separate build-mylib CI job).

In that PR, if I don't have this JULIA_PROJECT change, then CI fails (see 8308cf8, e.g. this example log) with the following:

ERROR: The following 1 direct dependency failed to precompile:

PackageCompiler [9b87118b-4619-50d2-8e1e-99f35a4d4d9d]

Error: Missing source file for PackageCompiler [9b87118b-4619-50d2-8e1e-99f35a4d4d9d

However, when I add this JULIA_PROJECT change, then CI passes (see f4bdaaf).

The only difference between the two commits (f4bdaaf and 8308cf8) is the JULIA_PROJECT change, same as this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants