Julia default version suggestions

[fonsp]

Hey all! Thanks for the amazing work on repo2docker!! I was looking through the code and I have a suggestion for using Julia's Manifest.toml, and I can answer some questions about the default Julia version.

New: Julia version in Manifest

Since Julia 1.7, Manifest.toml has a new format (the old one is still backwards compatible). One feature of this new format is that the Manifest.toml contains a top-level field julia_version with the exact Julia version number that was used to resolve the Manifest:

https://pkgdocs.julialang.org/v1/toml-files/#Manifest.toml-entries

I think this is a good fit for repo2docker, as it allows automatically reproducing the exact Julia version. My suggestion is to use this as follows:

• Download json with all Julia version numbers (already implemented)
• Try to read&parse Manifest.toml and try to get a top-level field julia_version. If this version is in the downloaded list, use it 🎉
• Otherwise, continue with the Project.toml-based algorithm as currently implemented.

This will not change anything for older repositories that use the old Manifest format, since the top-level julia_version is not defined.

Questions in comments about the current semver algorithm

This code:

repo2docker/repo2docker/buildpacks/julia/julia_project.py

Lines 49 to 67 in 239c4f5

	try:
	# For Project.toml files, install the latest julia version that
	# satisfies the given semver.
	compat = project_toml["compat"]["julia"]
	except:
	# Default version which needs to be manually updated with new major.minor releases
	#
	# NOTE: Updates to the default version should go hand in hand with
	# updates to tests/julia/project/verify where is intent to
	# test the version.
	#
	# FIXME: When compat = "1.6" is set below and passed to
	# find_semver_match, we get 1.8.2 as of 2022-10-09. We should
	# clarify the desired behavior and have a unit test of
	# find_semver_match to validate it.
	#
	compat = "1.6"
	match = find_semver_match(compat, self.all_julias)

finds the latest Julia version that is semver-compatible with the compat entry for julia in Project.toml. This seems to be working according to Julia Pkg spec, and this means:

Compat entry "1.6" is the same as "^1.6" (caret is default), which means [1.6.0, 2.0.0), see https://pkgdocs.julialang.org/v1/compatibility/#Caret-specifiers

So right now, this means that the latest Julia version will be installed: 1.11.1.

This FIXME comment:

The commented test is indeed meant to fail: the latest Julia version is intentional (because the fallback compat version is "1.6" which will resolve to latest 1.x).

If you want to test that compat can be used to define something other than the latest version, you could add this to Project.toml:

[compat]
julia = "~1.7"

And check v"1.7.0" <= VERSION < v"1.8.0".

This FIXME comment

The observed behaviour in the comment seems correct, the comment can be removed.

The default compat entry

repo2docker/repo2docker/buildpacks/julia/julia_project.py

Line 65 in 239c4f5

compat = "1.6"

This does not seem fully intentional. I think two good options here are:

• "1" (install the latest 1.y version of Julia) this is the same behaviour as currently, but
• "~1.10" (install 1.10.z) the current LTS cycle of Julia, this used to be 1.6 until this month.