Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider supporting "tar.xz" format #10

Closed
dtolnay opened this issue Feb 8, 2024 · 2 comments
Closed

Consider supporting "tar.xz" format #10

dtolnay opened this issue Feb 8, 2024 · 2 comments

Comments

@dtolnay
Copy link

dtolnay commented Feb 8, 2024

Official rustup components are available only in tar.gz and tar.xz. The tar.xz are significantly better compressed. rustup exclusively uses the tar.xz artifacts, and the plan is to stop providing tar.gz altogether (rust-lang/infra-team#89).

https://static.rust-lang.org/dist/2023-12-28

They may or may not provide zstd in the future (rust-lang/infra-team#97).

@bolinfest
Copy link
Contributor

For sure, adding support is likely easy from a technical perspective, so it's more about what our tolerance is for increased binary size of DotSlash itself.

@bolinfest
Copy link
Contributor

Interesting. My understanding is that xz priorities size compared to extraction time. For a build cluster talking to a CAS where presumably network is fast and disk is a bigger bottleneck, I feel like that would be a bad tradeoff. Though certainly if you on a personal machine and network isn't so great, then xz is probably what you want?

abhinav added a commit to abhinav/dotslash that referenced this issue Feb 10, 2024
Some projecst are only available as .tar.xz archives.
For example, [Zig releases](https://ziglang.org/download/)
for Linux and macOS are only available as .tar.xz archives.

This PR adds support to dotslash for .tar.xz archives,
using the xz2 crate to provide the functionality.
The xz2 create wraps the liblzma library.

In this change, the library is linked statically,
so that the resulting binary does not depend on the
liblzma library being installed on the system.

**File size impact**:

```
❯ uname -s
Darwin

❯ git checkout main
❯ cargo build --release
❯ wc -c target/release/dotslash
  985208 target/release/dotslash

❯ git checkout tarxz
❯ cargo build --release
❯ wc -c target/release/dotslash
 1052776 target/release/dotslash
```

This increases the binary size by 67,568 bytes on macOS.
It comes down to 16,704 if we use dynamic linking.
For now, I've chosen to use static linking.

**Testing**:
We'll need to update https://github.com/zertosh/dotslash_fixtures to
generate a .tar.xz archive as well to be able to add a fixture for it.

Resolves facebook#10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants