zstd support

[ricea]

Issue for discussion of adding support for the zstd format to the API.

[ricea]

Currently libzstd is not linked into Chromium. It would add roughly 700KB to the binary size (maybe can be trimmed a bit). This would be a hard sell if the compression standard is the only user.

[evanstade]

It seems that you did manage to trim the size a bit as this change only added 38k. Was that just by dropping everything besides decompression, or were there other tricks you made use of?

On the storage team we are also interested in using zstd for compressing IndexedDB values before sending them over IPC (or storing as blobs). Adding the compression half of zstd appears to add another 200kB.

So that's at least two users of zstd.

[ricea]

We didn't do anything special in Chromium to reduce the size increase, except for only compiling in the decompression code. I think we were just lucky.

I would like to move this specification to WHATWG before we add new features, since it is now implemented in Firefox and Safari and we would like to have consensus.

That means it will probably be some months before there's movement on adding zstd.

[saschanaz]

I would like to move this specification to WHATWG before we add new features, since it is now implemented in Firefox and Safari and we would like to have consensus.

Why would it take that much to move to WHATWG?

[telgareith]

Note: Chrome/Chromium, Firefox, and Opera have agreed to add zstd to content-encoding options. Chrome is planning on General Availability in 118.

[Jedipedia]

Another note: Firefox now supports Content-Encoding: zstd as of version 126 but there's still no support for new DecompressionStream("zstd") in either Firefox or Chromium.

[xpl]

Please add it to the specification, as it seemingly blocks from implementing new DecompressionStream("zstd") in browsers:

https://issues.chromium.org/issues/348499728

[saschanaz]

#34 (comment)

Just as brotli, we'd be open to ship only DecompressionStream support for zstd, as @jesup reported 600KB size increase to ship the compression side of the library.

[ricea]

#34 (comment)

Just as brotli, we'd be open to ship only DecompressionStream support for zstd, as @jesup reported 600KB size increase to ship the compression side of the library.

Thanks. I think this will also be Blink's position.

[nektro]

Bun would be interested in supporting this since we already ship with zstd support internally for other APIs