Lightweight ZSTD with support for Android

Hello, I noticed the lack of a proper Java-only implementation of ZSTD (I know zstd-jni is fine, but it uses native libraries and is kind of heavy too) that also supports Android

So meet the murine-aircompressor library

I noticed version 2.0 of aircompressor was exclusively Java-only and used JDK 8, the maintainer claimed Unsafe was not available for Android and therefore Android support was out of scope for it; however, that turns out not to be the case, Unsafe can be used on Android and there was only one single method and a static field which are missing on the Android implementation of Unsafe, I just made a helper method for the former, choosing the right implementation, while the static field was just a call to an existing method, and Bob's your uncle.

But not satisfied, I made a few changes, a lot of which are kanged from other forks the maintainer ignored:

- improved compression a lot (original fork that implemented this change here) - added an option to switch to the former behaviour too

- added a ring buffer mode and set as degault (also kanged from another fork, seems to improve performance over largely incompressible data)

- exposed levels for the streaming compressor

- added FAST algorithm to also support levels 1-2 on top of 3-4 which were the only supported ones in the original (level 2 is also pretty good and way faster; Claude is, however, mostly to praise [or blame] for this one)

- Improved overall performance on ARM (again, Claude is mostly to praise/blame here, but I did a lot of tests)

- added parallel compression support because fuck it

- added a wittle tar (with PAX) implementation because fuck it again, I wanted to make .tar.zst without qdding extra libraries

Using WindowSlideMode.HIGH_SPEED resets the library's (ORIGINAL) shit compression ratio for the stream, but makes it faster, I suppose; I default to HIGH_COMPRESSION which is still decently fast, and if you want it to be faster, it's much much better to use level 2 with HIGH_COMPRESSION than the default level 3 with HIGH_SPEED in my experience

Overall, the library is a little slower than zstd-jni with the (now) default parameters, but now matches its compression ratio, and may be a good compromise you'd rather want a lightweight full-java implementation than the heavy .so files in your apk

EDIT: I also noticed this may sound like I'm trashing the maintainer, I'm not lol, just stating facts, the maintainer is awesome for creating the library on the first place

submitted by /u/G3nghisKang
[link] [comments]