Historically, the MD5 checksums in our LFS files serve as a protection
against broken downloads, or accidentally corrupted source files.
While the sources are nowadays downloaded via HTTPS, it make sense to
beef up integrity protection for them, since transparently intercepting
TLS is believed to be feasible for more powerful actors, and the state
of the public PKI ecosystem is clearly not helping.
Therefore, this patch switches from MD5 to BLAKE2, updating all LFS
files as well as make.sh to deal with this checksum algorithm. BLAKE2 is
notably faster (and more secure) than SHA2, so the performance penalty
introduced by this patch is negligible, if noticeable at all.
In preparation of this patch, the toolchain files currently used have
been supplied with BLAKE2 checksums as well on
https://source.ipfire.org/.
Cc: Michael Tremer <michael.tremer@ipfire.org>
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Acked-by: Michael Tremer <michael.tremeripfire.org>
Bumping across one of our scripts with very long trailing whitespaces, I
thought it might be a good idea to clean these up. Doing so, some
missing or inconsistent licence headers were fixed.
There is no need in shipping all these files en bloc, as their
functionality won't change.
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
- Update from 1.5.1 to 1.5.2
- Update of rootfile
- Changelog
v1.5.2 (Jan, 2022)
perf: Regain Minimal memset()-ing During Reuse of Compression Contexts (@Cyan4973, #2969)
build: Build Zstd with `noexecstack` on All Architectures (@felixhandte, #2964)
doc: Clarify Licensing (@terrelln, #2981)
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Reviewed-by: Peter Müller <peter.mueller@ipfire.org>
- Update from 1.5.0 to 1.5.1
- Update of rootfile
- Changelog
v1.5.1 (Dec, 2021)
perf: rebalanced compression levels, to better match the intended speed/level curve,
by @senhuang42
perf: faster huffman decoder, using x64 assembly, by @terrelln
perf: slightly faster high speed modes (strategies fast & dfast), by @felixhandte
perf: improved binary size and faster compilation times, by @terrelln
perf: new row64 mode, used notably in level 12, by @senhuang42
perf: faster mid-level compression speed in presence of highly repetitive patterns,
by @senhuang42
perf: minor compression ratio improvements for small data at high levels, by @cyan4973
perf: reduced stack usage (mostly useful for Linux Kernel), by @terrelln
perf: faster compression speed on incompressible data, by @bindhvo
perf: on-demand reduced ZSTD_DCtx state size, using build macro
ZSTD_DECODER_INTERNAL_BUFFER, at a small cost of performance, by @bindhvo
build: allows hiding static symbols in the dynamic library, using build macro,
by @skitt
build: support for m68k (Motorola 68000's), by @cyan4973
build: improved AIX support, by @Helflym
build: improved meson unofficial build, by @eli-schwartz
cli : custom memory limit when training dictionary (#2925), by @embg
cli : report advanced parameters information when compressing in very verbose mode
(``-vv`), by @Svetlitski-FB
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Reviewed-by: Michael Tremer <michael.tremer@ipfire.org>
- Update from 1.4.9 to 1.5.0
- Update of rootfile
- Changelog
v1.5.0 (May 11, 2021)
api: Various functions promoted from experimental to stable API: (#2579-2581, @senhuang42)
`ZSTD_defaultCLevel()`
`ZSTD_getDictID_fromCDict()`
api: Several experimental functions have been deprecated and will emit a compiler warning (#2582, @senhuang42)
`ZSTD_compress_advanced()`
`ZSTD_compress_usingCDict_advanced()`
`ZSTD_compressBegin_advanced()`
`ZSTD_compressBegin_usingCDict_advanced()`
`ZSTD_initCStream_srcSize()`
`ZSTD_initCStream_usingDict()`
`ZSTD_initCStream_usingCDict()`
`ZSTD_initCStream_advanced()`
`ZSTD_initCStream_usingCDict_advanced()`
`ZSTD_resetCStream()`
api: ZSTDMT_NBWORKERS_MAX reduced to 64 for 32-bit environments (@Cyan4973)
perf: Significant speed improvements for middle compression levels (#2494, @senhuang42 @terrelln)
perf: Block splitter to improve compression ratio, enabled by default for high compression levels (#2447, @senhuang42)
perf: Decompression loop refactor, speed improvements on `clang` and for `--long` modes (#2614#2630, @Cyan4973)
perf: Reduced stack usage during compression and decompression entropy stage (#2522#2524, @terrelln)
bug: Improve setting permissions of created files (#2525, @felixhandte)
bug: Fix large dictionary non-determinism (#2607, @terrelln)
bug: Fix non-determinism test failures on Linux i686 (#2606, @terrelln)
bug: Fix various dedicated dictionary search bugs (#2540#2586, @senhuang42 @felixhandte)
bug: Ensure `ZSTD_estimateCCtxSize*() `monotonically increases with compression level (#2538, @senhuang42)
bug: Fix --patch-from mode parameter bound bug with small files (#2637, @occivink)
bug: Fix UBSAN error in decompression (#2625, @terrelln)
bug: Fix superblock compression divide by zero bug (#2592, @senhuang42)
bug: Make the number of physical CPU cores detection more robust (#2517, @PaulBone)
doc: Improve `zdict.h` dictionary training API documentation (#2622, @terrelln)
doc: Note that public `ZSTD_free*()` functions accept NULL pointers (#2521, @animalize)
doc: Add style guide docs for open source contributors (#2626, @Cyan4973)
tests: Better regression test coverage for different dictionary modes (#2559, @senhuang42)
tests: Better test coverage of index reduction (#2603, @terrelln)
tests: OSS-Fuzz coverage for seekable format (#2617, @senhuang42)
tests: Test coverage for ZSTD threadpool API (#2604, @senhuang42)
build: Dynamic library built multithreaded by default (#2584, @senhuang42)
build: Move `zstd_errors.h` and `zdict.h` to `lib/` root (#2597, @terrelln)
build: Allow `ZSTDMT_JOBSIZE_MIN` to be configured at compile-time, reduce default to 512KB (#2611, @Cyan4973)
build: Single file library build script moved to `build/` directory (#2618, @felixhandte)
build: `ZBUFF_*()` is no longer built by default (#2583, @senhuang42)
build: Fixed Meson build (#2548, @SupervisedThinking @kloczek)
build: Fix excessive compiler warnings with clang-cl and CMake (#2600, @nickhutchinson)
build: Detect presence of `md5` on Darwin (#2609, @felixhandte)
build: Avoid SIGBUS on armv6 (#2633, @bmwiedmann)
cli: `--progress` flag added to always display progress bar (#2595, @senhuang42)
cli: Allow reading from block devices with `--force` (#2613, @felixhandte)
cli: Fix CLI filesize display bug (#2550, @Cyan4973)
cli: Fix windows CLI `--filelist` end-of-line bug (#2620, @Cyan4973)
contrib: Various fixes for linux kernel patch (#2539, @terrelln)
contrib: Seekable format - Decompression hanging edge case fix (#2516, @senhuang42)
contrib: Seekable format - New seek table-only API (#2113#2518, @mdittmer @Cyan4973)
contrib: Seekable format - Fix seek table descriptor check when loading (#2534, @foxeng)
contrib: Seekable format - Decompression fix for large offsets, (#2594, @azat)
misc: Automatically published release tarballs available on Github (#2535, @felixhandte)
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
- Update from 1.4.5 to 1.4.9
- Update rootfiles
- Changelog
v1.4.9 (Mar 1, 2021)
bug: Use `umask()` to Constrain Created File Permissions (#2495, @felixhandte)
bug: Make Simple Single-Pass Functions Ignore Advanced Parameters (#2498, @terrelln)
api: Add (De)Compression Tracing Functionality (#2482, @terrelln)
api: Support References to Multiple DDicts (#2446, @senhuang42)
api: Add Function to Generate Skippable Frame (#2439, @senhuang42)
perf: New Algorithms for the Long Distance Matcher (#2483, @mpu)
perf: Performance Improvements for Long Distance Matcher (#2464, @mpu)
perf: Don't Shrink Window Log when Streaming with a Dictionary (#2451, @terrelln)
cli: Fix `--output-dir-mirror`'s Rejection of `..`-Containing Paths (#2512, @felixhandte)
cli: Allow Input From Console When `-f`/`--force` is Passed (#2466, @felixhandte)
cli: Improve Help Message (#2500, @senhuang42)
tests: Remove Flaky Tests (#2455, #2486, #2445, @Cyan4973)
tests: Correctly Invoke md5 Utility on NetBSD (#2492, @niacat)
tests: Avoid Using `stat -c` on NetBSD (#2513, @felixhandte)
build: Zstd CLI Can Now be Linked to Dynamic `libzstd` (#2457, #2454 @Cyan4973)
build: Hide and Avoid Using Static-Only Symbols (#2501, #2504, @skitt)
build: CMake: Enable Only C for lib/ and programs/ Projects (#2498, @concatime)
build: CMake: Use `configure_file()` to Create the `.pc` File (#2462, @lazka)
build: Fix Fuzzer Compiler Detection & Update UBSAN Flags (#2503, @terrelln)
build: Add Guards for `_LARGEFILE_SOURCE` and `_LARGEFILE64_SOURCE` (#2444, @indygreg)
build: Improve `zlibwrapper` Makefile (#2437, @Cyan4973)
contrib: Add `recover_directory` Program (#2473, @terrelln)
doc: Change License Year to 2021 (#2452 & #2465, @terrelln & @senhuang42)
doc: Fix Typos (#2459, @ThomasWaldmann)
v1.4.8 (Dec 18, 2020)
hotfix: wrong alignment of an internal buffer
v1.4.7 (Dec 16, 2020)
perf: stronger --long mode at high compression levels, by @senhuang42
perf: stronger --patch-from at high compression levels, thanks to --long improvements
perf: faster dictionary compression at medium compression levels, by @felixhandte
perf: small speed & memory usage improvements for ZSTD_compress2(), by @terrelln
perf: improved fast compression speeds with Visual Studio, by @animalize
cli : Set nb of threads with environment variable ZSTD_NBTHREADS, by @senhuang42
cli : accept decompressing files with *.zstd suffix
cli : provide a condensed summary by default when processing multiple files
cli : fix : stdin input no longer confused as user prompt
cli : improve accuracy of several error messages
api : new sequence ingestion API, by @senhuang42
api : shared thread pool: control total nb of threads used by multiple compression jobs, by @marxin
api : new ZSTD_getDictID_fromCDict(), by @LuAPi
api : zlibWrapper only uses public API, and is compatible with dynamic library, by @terrelln
api : fix : multithreaded compression has predictable output even in special cases (see #2327) (issue not accessible from cli)
api : fix : dictionary compression correctly respects dictionary compression level (see #2303) (issue not accessible from cli)
build: fix cmake script when using path with spaces, by @terrelln
build: improved compile-time detection of aarch64/neon platforms, by @bsdimp
build: Fix building on AIX 5.1, by @likema
build: compile paramgrill with cmake on Windows, requested by @mirh
doc : clarify repcode updates in format specification, by @felixhandte
v1.4.6
fix : Always return dstSize_tooSmall when that is the case
fix : Fix ZSTD_initCStream_advanced() with static allocation and no dictionary
perf: Improve small block decompression speed by 20%+, by @terrelln
perf: Reduce compression stack usage by 1 KB, by @terrelln
perf: Improve decompression speed by improving ZSTD_wildcopy, by @helloguo (#2252, #2256)
perf: Improve histogram construction, by @cyan4973 (#2253)
cli : Add --output-dir-mirror option, by @xxie24 (#2219)
cli : Warn when (de)compressing multiple files into a single output, by @senhuang42 (#2279)
cli : Improved progress bar and status summary when (de)compressing multiple files, by @senhuang42 (#2283)
cli : Call stat less often, by @felixhandte (#2262)
cli : Allow --patch-from XXX and --filelist XXX in addition to --patch-from=XXX and --filelist=XXX, by @cyan4973 (#2250)
cli : Allow --patch-from to compress stdin with --stream-size, by @bimbashrestha (#2206)
api : Do not install zbuff.h, since it has long been deprecated, by @cyan4973 (#2166).
api : Fix ZSTD_CCtx_setParameter() with ZSTD_c_compressionLevel to make 0 mean default level, by @i-do-cpp (#2291)
api : Rename ZSTDMT_NBTHREADS_MAX to ZSTDMT_NBWORKERS_MAX, by @marxin (#2228).
build: Install pkg-config file with CMake and MinGW, by @tonytheodore (#2183)
build: Install DLL with CMake on Windows, by @BioDataAnalysis (#2221)
build: Fix DLL install location with CMake, by @xantares and @bimbashrestha (#2186)
build: Add ZSTD_NO_UNUSED_FUNCTIONS macro to hide unused functions
build: Add ZSTD_NO_INTRINSICS macro to avoid explicit intrinsics
build: Add STATIC_BMI2 macro for compile time detection of BMI2 on MSVC, by @Niadb (#2258)
build: Fix -Wcomma warnings, by @cwoffenden
build: Remove distutils requirement for meson build, by @neheb (#2197)
build: Fix cli compilation with uclibc
build: Fix cli compilation without st_mtime, by @ffontaine (#2246)
build: Fix shadowing warnings in library
build: Fix single file library compilation with Enscripten, by @yoshihitoh (#2227)
misc: Improve single file library and include dictBuilder, by @cwoffenden
misc: Allow compression dictionaries with missing symbols
misc: Add freestanding translation script in contrib/freestanding_lib
misc: Collect all of zstd's libc dependencies into zstd_deps.h
doc : Add ZSTD_versionString() to manual, by @animalize
doc : Fix documentation for ZSTD_CCtxParams_setParameter(), by @felixhandte (#2270)
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
