- The addition of pyfuse3 requires a total of 11 python3 module dependencies and the
addition of python3-Cython during the build
- The other dependencies etc are submitted in the rest of this patch series.
Fixes: Bug#12611
Tested-by: Adolf Belka <adolf.belka@ipfire.org>
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
- When borgbackup was upgraded from version 1.1.17 to 1.2.0 the build was sucessfully
completed but there was no testing feedback till after full release. It turned out
that it did not successfully run.
- python3-packaging which had been installed for the build of borgbackup needed to also
be available for the execution.
- When borgbackup was upgraded to 1.2.0 it was noticed that the old python3-msgpack was
no longer needed as borgbackup used its own bundled msgpack since around version 1.1.10
What was not seen was that in version 1.1.19 or 1.1.18 the bundled version of msgpack
had been removed and that the newer version of python3-msgpack now needed to be
installed but the version number has to meet the borgbackup requirements which currently
require it to be =<1.0.3
- This patch adds the python3-packaging and python3-msgpack modules as dependencies for
borgbackup
- The egg-info files are uncommented in the rootfile so that the borgbackup metadata can
be found by python.
- The updated borgbackup build together with the python3-packaging and python3-msgpack
modules were installed into a vm system using the .ipfire packages.
Successfully initialised a borgbackup repo and ran two backups to the repo and checked
the stats for the backup. Everything ran fine.
Fixes: Bug #12884
Tested-by: Adolf Belka <adolf.belka@ipfire.org>
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
- Update from 1.1.17 to 1.2.0
- Update of rootfile
- v2 version has x86_64 replaced by xxxMACHINExxx in the rootfile
- borgbackup now requires the python module pkgconfig, installed as a set with this patch
- Changelog
Compatibility notes:
dropped support / testing for older Pythons, minimum requirement is 3.8. In
case your OS does not provide Python >= 3.8, consider using our binary,
which does not need an external Python interpreter. Or continue using
borg 1.1.x, which is still supported.
freeing repository space only happens when “borg compact” is invoked.
mount: the default for --numeric-ids is False now (same as borg extract)
borg create --noatime is deprecated. Not storing atime is the default
behaviour now (use --atime if you want to store the atime).
list: corrected mix-up of “isomtime” and “mtime” formats. Previously,
“isomtime” was the default but produced a verbose human format, while
“mtime” produced a ISO-8601-like format. The behaviours have been swapped
(so “mtime” is human, “isomtime” is ISO-like), and the default is now
“mtime”. “isomtime” is now a real ISO-8601 format (“T” between date and
time, not a space).
create/recreate --list: file status for all files used to get announced
AFTER the file (with borg < 1.2). Now, file status is announced BEFORE the
file contents are processed. If the file status changes later (e.g. due to
an error or a content change), the updated/final file status will be
printed again.
removed deprecated-since-long stuff (deprecated since):
command “borg change-passphrase” (2017-02), use “borg key …”
option “--keep-tag-files” (2017-01), use “--keep-exclude-tags”
option “--list-format” (2017-10), use “--format”
option “--ignore-inode” (2017-09), use “--files-cache” w/o “inode”
option “--no-files-cache” (2017-09), use “--files-cache=disabled”
removed BORG_HOSTNAME_IS_UNIQUE env var. to use borg you must implement one
of these 2 scenarios:
the combination of FQDN and result of uuid.getnode() must be unique
and stable (this should be the case for almost everybody, except
when having duplicate FQDN and MAC address or all-zero MAC address)
if you are aware that 1) is not the case for you, you must set
BORG_HOST_ID env var to something unique.
exit with 128 + signal number, #5161. if you have scripts expecting rc == 2
for a signal exit, you need to update them to check for >= 128.
Fixes:
diff: reduce memory consumption, fix is_hardlink_master, #6295
compact: fix / improve freeable / freed space log output
derive really freed space from quota use before/after, #5679
do not say “freeable”, but “maybe freeable” (based on hint, unsure)
fix race conditions in internal SaveFile function, #6306#6028
implement internal safe_unlink (was: truncate_and_unlink) function more
safely: usually it does not truncate any more, only under “disk full”
circumstances and only if there is only one hardlink. see:
https://github.com/borgbackup/borg/discussions/6286
Other changes:
info: use a pre12-meta cache to accelerate stats for borg < 1.2 archives.
the first time borg info is invoked on a borg 1.1 repo, it can take a
rather long time computing and caching some stats values for 1.1 archives,
which borg 1.2 archives have in their archive metadata structure. be
patient, esp. if you have lots of old archives. following invocations are
much faster due to the cache. related change: add archive name to
calc_stats progress display.
docs:
add borg 1.2 upgrade notes, #6217
link to borg placeholders and borg patterns help
init: explain the encryption modes better
clarify usage of patternfile roots
put import-tar docs into same file as export-tar docs
explain the difference between a path that ends with or without a slash,
#6297
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Historically, the MD5 checksums in our LFS files serve as a protection
against broken downloads, or accidentally corrupted source files.
While the sources are nowadays downloaded via HTTPS, it make sense to
beef up integrity protection for them, since transparently intercepting
TLS is believed to be feasible for more powerful actors, and the state
of the public PKI ecosystem is clearly not helping.
Therefore, this patch switches from MD5 to BLAKE2, updating all LFS
files as well as make.sh to deal with this checksum algorithm. BLAKE2 is
notably faster (and more secure) than SHA2, so the performance penalty
introduced by this patch is negligible, if noticeable at all.
In preparation of this patch, the toolchain files currently used have
been supplied with BLAKE2 checksums as well on
https://source.ipfire.org/.
Cc: Michael Tremer <michael.tremer@ipfire.org>
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Acked-by: Michael Tremer <michael.tremeripfire.org>
* Add a Summary and Services field to all pak lfs files
* Replace occurances of INSTALL_INITSCRIPT with new INSTALL_INITSCRIPTS
macro in all pak lfs files.
Signed-off-by: Robin Roevens <robin.roevens@disroot.org>
Reviewed-by: Peter Müller <peter.mueller@ipfire.org>
- Update from 1.1.13 to 1.1.17
- Update of rootfile
- Changelog
Version 1.1.17 (2021-07-12)
- 1.1.17 install_requires the "packaging" pypi package now.
Fixes:
- pyinstaller dir-mode: fix pyi detection / LIBPATH treatment, #5897
- handle crash due to kill stale lock race, #5828
- fix BORG_CACHE_DIR crashing borg if empty, #5216
- create --dry-run: fix display of kept tagfile, #5834
- fix missing parameter in "did not consistently fail" msg, #5822
- missing / healed chunks: always tell chunk ID, #5704
- benchmark: make sure cleanup happens even on exceptions, #5630
New features:
- implement BORG_SELFTEST env variable, #5871.
this can be used to accelerate borg startup a bit. not recommended for
normal usage, but borg mass hosters with a lot of borg invocations can
save some resources with this. on my laptop, this saved ~100ms cpu time
(sys+user) per borg command invocation.
- implement BORG_LIBC env variable to give the libc filename, #5870.
you can use this if a borg does not find your libc.
- check: add progress indicator for archive check.
- allow --files-cache=size (not recommended, make sure you know what you do)
Other changes:
- Python 3.10 now officially supported!
we test on py310-dev on github CI since a while and now also on the vagrant
machines, so it should work ok.
- github CI: test on py310 (again)
- get rid of distutils, use packaging and setuptools.
distutils is deprecated and gives warnings on py 3.10.
- setup.py: rename "clean" to "clean2" to avoid shadowing the "clean" command.
- remove libc filename fallback for the BSDs (there is no "usual" name)
- cleanup flake8 checks, fix some pep8 violations.
- docs building: replace deprecated function ".add_stylesheet()" for Sphinx 4 compatibility
- docs:
- add a hint on sleeping computer and ssh connections, #5301
- update the documentation on hacked backup client, #5480
- improve docs/FAQ about append-only remote repos, #5497
- complement the documentation for pattern files and exclude files, #5520
- "filename with spaces" example added to exclude file, #5236
note: no whitespace escaping needed, processed by borg.
- add info on renaming repositories, #5240
- clarify borg check --verify-data, #5808
- add notice about defective hardware to check documentation, #5753
- add paragraph added in #5855 to utility documentation source
- add missing leading slashes in help patterns, #5857
- clarify "you will need key and passphrase" borg init warning, #4622
- pull mode: add some warnings, #5827
- mention tar --compare (compare archive to fs files), #5880
- fix typos, backport of #5597
- vagrant:
- add py3.7.11 for binary build, also add 3.10-dev.
- use latest Cython 0.29.23 for py310 compat fixes.
- more RAM for openindiana upgrade plan resolver, it just hangs (swaps?) if
there is too little RAM.
- fix install_pyenv to adapt to recent changes in pyenv (same as in master now).
- use generic/netbsd9 box, copied from master branch.
Version 1.1.16 (2021-03-23)
Fixes:
- setup.py: add special openssl prefix for Apple M1 compatibility
- do not recurse into duplicate roots, #5603
- remove empty shadowed_segments lists, #5275, #5614
- fix libpython load error when borg fat binary / dir-based binary is invoked
via a symlink by upgrading pyinstaller to v4.2, #5688
- config: accept non-int value (like 500M or 100G) for max_segment_size or
storage_quota, #5639.
please note: when setting a non-int value for this in a repo config,
using the repo will require borg >= 1.1.16.
New features:
- bundled msgpack: drop support for old buffer protocol to support Python 3.10
- verbose files cache logging via --debug-topic=files_cache, #5659.
Use this if you suspect that borg does not detect unmodified files as expected.
- create/extract: add --noxattrs and --noacls option, #3955.
when given with borg create, borg will not get xattrs / ACLs from input files
(and thus, it will not archive xattrs / ACLs). when given with borg extract,
borg will not read xattrs / ACLs from archive and will not set xattrs / ACLs
on extracted files.
- diff: add --json-lines option, #3765
- check: debug log segment filename
- borg debug dump-hints
Other changes:
- Tab completion support for additional archives for 'borg delete'
- repository: deduplicate code of put and delete, no functional change
- tests: fix result order issue (sporadic test failure on openindiana)
- vagrant:
- upgrade pyinstaller to v4.2, #5671
- avoid grub-install asking interactively for device
- remove the xenial box
- update freebsd box to 12.1
- docs:
- update macOS install instructions, #5677
- use macFUSE (not osxfuse) for Apple M1 compatibility
- update docs for dev environment installation instructions, #5643
- fix grammar in faq
- recomend running tests only on installed versions for setup
- add link back to git-installation
- remove /var/cache exclusion in example commands, #5625.
This is generally a poor idea and shouldn't be promoted through examples.
- add repology.org badge with current packaging status
- explain hash collision
- add unsafe workaround to use an old repo copy, #5722
Version 1.1.15 (2020-12-25)
Fixes:
- extract:
- improve exception handling when setting xattrs, #5092.
- emit a warning message giving the path, xattr key and error message.
- continue trying to restore other xattrs and bsdflags of the same file
after an exception with xattr-setting happened.
- export-tar:
- set tar format to GNU_FORMAT explicitly, #5274
- fix memory leak with ssh: remote repository, #5568
- fix potential memory leak with ssh: remote repository with partial extraction
- create: fix --dry-run and --stats coexistence, #5415
- use --timestamp for {utcnow} and {now} if given, #5189
New features:
- create: implement --stdin-mode, --stdin-user and --stdin-group, #5333
- allow appending the files cache filename with BORG_FILES_CACHE_SUFFIX env var
Other changes:
- drop python 3.4 support, minimum requirement is 3.5 now.
- enable using libxxhash instead of bundled xxh64 code
- update llfuse requirements (1.3.8)
- set cython language_level in some files to fix warnings
- allow EIO with warning when trying to hardlink
- PropDict: fail early if internal_dict is not a dict
- update shell completions
- tests / CI
- add a test for the hashindex corruption bug, #5531#4829
- fix spurious failure in test_cache_files, #5438
- added a github ci workflow
- reduce testing on travis, no macOS, no py3x-dev, #5467
- travis: use newer dists, native py on dist
- vagrant:
- remove jessie and trusty boxes, #5348#5383
- pyinstaller 4.0, build on py379
- binary build on stretch64, #5348
- remove easy_install based pip installation
- docs:
- clarify '--one-file-system' for btrfs, #5391
- add example for excluding content using the --pattern cmd line arg
- complement the documentation for pattern files and exclude files, #5524
- made ansible playbook more generic, use package instead of pacman. also
change state from "latest" to "present".
- complete documentation on append-only remote repos, #5497
- internals: rather talk about target size than statistics, #5336
- new compression algorithm policy, #1633#5505
- faq: add a hint on sleeping computer, #5301
- note requirements for full disk access on macOS Catalina, #5303
- fix/improve description of borg upgrade hardlink usage, #5518
- modernize 1.1 code:
- drop code/workarounds only needed to support Python 3.4
- remove workaround for pre-release py37 argparse bug
- removed some outdated comments/docstrings
- requirements: remove some restrictions, lock on current versions
Version 1.1.14 (2020-10-07)
- 1.1.14 changes return codes due to a bug fix:
In case you have scripts expecting rc == 2 for a signal exit, you need to
update them to check for >= 128 (as documented since long).
Fixes:
- check --repair: fix potential data loss when interrupting it, #5325
- exit with 128 + signal number (as documented) when borg is killed by a signal, #5161
- fix hardlinked CACHEDIR.TAG processing, #4911
- create --read-special: .part files also should be regular files, #5217
- llfuse dependency: choose least broken 1.3.6/1.3.7.
1.3.6 is broken on python 3.9, 1.3.7 is broken on FreeBSD.
Other changes:
- upgrade bundled xxhash to 0.7.4
- self test: if it fails, also point to OS and hardware, #5334
- pyinstaller: compute basepath from spec file location
- prettier error message when archive gets too big, #5307
- check/recreate are not "experimental" any more (but still potentially dangerous):
- recreate: remove extra confirmation
- rephrase some warnings, update docs, #5164
- shell completions:
- misc. updates / fixes
- support repositories in fish tab completion, #5256
- complete $BORG_RECREATE_I_KNOW_WHAT_I_AM_DOING
- rewrite zsh completion:
- completion for almost all optional and positional arguments
- completion for Borg environment variables (parameters)
- use "allow/deny list" instead of "white/black list" wording
- declare "allow_cache_wipe" marker in setup.cfg to avoid pytest warning
- vagrant / tests:
- misc. fixes / updates
- use python 3.5.10 for binary build
- build directory-based binaries additionally to the single file binaries
- add libffi-dev, required to build python
- use cryptography<3.0, more recent versions break the jessie box
- test on python 3.9
- do brew update with /dev/null redirect to avoid "too much log output" on travis-ci
- docs:
- add ssh-agent pull backup method docs, #5288
- how to approach borg speed issues, #5371
- mention double --force in prune docs
- update Homebrew install instructions, #5185
- better description of how cache and rebuilds of it work
- point to borg create --list item flags in recreate usage, #5165
- add security faq explaining AES-CTR crypto issues, #5254
- add a note to create from stdin regarding files cache, #5180
- fix borg.1 manpage generation regression, #5211
- clarify how exclude options work in recreate, #5193
- add section for retired contributors
- update Homebrew install instructions, #5185
- better description of how cache and rebuilds of it work
- point to borg create --list item flags in recreate usage, #5165
- add security faq explaining AES-CTR crypto issues, #5254
- add a note to create from stdin regarding files cache, #5180
- fix borg.1 manpage generation regression, #5211
- clarify how exclude options work in recreate, #5193
- add section for retired contributors
- hint about not misusing private email addresses of contributors for borg support
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Reviewed-by: Peter Müller <peter.mueller@ipfire.org>
BorgBackup seems to need this testsuite on all systems, because it does
some selftests when starting a backup.
Fixes: #12438
Signed-off-by: Jonatan Schlag <jonatan.schlag@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Not sure why this has ever been there. This simply makes it
nicer to read and edit because we can have line-breaks now.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Signed-off-by: Arne Fitzenreiter <arne_f@ipfire.org>
