Historically, the MD5 checksums in our LFS files serve as a protection
against broken downloads, or accidentally corrupted source files.
While the sources are nowadays downloaded via HTTPS, it make sense to
beef up integrity protection for them, since transparently intercepting
TLS is believed to be feasible for more powerful actors, and the state
of the public PKI ecosystem is clearly not helping.
Therefore, this patch switches from MD5 to BLAKE2, updating all LFS
files as well as make.sh to deal with this checksum algorithm. BLAKE2 is
notably faster (and more secure) than SHA2, so the performance penalty
introduced by this patch is negligible, if noticeable at all.
In preparation of this patch, the toolchain files currently used have
been supplied with BLAKE2 checksums as well on
https://source.ipfire.org/.
Cc: Michael Tremer <michael.tremer@ipfire.org>
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Acked-by: Michael Tremer <michael.tremeripfire.org>
- Update from 10.3.2 to 10.4.0
- Update of rootfile
- Changelog
10.4.0: release
* Add --allow-weak-crypto option to suppress warnings about use of
weak cryptographic algorithms. Update documentation around this
issue. Fixes#358.
* Relax xref recovery logic a bit so that files whose objects are
either missing endobj or have endobj at other than the beginning
of a line can still be recovered. Fixes#573.
* Add support for OpenSSL 3. Fixes#568.
The OpenSSL version is detected at compile-time. If you want to
build with OpenSSL 3 on a system that has OpenSSL 1 installed, you
can run configure like this (or similar to this depending on how
you installed openssl3):
pc_openssl_CFLAGS=-I/path/to/openssl3/include \
pc_openssl_LIBS='-L/path/to/openssl3/lib64 -lssl -lcrypto' \
./configure
where /path/to/openssl3 is wherever your OpenSSL 3 distribution is
installed. You may also need to set the LD_LIBRARY_PATH
environment variable if it's not installed in a standard location.
* Add range check in QPDFNumberTreeObjectHelper (fuzz issue 37740).
* Add QIntC::range_check_subtract to do range checking on
subtraction, which has different boundary conditions from
addition.
* Bug fix: fix crash that could occur under certain conditions
when using --pages with files that had form fields. Fixes#548.
* Add an extra check to the library to detect when foreign objects
are inserted directly (instead of using
<function>QPDF::copyForeignObject</function>) at the time of
insertion rather than when the file is written. Catching the error
sooner makes it much easier to locate the incorrect code.
* Bug fix: make overlay/underlay work on a page with no resource
dictionary. Fixes#527.
* Add QPDF::findPage to the public API. This is primarily to help
improve the efficiency of code that wraps the qpdf library, such
as pikepdf. Fixes#516.
* zlib-flate: warn and exit with code 3 when there is corrupted
input data even when decompression is possible. We do this in the
zlib-flate CLI so that it can be more reliably used to test the
validity of zlib streams, but we don't warn by default in qpdf
itself because PDF files in the wild exist with this problem and
other readers appear to tolerate it. There is a PDF in the qpdf
test suite (form-filled-by-acrobat.pdf) that was written by a
version of Adobe Acrobat that exhibits this problem. Fixes#562.
* Add Pl_Flate::setWarnCallback to make it possible to be notified
of data errors that are recoverable but still indicate invalid
data.
* Improve error reporting when someone forgets the -- after
--pages. Fixes#555.
* Bug fix: ensure we don't overflow any string bounds while
handling completion, even when we are given bogus input values.
Fixes#441.
* Improve performance of preservation of object streams by
avoiding unnecessary traversal of objects when there are no object
streams.
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Reviewed-by: Michael Tremer <michael.tremer@ipfire.org>
we have no supported armv5tel board left so we can switch to the higher
arch. This now can use the vpu (still in softfp calling convention to
not break existing installations.)
this fix many compile problems, also boost is now working again.
Signed-off-by: Arne Fitzenreiter <arne_f@ipfire.org>
- Update from 10.3.0 to 10.3.2
- Update rootfiles
- Changelog
* 10.3.2: release
* Fix problem that caused the generated manual from being included
in the Windows distributions. Fixes#521.
* Fix 11-year-old bug of leaving unreferenced objects in preserved
object streams. Fixes#520.
* Portability fix: use tm_gmtoff rather than global timezone
variable if available to get timezone offset. This fixes
compilation on BSD and also results in a daylight saving
time-aware offset for Linux or other GNU systems. Fixes#515.
* When adding a page, if the page already exists, make a shallow
copy of the page instead of throwing an exception. This makes the
behavior of adding a page from the library consistent with what
the CLI does and also with what the library does if it starts with
a file that already has a duplicated page. Note that this means
that, in some cases, the page you pass to addPage or addPageAt
(either in QPDF or QPDFPageDocumentHelper) will not be the same
object that actually gets added. (This has actually always been
the case.) That means that, if you are going to do subsequent
modification on the page, you should retrieve it again.
* 10.3.1: release
* Bug fix: allow /DR to be direct in /AcroForm
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
- Update qpdf from 10.1.0 to 10.3.0
- Updated rootfile
- Changelog is too long to fully include here
See ChangeLog file in source tarball
Bug fixes in 10.3.0
* The last several changes are in support of fixing more complex
cases of keeping form fields working properly through page copying
operations. Fixes#509.
Bug fixes in 10.2.0
* From qpdf CLI, --pages and --split-pages will properly preserve
interactive form functionality. Fixes#340.
* From qpdf CLI, --overlay and --underlay will copy annotations
and form fields from overlay/underlay file. Fixes#395.
* Add new option --password-file=file for reading the decryption
password from a file. file may be "-" to read from standard input.
Fixes#499.
* By default, give an error if a user attempts to encrypt a file
with a 256-bit key, a non-empty user password, and an empty owner
password. Such files are insecure since they can be opened with no
password. To allow explicit creation of files like this, pass the
new --allow-insecure option. Thanks to github user RobK88 for a
detailed analysis and for reporting this issue. Fixes#501.
* Bug fix: if a form XObject lacks a resources dictionary,
consider any names in that form XObject to be referenced from the
containing page. This is compliant with older PDF versions. Also
detect if any form XObjects have any unresolved names and, if so,
don't remove unreferenced resources from them or from the page
that contains them. Fixes#494.
* Give warnings instead of segfaulting if a QPDF operation is
attempted after calling closeInputSource(). Fixes#495.
Signed-off-by: Adolf Belka (ipfire) <adolf.belka@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
- Update qpdf from 10.0.1 to 10.1.0
- Update rootfile
- Changelog
2021-01-05 Jay Berkenbilt <ejb@ql.org>
* 10.1.0: release
2021-01-04 Jay Berkenbilt <ejb@ql.org>
* When qpdf CLI extracts pages, it now only attempts to remove
unreferenced resources from the pages that it is keeping. This
change dramatically reduces the time it takes to extract a small
number of pages from a large, complex file.
* Move getNext()->write() calls in some pipelines to ensure that
state gates properly reset even if the next pipeline's write
throws an exception (fuzz issue 28262).
2021-01-03 Jay Berkenbilt <ejb@ql.org>
* Don't include -o nospace with zsh completion setup so file
completion works normally. Fixes#473.
2021-01-02 Jay Berkenbilt <ejb@ql.org>
* Make QPDFPageObjectHelper methods pipeContents, parseContents,
and addContentTokenFilter work with form XObjects.
* Rename some QPDFPageObjectHelper methods and make them support
form XObjects as well as pages. The old names will be preserved
from compatibility.
- pipePageContents -> pipeContents
- parsePageContents -> parseContents
* Add QPDFObjectHandle::parseAsContents to apply ParserCallbacks
to a form XObject.
* QPDFPageObjectHelper::externalizeInlineImages can be called with
form XObjects as well as pages.
* Bug fix: QPDFPageObjectHelper::externalizeInlineImages was not
descending into form XObjects on a page. It now does this by
default. In the extremely unlikely event that anyone was actually
depending on the old behavior, it is available by passing
shallow=true to the externalizeInlineImages call.
* Bug fix: QPDFObjectHandle::filterPageContents was broken for
pages with an array of content streams. This caused
externalize-inline-images to also be broken for this case.
2021-01-01 Jay Berkenbilt <ejb@ql.org>
* Add methods to QPDFPageObjectHelper: forEachXObject,
forEachImage, forEachFormXObject to call a function on each
XObject (or image or form XObject) in a page or form XObject,
possibly recursing into nested form XObjects.
* Add method QPDFPageObjectHelper::getFormXObjects to return a map
of keys to form XObjects (non-recursively) from a page or form
XObject.
* Add method QPDFObjectHandle::isImage to test whether an object
is an image.
2020-12-31 Jay Berkenbilt <ejb@ql.org>
* QPDFPageObjectHelper::removeUnreferencedResources can now be
called with a QPDFPageObjectHelper created from a form XObject.
The method already recursed into form XObjects.
* Rename some QPDFPageObjectHelper methods and make them support
form XObjects as well as pages. The old names will be preserved
from compatibility.
- getPageImages -> getImages
- filterPageContents -> filterContents
* Add QPDFObjectHandle::isFormXObject to test whether an object is
a form XObject.
2020-12-30 Jay Berkenbilt <ejb@ql.org>
* Add QPDFPageObjectHelper::flattenRotation and --flatten-rotation
option to the qpdf CLI. The flattenRotation method removes any
/Rotate key from a page dictionary and implements the same
rotation by modifying the page's contents such that the various
page boxes are altered and the page renders identically. This can
be used to work around buggy PDF applications that don't properly
handle page rotation. The --flatten-rotation option to the qpdf
CLI calls flattenRotation for every page.
2020-12-26 Jay Berkenbilt <ejb@ql.org>
* Add QPDFObjectHandle::setFilterOnWrite, which can be used to
tell QPDFWriter not to filter a stream on output even if it can.
You can use this to prevent QPDFWriter from touching a stream
(either uncompressing or compressing) that you have optimized or
otherwise ensured looks exactly the way you want it, even if
decode level or stream compression would otherwise cause
QPDFWriter to modify the stream.
* Add ostream << for QPDFObjGen. (Don't ask why it took 7.5 years
for me to decide to do this.)
2020-12-25 Jay Berkenbilt <ejb@ql.org>
* Refactor write code to eliminate an extra full traversal of
objects in the file and to remove assumptions that preclude stream
references from appearing in /DecodeParms of filterable streams.
This results in an approximately 8% performance reduction in write
times.
2020-12-23 Jay Berkenbilt <ejb@ql.org>
* Allow library users to provide their own decoders for stream
filters by deriving classes from QPDFStreamFilter and registering
them using QPDF::registerStreamFilter. Registered stream filters
provide code to validate and interpret /DecodeParms for a specific
/Filter and also to provide a pipeline that will decode. Note that
it is possible to encode to a filter type that is not supported
even without this feature. See examples/pdf-custom-filter.cc for
an example of using custom stream filters.
2020-12-22 Jay Berkenbilt <ejb@ql.org>
* Add QPDFObjectHandle::makeDirect(bool allow_streams) -- if
allow_streams is true, preserve indirect references to streams
rather than throwing an exception. This allows the object to be
made as direct as possible while preserving stream references.
2020-12-20 Jay Berkenbilt <ejb@ql.org>
* Add qpdf_register_progress_reporter method to C API,
corresponding to QPDFWriter::registerProgressReporter. Fixes#487.
2020-11-28 Jay Berkenbilt <ejb@ql.org>
* Add new functions to the C API for manipulating
QPDFObjectHandles. The new functions allow creation and
modification of objects, which brings a lot of additional power to
the C API. See include/qpdf/qpdf-c.h for details and
examples/pdf-c-objects.c for a simple example.
2020-11-21 Jay Berkenbilt <ejb@ql.org>
* 10.0.4: release
* Fix QIntC::range_check to handle negative numbers properly (fuzz
issue 26994).
2020-11-11 Jay Berkenbilt <ejb@ql.org>
* Treat a direct page object as a runtime error rather than a
logic error since it is actually possible to create a file that
has this (fuzz issue 27393).
2020-11-09 Jay Berkenbilt <ejb@ql.org>
* Handle "." appearing in --pages not preceded by a numeric range
as a special case in command-line parsing code.
2020-11-04 Jay Berkenbilt <ejb@ql.org>
* Ignore the value of the offset/generation field in an xref entry
for a deleted object. Also attempt file recovery on lower-level
exceptions thrown while reading the xref table. Fixes#482.
2020-10-31 Jay Berkenbilt <ejb@ql.org>
* 10.0.3: release
* Don't enter extension initialization in QPDFWriter on a direct
object. Fixes stack overflow in pathological case of /Root being a
direct object (fuzz issue 26761).
* My previous fix to #449 (handling foreign streams with indirect
objects in /Filter and/or /DecodeParms) was incorrect and caused
other problems. There is a now a correct fix to the original
problem. Fixes#478.
2020-10-27 Jay Berkenbilt <ejb@ql.org>
* 10.0.2: release
2020-10-25 Jay Berkenbilt <ejb@ql.org>
* When signing distribution files, generate sha256 checksums
instead of md5, sha1, and sha512. sha256 seems to be more widely
used, and there's no reason to use md5 or sha1 anymore.
* Official Windows releases are now built using the openssl crypto
provider. The native provider is still available for selection at
runtime using the QPDF_CRYPTO_PROVIDER environment variable.
* Bug fix: --no-warn was not suppressing some warnings that might
be generated by --split-pages.
2020-10-23 Jay Berkenbilt <ejb@ql.org>
* Bug fix: when concatenating content streams, insert a newline if
needed to prevent the last token from the old stream from being
merged with the first token of the new stream. Qpdf was mistakenly
concatenating the streams without regard to the specification that
content streams are to be broken on token boundaries. Fixes#444.
* fix-qdf: handle empty streams better with ignore newline by
treating them as empty even though, technically, a blank line
would be required inside the Stream. This just makes it easier to
add place-holder empty streams while editing qdf files by hand.
2020-10-22 Jay Berkenbilt <ejb@ql.org>
* Fix memory leak that could occur if objects in object streams
were resolved more than once and the objects within the object
streams contained circular references. This leak could be
triggered when qpdf was run with --object-streams=generate on
files that already had object streams containing circular
references (fuzz issue 23642).
* Add QIntC::range_check for checking to see whether adding two
numbers together will cause an overflow.
* Fix loop detection problem when traversing page thumbnails
during optimization (fuzz issue 23172).
2020-10-21 Jay Berkenbilt <ejb@ql.org>
* Bug fix: properly handle copying foreign streams that have
indirect /Filter or /DecodeParms keys when stream data has been
replaced. The circumstances leading to this bug are very unusual
but would cause qpdf to either generate an internal error or some
other kind of warning situation if it would occur. Fixes#449.
* Qpdf's build and CI has been migrated from Azure Pipelines
(Azure DevOps) to GitHub Actions.
* Remove some fuzz files that triggered Mal/PDFEx-H with some
virus scanners. There's plenty of coverage in the fuzz corpus
without these files, and it's a nuisance to have virus checkers
remove them. Fixes#460.
* Ensure that numeric conversion is not affected by the user's
global locale setting. Fixes#459.
* Add qpdf-<version>-linux-x86_64.zip to the list of built
distributions. This is a simple zip file that contains just the
qpdf executables and the dependent shared libraries that would not
ordinarily be present on a base system. This minimal binary
distribution works as is when used as a Lambda layer in AWS and
could be suitable for inclusion in a docker image or other
standalone Linux/x86_64 environment where you want minimal support
for running the qpdf executable. Fixes#352.
2020-10-20 Jay Berkenbilt <ejb@ql.org>
* Add --warning-exit-0 option to the qpdf command line. When
specified, qpdf will exit with a status of 0 rather than 3 when
there are warnings without errors. Combine with --no-warn to
completely ignore warnings.
* Bug fix: fix further cases in which errors were written to
stdout. Fixes#438.
* Build option: add --disable-rpath option to ./configure, which
disables passing -rpath to the linker when building shared
libraries with libtool. Fixes#422.
2020-10-16 Jay Berkenbilt <ejb@ql.org>
* Accept pull request that improves how the Windows native crypto
provider is obtained.
* Accept pull request that improves performance in processing
files in memory.
* Accept pull requests that improve openssl configuration and
error reporting.
* Build using GitHub Actions. The intention is that this will
replace Azure Pipelines as the official CI for qpdf for the next
release.
2020-10-15 Jay Berkenbilt <ejb@ql.org>
* Make many minor improvements to the build process and code
health, including fixing a lgtm warning and compiler warnings from
newer version of gcc and MSVC toolchains. Add several cosmetic
improvements to build output in CI.
* Added LL_FMT to config.h.in. This is populated automatically by
autoconf, but if build with your own build system, you may need to
define it as whatever the format string needed by printf for long
long is. Usually this is "%lld", but it can be "%I64d" for some
older Windows-based compilers.
2020-04-29 Jay Berkenbilt <ejb@ql.org>
* Bug fix: qpdf --check was writing errors and warnings reported
by checkLinearization to stdout instead of stderr. Fixes#438.
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Most of these files still used old dates and/or domain names for contact
mail addresses. This is now replaced by an up-to-date copyright line.
Just some housekeeping... :-)
Signed-off-by: Peter Müller <peter.mueller@link38.eu>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
