- Update from version 3390200 to 3400000
- Update of rootfile not required
- Changelog
version 3.40.0 On 2022-11-16
Add support for compiling SQLite to WASM and running it in web browsers. NB: The WASM build and its interfaces are considered "beta" and are subject to minor changes if the need arises. We anticipate finalizing the interface for the next release.
Add the recovery extension that might be able to recover some content from a corrupt database file.
Query planner enhancements:
Recognize covering indexes on tables with more than 63 columns where columns beyond the 63rd column are used in the query and/or are referenced by the index.
Extract the values of expressions contained within expression indexes where practical, rather than recomputing the expression.
The NOT NULL and IS NULL operators (and their equivalents) avoid loading the content of large strings and BLOB values from disk.
Avoid materializing a view on which a full scan is performed exactly once. Use and discard the rows of the view as they are computed.
Allow flattening of a subquery that is the right-hand operand of a LEFT JOIN in an aggregate query.
A new typedef named sqlite3_filename is added and used to represent the name of a database file. Various interfaces are modified to use the new typedef instead of "char*". This interface change should be fully backwards compatible, though it might cause (harmless) compiler warnings when rebuilding some legacy applications.
Add the sqlite3_value_encoding() interface.
Security enhancement: SQLITE_DBCONFIG_DEFENSIVE is augmented to prohibit changing the schema_version. The schema_version becomes read-only in defensive mode.
Enhancements to the PRAGMA integrity_check statement:
Columns in non-STRICT tables with TEXT affinity should not contain numeric values.
Columns in non-STRICT tables with NUMERIC affinity should not contain TEXT values that could be converted into numbers.
Verify that the rows of a WITHOUT ROWID table are in the correct order.
Enhance the VACUUM INTO statement so that it honors the PRAGMA synchronous setting.
Enhance the sqlite3_strglob() and sqlite3_strlike() APIs so that they are able to accept NULL pointers for their string parameters and still generate a sensible result.
Provide the new SQLITE_MAX_ALLOCATION_SIZE compile-time option for limiting the size of memory allocations.
Change the algorithm used by SQLite's built-in pseudo-random number generator (PRNG) from RC4 to Chacha20.
Allow two or more indexes to have the same name as long as they are all in separate schemas.
Miscellaneous performance optimizations result in about 1% fewer CPU cycles used on typical workloads.
version 3.39.3 (2022-09-05):
Use a statement journal on DML statement affecting two or more database rows if the statement makes use of a SQL functions that might abort. See forum thread 9b9e4716c0d7bbd1.
Use a mutex to protect the PRAGMA temp_store_directory and PRAGMA data_store_directory statements, even though they are deprecated and documented as not being threadsafe. See forum post 719a11e1314d1c70.
Other bug and warning fixes. See the timeline for details.
version 3.39.4 (2022-09-29):
Fix the build on Windows so that it works with -DSQLITE_OMIT_AUTOINIT
Fix a long-standing problem in the btree balancer that might, in rare cases, cause database corruption if the application uses an application-defined page cache.
Enhance SQLITE_DBCONFIG_DEFENSIVE so that it disallows CREATE TRIGGER statements if one or more of the statements in the body of the trigger write into shadow tables.
Fix a possible integer overflow in the size computation for a memory allocation in FTS3.
Fix a misuse of the sqlite3_set_auxdata() interface in the ICU Extension.
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Reviewed-by: Michael Tremer <michael.tremer@ipfire.org>
- Update from version 3390000 to 3390200
- Update of rootfile not required
- Changelog
version 3.39.2 (2022-07-21):
Fix a performance regression in the query planner associated with rearranging
the order of FROM clause terms in the presences of a LEFT JOIN.
Apply fixes for CVE-2022-35737, Chromium bugs 1343348 and 1345947, forum post
3607259d3c, and other minor problems discovered by internal testing.
version 3.39.1 (2022-07-13):
Fix an incorrect result from a query that uses a view that contains a
compound SELECT in which only one arm contains a RIGHT JOIN and where the
view is not the first FROM clause term of the query that contains the view.
forum post 174afeae5734d42d.
Fix some harmless compiler warnings.
Fix a long-standing problem with ALTER TABLE RENAME that can only arise if
the sqlite3_limit(SQLITE_LIMIT_SQL_LENGTH) is set to a very small value.
Fix a long-standing problem in FTS3 that can only arise when compiled with
the SQLITE_ENABLE_FTS3_PARENTHESIS compile-time option.
Fix the build so that is works when the SQLITE_DEBUG and
SQLITE_OMIT_WINDOWFUNC compile-time options are both provided at the same time.
Fix the initial-prefix optimization for the REGEXP extension so that it works
correctly even if the prefix contains characters that require a 3-byte UTF8
encoding.
Enhance the sqlite_stmt virtual table so that it buffers all of its output.
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Reviewed-by: Peter Müller <peter.mueller@ipfire.org>
- Update from version 3380500 to 3890000
- Update of rootfile not required
- Changelog
Release 3.39.0 On 2022-06-25
Add (long overdue) support for RIGHT and FULL OUTER JOIN.
Add new binary comparison operators IS NOT DISTINCT FROM and IS DISTINCT FROM that are
equivalent to IS and IS NOT, respective, for compatibility with PostgreSQL and SQL
standards.
Add a new return code (value "3") from the sqlite3_vtab_distinct() interface that
indicates a query that has both DISTINCT and ORDER BY clauses.
Added the sqlite3_db_name() interface.
The unix os interface resolves all symbolic links in database filenames to create a
canonical name for the database before the file is opened.
Defer materializing views until the materialization is actually needed, thus avoiding
unnecessary work if the materialization turns out to never be used.
The HAVING clause of a SELECT statement is now allowed on any aggregate query, even
queries that do not have a GROUP BY clause.
Many microoptimizations collectively reduce CPU cycles by about 2.3%.
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Reviewed-by: Peter Müller <peter.mueller@ipfire.org>
- Update from version 3380300 to 3380500
- Update of rootfile not required
- Changelog
Version 3.38.5
The 3.38.4 patch release included a minor change to the CLI source code that did not
work. The release manager only ran a subset of the normal release tests, and hence
did not catch the problem. As a result, the CLI will segfault when using columnar
output modes in version 3.38.4. This blunder did not affect the core SQLite library.
It only affected the CLI.
Take-away lesson: Always run all of your tests prior to a release - even a trival
patch release. Always.
The 3.38.5 patch release fixes the 3.38.4 blunder.
Version 3.38.4
Another user-discovered problem in the new Bloom filter optimization is fixed in this
patch release. Without the fix, it is possible for a multi-way join that uses a
Bloom filters for two or more tables in the join to enter an infinite loop if the
key constraint on one of those tables contains a NULL value.
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Reviewed-by: Peter Müller <peter.mueller@ipfire.org>
- Update from version 3380000 to 3380300
- Update of rootfile not required
- Changelog
3.38.3 (2022-04-27):
Fix a case of the query planner be overly aggressive with optimizing
automatic-index and Bloom-filter construction, using inappropriate ON clause
terms to restrict the size of the automatic-index or Bloom filter, and
resulting in missing rows in the output. Forum thread 0d3200f4f3bcd3a3.
Other minor patches. See the timeline for details.
3.38.2 (2022-03-26):
Fix a user-discovered problem with the new Bloom filter optimization that
might cause an incorrect answer when doing a LEFT JOIN with a WHERE clause
constraint that says that one of the columns on the right table of the LEFT
JOIN is NULL. See forum thread 031e262a89b6a9d2.
Other minor patches. See the timeline for details.
3.38.1 (2022-03-12):
Fix problems with the new Bloom filter optimization that might cause some
obscure queries to get an incorrect answer.
Fix the localtime modifier of the date and time functions so that it
preserves fractional seconds.
Fix the sqlite_offset SQL function so that it works correctly even in corner
cases such as when the argument is a virtual column or the column of a view.
Fix row value IN operator constraints on virtual tables so that they work
correctly even if the virtual table implementation relies on bytecode to
filter rows that do not satisfy the constraint.
Other minor fixes to assert() statements, test cases, and documentation. See
the source code timeline for details.
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Reviewed-by: Peter Müller <peter.mueller@ipfire.org>
Historically, the MD5 checksums in our LFS files serve as a protection
against broken downloads, or accidentally corrupted source files.
While the sources are nowadays downloaded via HTTPS, it make sense to
beef up integrity protection for them, since transparently intercepting
TLS is believed to be feasible for more powerful actors, and the state
of the public PKI ecosystem is clearly not helping.
Therefore, this patch switches from MD5 to BLAKE2, updating all LFS
files as well as make.sh to deal with this checksum algorithm. BLAKE2 is
notably faster (and more secure) than SHA2, so the performance penalty
introduced by this patch is negligible, if noticeable at all.
In preparation of this patch, the toolchain files currently used have
been supplied with BLAKE2 checksums as well on
https://source.ipfire.org/.
Cc: Michael Tremer <michael.tremer@ipfire.org>
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Acked-by: Michael Tremer <michael.tremeripfire.org>
- Update from version 3370200 to 3380000
- Update of rootfile not required
- Changelog
SQLite Release 3.38.0 On 2022-02-22
Added the -> and ->> operators for easier processing of JSON. The new operators are
compatible with MySQL and PostgreSQL.
The JSON functions are now built-ins. It is no longer necessary to use the
-DSQLITE_ENABLE_JSON1 compile-time option to enable JSON support. JSON is on by
default. Disable the JSON interface using the new -DSQLITE_OMIT_JSON compile-time
option.
Enhancements to date and time functions:
Added the unixepoch() function.
Added the auto modifier and the julianday modifier.
Rename the printf() SQL function to format() for better compatibility. The original
printf() name is retained as an alias for backwards compatibility.
Added the sqlite3_error_offset() interface, which can sometimes help to localize an
SQL error to a specific character in the input SQL text, so that applications can
provide better error messages.
Enhanced the interface to virtual tables as follows:
Added the sqlite3_vtab_distinct() interface.
Added the sqlite3_vtab_rhs_value() interface.
Added new operator types SQLITE_INDEX_CONSTRAINT_LIMIT and
SQLITE_INDEX_CONSTRAINT_OFFSET.
Added the sqlite3_vtab_in() interface (and related) to enable a virtual table to
process IN operator constraints all at once, rather than processing each value of
the right-hand side of the IN operator separately.
CLI enhancements:
Columnar output modes are enhanced to correctly handle tabs and newlines embedded
in text.
Added options like "--wrap N", "--wordwrap on", and "--quote" to the columnar
output modes.
Added the .mode qbox alias.
The .import command automatically disambiguates column names.
Use the new sqlite3_error_offset() interface to provide better error messages.
Query planner enhancements:
Use a Bloom filter to speed up large analytic queries.
Use a balanced merge tree to evaluate UNION or UNION ALL compound SELECT
statements that have an ORDER BY clause.
The ALTER TABLE statement is changed to silently ignores entries in the sqlite_schema
table that do not parse when PRAGMA writable_schema=ON.
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Reviewed-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
- Update from 3370000 to 3370200
- Update of rootfile not required
- Changelog
Fix a bug introduced in version 3.35.0 (2021-03-12) that can cause database corruption
if a SAVEPOINT is rolled back while in PRAGMA temp_store=MEMORY mode, and other
changes are made, and then the outer transaction commits. Check-in 73c2b50211d3ae26
Fix a long-standing problem with ON DELETE CASCADE and ON UPDATE CASCADE in which a
cache of the bytecode used to implement the cascading change was not being reset
following a local DDL change. Check-in 5232c9777fe4fb13.
Other minor fixes that should not impact production builds.
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Reviewed-by: Peter Müller <peter.mueller@ipfire.org>
- Update from 3.36.0 to 3.37.0
- Update of rootfile not required
- Changelog
Release 3.37.0 On 2021-11-27
STRICT tables provide a prescriptive style of data type management, for developers
who prefer that kind of thing.
When adding columns that contain a CHECK constraint or a generated column containing
a NOT NULL constraint, the ALTER TABLE ADD COLUMN now checks new constraints against
preexisting rows in the database and will only proceed if no constraints are violated.
Added the PRAGMA table_list statement.
CLI enhancements:
Add the .connection command, allowing the CLI to keep multiple database
connections open at the same time.
Add the --safe command-line option that disables dot-commands and SQL statements
that might cause side-effects that extend beyond the single database file named
on the command-line.
Performance improvements when reading SQL statements that span many lines.
Added the sqlite3_autovacuum_pages() interface.
The sqlite3_deserialize() does not and has never worked for the TEMP database. That
limitation is now noted in the documentation.
The query planner now omits ORDER BY clauses on subqueries and views if removing
those clauses does not change the semantics of the query.
The generate_series table-valued function extension is modified so that the first
parameter ("START") is now required. This is done as a way to demonstrate how to
write table-valued functions with required parameters. The legacy behavior is
available using the -DZERO_ARGUMENT_GENERATE_SERIES compile-time option.
Added new sqlite3_changes64() and sqlite3_total_changes64() interfaces.
Added the SQLITE_OPEN_EXRESCODE flag option to sqlite3_open_v2().
Use less memory to hold the database schema.
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Reviewed-by: Michael Tremer <michael.tremer@ipfire.org>
- Update from 3350500 to 3360000
- Update of rootfile not required
- Changelog
Improvement to the EXPLAIN QUERY PLAN output to make it easier to understand.
Byte-order marks at the start of a token are skipped as if they were whitespace.
An error is raised on any attempt to access the rowid of a VIEW or subquery. Formerly, the rowid of a VIEW would be indeterminate and often would be NULL. The -DSQLITE_ALLOW_ROWID_IN_VIEW compile-time option is available to restore the legacy behavior for applications that need it.
The sqlite3_deserialize() and sqlite3_serialize() interfaces are now enabled by default. The -DSQLITE_ENABLE_DESERIALIZE compile-time option is no longer required. Instead, there is is a new -DSQLITE_OMIT_DESERIALIZE compile-time option to omit those interfaces.
The "memdb" VFS now allows the same in-memory database to be shared among multiple database connections in the same process as long as the database name begins with "/".
Back out the EXISTS-to-IN optimization (item 8b in the SQLite 3.35.0 change log) as it was found to slow down queries more often than speed them up.
Improve the constant-propagation optimization so that it works on non-join queries.
The REGEXP extension is now included in CLI builds.
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Signed-off-by: Arne Fitzenreiter <arne_f@ipfire.org>
- Update from 3.35.4 to 3.35.5
- Update of rootfile not required
- Changelog
Fix defects in the new ALTER TABLE DROP COLUMN feature that could
corrupt the database file.
Fix an obscure query optimizer problem that might cause an incorrect
query result.
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
- Update from 3.34.1 to 3.35.4
- Update of rootfile not required
- Changelog
3.35.0
Added built-in SQL math functions(). (Requires the -DSQLITE_ENABLE_MATH_FUNCTIONS compile-time option.)
Added support for ALTER TABLE DROP COLUMN.
Generalize UPSERT:
Allow multiple ON CONFLICT clauses that are evaluated in order,
The final ON CONFLICT clause may omit the conflict target and yet still use DO UPDATE.
Add support for the RETURNING clause on DELETE, INSERT, and UPDATE statements.
Use less memory when running VACUUM on databases containing very large TEXT or BLOB values. It is no longer necessary to hold the entire TEXT or BLOB in memory all at once.
Add support for the MATERIALIZED and NOT MATERIALIZED hints when specifying common table expressions. The default behavior was formerly NOT MATERIALIZED, but is now changed to MATERIALIZED for CTEs that are used more than once.
The SQLITE_DBCONFIG_ENABLE_TRIGGER and SQLITE_DBCONFIG_ENABLE_VIEW settings are modified so that they only control triggers and views in the main database schema or in attached database schemas and not in the TEMP schema. TEMP triggers and views are always allowed.
Query planner/optimizer improvements:
Enhancements to the min/max optimization so that it works better with the IN operator and the OP_SeekScan optimization of the previous release.
Attempt to process EXISTS operators in the WHERE clause as if they were IN operators, in cases where this is a valid transformation and seems likely to improve performance.
Allow UNION ALL sub-queries to be flattened even if the parent query is a join.
Use an index, if appropriate, on IS NOT NULL expressions in the WHERE clause, even if STAT4 is disabled.
Expressions of the form "x IS NULL" or "x IS NOT NULL" might be converted to simply FALSE or TRUE, if "x" is a column that has a "NOT NULL" constraint and is not involved in an outer join.
Avoid checking foreign key constraints on an UPDATE statement if the UPDATE does not modify any columns associated with the foreign key.
Allow WHERE terms to be pushed down into sub-queries that contain window functions, as long as the WHERE term is made up of entirely of constants and copies of expressions found in the PARTITION BY clauses of all window functions in the sub-query.
CLI enhancements:
Enhance the ".stats" command to accept new arguments "stmt" and "vmstep", causing prepare statement statistics and only the virtual-machine step count to be shown, respectively.
Add the ".filectrl data_version" command.
Enhance the ".once" and ".output" commands so that if the destination argument begins with "|" (indicating that output is redirected into a pipe) then the argument does not need to be quoted.
Bug fixes:
Fix a potential NULL pointer dereference when processing a syntactically incorrect SELECT statement with a correlated WHERE clause and a "HAVING 0" clause. (Also fixed in the 3.34.1 patch release.)
Fix a bug in the IN-operator optimization of version 3.33.0 that can cause an incorrect answer.
Fix incorrect answers from the LIKE operator if the pattern ends with "%" and there is an "ESCAPE '_'" clause.
3.35.1
Fix a bug in the new DROP COLUMN feature when used on columns that are indexed and that are quoted in the index definition.
Improve the built-in documentation for the .dump command in the CLI.
3.35.2
Fix a problem in the appendvfs.c extension that was introduced into version 3.35.0.
Ensure that date/time functions with no arguments (which generate responses that depend on the current time) are treated as non-deterministic functions. Ticket 2c6c8689fb5f3d2f
Fix a problem in the sqldiff utility program having to do with unusual whitespace characters in a virtual table definition.
Limit the new UNION ALL optimization described by item 8c in the 3.35.0 release so that it does not try to make too many new subqueries. See forum thread 140a67d3d2 for details.
3.35.3
Enhance the OP_OpenDup opcode of the bytecode engine so that it works even if the cursor being duplicated itself came from OP_OpenDup. Fix for ticket bb8a9fd4a9b7fce5. This problem only came to light due to the recent MATERIALIZED hint enhancement.
When materializing correlated common table expressions, do so separately for each use case, as that is required for correctness. This fixes a problem that was introduced by the MATERIALIZED hint enhancement.
Fix a problem in the filename normalizer of the unix VFS.
Fix the "box" output mode in the CLI so that it works with statements that returns one or more rows of zero columns (such as PRAGMA incremental_vacuum). Forum post afbbcb5b72.
Improvements to error messages generated by faulty common table expressions. Forum post aa5a0431c99e.
Fix some incorrect assert() statements.
Fix to the SELECT statement syntax diagram so that the FROM clause syntax is shown correctly. Forum post 9ed02582fe.
Fix the EBCDIC character classifier so that it understands newlines as whitespace. Forum post 58540ce22dcd.
Improvements the xBestIndex method in the implementation of the (unsupported) wholenumber virtual table extension so that it does a better job of convincing the query planner to avoid trying to materialize a table with an infinite number of rows. Forum post b52a020ce4.
3.35.4
Fix a defect in the query planner optimization identified by item 8b above. Ticket de7db14784a08053.
Fix a defect in the new RETURNING syntax. Ticket 132994c8b1063bfb.
Fix the new RETURNING feature so that it raises an error if one of the terms in the RETURNING clause references a unknown table, instead of silently ignoring that error.
Fix an assertion associated with aggregate function processing that was incorrectly triggered by the push-down optimization.
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
- Update sqlite from 3.34.0 to 3.34.1
- Update rootfile
- Changelog
Fix a potential use-after-free bug when processing a a subquery with
both a correlated WHERE clause and a "HAVING 0" clause and where the
parent query is an aggregate.
Fix documentation typos
Fix minor problems in extensions.
Signed-off-by: Adolf Belka (ipfire) <adolf.belka@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
-Update sqlite from 3.26.0 to 3.34.0
See https://sqlite[.]org/chronology[.]html for history between
these releases.
-Have reviewed all release notes between these two releases and there
are no deprecations.
-No change to rootfile.
Signed-off-by: Adolf Belka <ahb.ipfire@gmail.com>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Most of these files still used old dates and/or domain names for contact
mail addresses. This is now replaced by an up-to-date copyright line.
Just some housekeeping... :-)
Signed-off-by: Peter Müller <peter.mueller@link38.eu>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
