Commit Graph

19064 Commits

Author SHA1 Message Date
Michael Tremer
713d681232 misc-progs: Set a reasonable default PATH variable
We have some scripts in /usr/local/bin which cannot be found by any
misc-progs which is fixed by this patch.

Fixes: #12811
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Reviewed-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-30 13:43:41 +00:00
Michael Tremer
b7fc708ffc core166: Ship backup script
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-30 13:43:25 +00:00
Michael Tremer
b275771fdd backup: Only list files that exist
This will prevent tar from throwing any errors later on

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-30 13:43:25 +00:00
Michael Tremer
4f0e7f24f2 backup: Ignore any empty lines in backup include list
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-30 13:43:25 +00:00
Michael Tremer
13f6473a4f backup: Fix broken globbing expansion
This patch fixes globbing expansion in the backup include file list
which got broken in c7e0d73e7c.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Tested-by: Adolf Belka <adolf.belka@ipfire.org>
Tested-by: Bernhard Bitsch <bernhard.bitsch@ipfire.org>
2022-03-30 13:43:25 +00:00
Michael Tremer
b7771d5861 core166: Start a new Core Update
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-30 13:43:19 +00:00
Stefan Schantl
fa7663a1b5 ids.cgi: Remove newly added provider if the rules could not be
downloaded.

When adding a new provider and in case the rules file or tarball can not
be downloaded, the provider remains as configured.

To avoid that, the provider needs to be removed again.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 12:26:35 +01:00
Stefan Schantl
8114440752 convert-ids-modification-files: New converter.
This converter is responsible to convert the old oinkmaster modification
files into the new files and format.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 12:12:55 +01:00
Stefan Schantl
432b8ed21e ids.cgi: Drop last fragments from old modify sids backend.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:54:19 +01:00
Stefan Schantl
849fc8ea15 ids-functions.pl: Drop oinkmaster related functions and declarations.
They are not longer needed and safely can be dropped.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:45:17 +01:00
Stefan Schantl
443ad51d1c ids.cgi: Allow to configure IDS/IPS mode individually for each provider.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:37:21 +01:00
Stefan Schantl
4c98be8bd2 ids.cgi: Use new provider modifications backend.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:37:02 +01:00
Stefan Schantl
9f353f8518 ids.cgi: Use new backend to store the ruleset modifications of a
provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:36:44 +01:00
Stefan Schantl
2deba6bf4a ids-functions.pl: Use "enabled/disabled" to mark if a rule should be
altered.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:36:24 +01:00
Stefan Schantl
794469483f ids-functions.pl: Replace call of external oinkmaster.pl to newly
introduced process_ruleset function.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:35:48 +01:00
Stefan Schantl
5a6c7bbe85 ids-functions.pl: Add process_ruleset() function.
This function is going to replace the part which currently the
oinkmaster.pl script does.

It will read in the extracted ruleset, remove duplicates and alter the
rules to alert or drop in case they match. Also rules will be enabled or
disabled if the used requested this.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:34:30 +01:00
Stefan Schantl
518cbdd389 ids-functions.pl: Add get_provider_ruleset_modifications_file().
This function will obosolete the old oinkmaster modifications files.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:34:11 +01:00
Stefan Schantl
e246285af4 ids-functions.pl: Add private function to obtain the sid and rev of a
rule.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:33:47 +01:00
Stefan Schantl
e0eb5bc737 ids-functions.pl: Add get_providers_mode() function.
This function is used to gather the modes of the configured providers
and return them as hash.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:33:28 +01:00
Stefan Schantl
ff780d8b3f update-ids-ruleset: Fix typo in return code.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:27:01 +01:00
Stefan Schantl
74019d3044 update-ids-ruleset: Skip providers which are not enabled.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:23:44 +01:00
Stefan Schantl
9a3f9c2b23 update-ids-ruleset: Log and abort if to less free disk space is
available.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:22:50 +01:00
Stefan Schantl
c9c3eadbbf update-ids-ruleset: Add logging for various events.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:22:08 +01:00
Stefan Schantl
d1f7542659 update-ids-ruleset: Add function to iherit with the syslog daemon.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:19:31 +01:00
Stefan Schantl
65e3aef583 ids-functionsn.pl: Remove logging calls when checking free diskspace.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:17:06 +01:00
Stefan Schantl
52a557a848 ids-functions.pl: Remove logging calls from downloader.
The download script should not directly do the logging stuff.

It simply should download the files for the requested provider and
return an error code on fail.

The logging should be done at another place.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:14:40 +01:00
Stefan Schantl
e26edcc1c7 ids-functions.pl: Provide better return codes, if the downloader fails.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-25 06:03:40 +01:00
Stefan Schantl
0f2c5211f6 ids-functions.pl: Limit downloader to only one provider.
Remove the option and required code to download the rulesets
for all configured and enabled providers by just calling the downloader
function.

This cause a lot of troubles and if required, directly should be handled
by the processing script.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 21:23:31 +01:00
Stefan Schantl
235e3e92a3 ids-functions.pl: Add get_subscription_code() function.
This function can be used to obtain the subscription code of a given
configured provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 21:17:08 +01:00
Stefan Schantl
061391e776 ids-functions.pl: Use If-Modified-Since header to reduce file downloads.
When using the "If-Modified-Since" header, the server can be requested
if a modified version of the file can be served.

In case that is true, the file will be sent and stored by the downloader
function. If the file has not been touched since the last time, the
server will respond with the code "304" (Not modified).

This tells us, that the current stored file is the latest one (still up-to-date)
and we safely can skip the download attempt for this provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 20:51:56 +01:00
Stefan Schantl
f264adda35 ids-functions.pl: Re-order download request handler creation.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 20:29:21 +01:00
Stefan Schantl
7d8956083b ids-functions.pl: Early load required perl modules.
This will help us to determine if all required perl modules and their
dependencies are avail and load-able.

It also prevents us from doubble loading modules and makes development
and maintainance more easy.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 20:18:58 +01:00
Peter Müller
3110d7a98f module-init-tools: Delete orphaned LFS file
This was dropped 2013 in commit ba109afd0d,
but the LFS file remained there.

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-24 19:06:47 +00:00
Peter Müller
4eb92c5ab1 Core Update 166: Delete leftovers from LFS rootfile updates
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-24 19:05:35 +00:00
Stefan Schantl
304ce130fd ids-functions.pl: Remove temporary file, if the download failed.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 20:04:30 +01:00
Stefan Schantl
e71804fb82 ids-functions.pl: Allow "3" download attempts for each provider before fail.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 20:04:03 +01:00
Stefan Schantl
de1199e2a3 ids-functions.pl: Drop downloader code for sourcefire based ruleset.
Even if the servers do not support HEAD requests, the remote filesize
(content_length) can be obtained from the connection headers.

This generic method works for all servers and therefore we do not need
the code for handle sourcefire servers in a different way anymore.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 20:01:30 +01:00
Peter Müller
729b27da0d Core Update 166: Ship zlib
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-24 18:59:48 +00:00
Peter Müller
37afa45661 zlib: Pick up upstream patch for memory corruption fix
See: https://www.openwall.com/lists/oss-security/2022/03/24/1

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Reviewed-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-24 18:59:36 +00:00
Stefan Schantl
c55f5c8eaa rules.pl: Fix creating rules for location based groups.
The former used hash value only contains the country code when
a rule for a single country should be created.

In case a location group is used the hash value refers to the group name,
which does not work here.

The required country code is part of the processed string and can be omitted
from here. This works well for single codes and location groups, because those
are processed in a loop.

Fixes #12809.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Acked-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-23 17:15:18 +00:00
Stefan Schantl
abf148cebd rules.pl: Fix creating rules for location based groups.
The former used hash value only contains the country code when
a rule for a single country should be created.

In case a location group is used the hash value refers to the group name,
which does not work here.

The required country code is part of the processed string and can be omitted
from here. This works well for single codes and location groups, because those
are processed in a loop.

Fixes #12809.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Acked-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-23 17:12:37 +00:00
Peter Müller
26926c4d12 firewall: Fix placement of HOSTILE chains
They were mistakenly placed after the IPS chains in commit
7b529f5417, but should be placed after the
connection tracking and before the IPS.

Fixes: #12815

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Reviewed-by: Michael Tremer <michael.tremer@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-23 16:12:16 +00:00
Michael Tremer
77e3829dc1 linux-firmware: Compress firmware on disk
This patch enabled that we can compress any firmware files on disk. This
will save some space since /lib/firmware is becoming larger with every
release.

From formerly 828MiB, this is now using ~349MiB which is a saving of
about 480MiB on disk. This is helping us a lot fighting to contain the
distribution to 2GB on /.

Some other firmware that is installed in other packages is not
compressed with this patch which is a bit sad, but potentially not worth
the effort.

In order to ship this change with a Core Update, it might be intuitive
to remove /lib/firmware first and then extract the new update with all
new files. However, I do not know if this all will compress as well as
before since now the files are already individually compressed. It might
be a challenge to ship this.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-23 16:11:14 +00:00
Peter Müller
247e97800d firewall: Fix placement of HOSTILE chains
They were mistakenly placed after the IPS chains in commit
7b529f5417, but should be placed after the
connection tracking and before the IPS.

Fixes: #12815

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Reviewed-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-23 16:07:36 +00:00
Peter Müller
e68cfdb140 Core Update 166: Ship bind
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-23 09:42:57 +00:00
Matthias Fischer
5019469505 bind: Update to 9.16.27
For details see:
https://downloads.isc.org/isc/bind9/9.16.27/doc/arm/html/notes.html#notes-for-bind-9-16-27

"Security Fixes

    The rules for acceptance of records into the cache have been
    tightened to prevent the possibility of poisoning if forwarders send
    records outside the configured bailiwick. (CVE-2021-25220)

    ISC would like to thank Xiang Li, Baojun Liu, and Chaoyi Lu from
    Network and Information Security Lab, Tsinghua University, and
    Changgen Zou from Qi An Xin Group Corp. for bringing this
    vulnerability to our attention. [GL #2950]

    TCP connections with keep-response-order enabled could leave the TCP
    sockets in the CLOSE_WAIT state when the client did not properly
    shut down the connection. (CVE-2022-0396) [GL #3112]

Feature Changes

    DEBUG(1)-level messages were added when starting and ending the BIND
    9 task-exclusive mode that stops normal DNS operation (e.g. for
    reconfiguration, interface scans, and other events that require
    exclusive access to a shared resource). [GL #3137]

Bug Fixes

    The max-transfer-time-out and max-transfer-idle-out options were not
    implemented when the BIND 9 networking stack was refactored in 9.16.
    The missing functionality has been re-implemented and outgoing zone
    transfers now time out properly when not progressing. [GL #1897]

    TCP connections could hang indefinitely if the other party did not
    read sent data, causing the TCP write buffers to fill. This has been
    fixed by adding a “write” timer. Connections that are hung while
    writing now time out after the tcp-idle-timeout period has elapsed.
    [GL #3132]

    The statistics counter representing the current number of clients
    awaiting recursive resolution results (RecursClients) could
    be miscalculated in certain resolution scenarios, potentially
    causing the value of the counter to drop below zero. This has been
    fixed. [GL #3147]"

Signed-off-by: Matthias Fischer <matthias.fischer@ipfire.org>
Reviewed-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-23 09:42:41 +00:00
Peter Müller
b889a5004a Core Update 166: Ship rrdtool
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-23 09:42:17 +00:00
Matthias Fischer
24005371e1 rrdtool: Update to 1.8.0
For details see:
https://github.com/oetiker/rrdtool-1.x/releases/tag/v1.8.0

"Bugfixes

    python bindings: properly convert double values of rrd info
    failed to expand 'Py_UNUSED', Invalid usage when expanding 'Py_UNUSED'
    document --showtime in xport help output
    fix --use-nan-for-all-missing-data
    update rrdruby.pod
    add missing rrdruby.pod and rrdpython.pod to dist
    Set first_weekday to 0 (Sunday), when HAVE__NL_TIME_WEEK_1STDAY is not defined
    fix median calculation for all NaN inputs
    fix potential leak in xport during failure
    fix many warnings raised by Cppcheck
    fix many compiler warnings from latest gcc
    ensure proper initialization in rrd_daemon
    cleanup testsuite
    better testing
    avoid invalid read in rrd_client
    add symbols from rrdc to librrd
    Fix duplicate write_changes_to_disk() calls when HAVE_LIBRADOS is true and HAVE_MMAP is false
    documentation updates
    for SMIN example in docs
    fix for pyton3 compatibility
    freemem only for valid status <Christian Kr"oger>
    fix double meaning of time 0 as uninitialized value
    fix for zfs not supporting fallocate. this makes resize work on zfs
    add rrdrados.pod to dist
    fetch - do not call rrd_freemem on uninitialized pointers
    use separate pango fontmap per thread
    switch to python 3
    do not leak filename when opening a broken file
    fix leaks in rrdcached
    avoid segfault when flushing cache
    escape json in legend entries
    fix leak in xport
    make rrdcgi param parsing more robust
    fix race in journal_write"

Signed-off-by: Matthias Fischer <matthias.fischer@ipfire.org>
Reviewed-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-23 09:41:57 +00:00
Peter Müller
afd8dc466b Nmap: Update to 7.92
Please refer to https://nmap.org/changelog#7.92 for the changelog of
this version; it is too long to include it here.

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Reviewed-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-22 17:35:52 +00:00
Peter Müller
b5abd5ccd3 Core Update 166: There is no need for /dev/null in that GPG call
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-22 17:35:11 +00:00