Commit Graph

18393 Commits

Author SHA1 Message Date
Stefan Schantl
44d41fd692 ids.cgi: Add oinkmaster_web () function.
This function is used to regenerate the entire ruleset similar to the
one from ids-functions, but is enhanced to print additional status
messages.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-04-10 11:23:49 +02:00
Stefan Schantl
1aaa347774 ids.cgi: Allow to split working_notice function into two parts.
This allows to open the notice and close it at a later time.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-04-10 11:19:41 +02:00
Stefan Schantl
25652a75d4 ids.cgi: Keep IDS/IPS mode settings when enabling/disabling a provider
or autoupdate for it.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-04-10 11:17:05 +02:00
Stefan Schantl
30c4a9ff35 ids.cgi: Adjust code to use new used-rulesfiles backend.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-04-09 15:00:21 +02:00
Stefan Schantl
8d6714edc8 ids-functions.pl: Change backend to use one file to load the used
rulefiles.

Suricata seems to struggle when using multiple and/or nested includes in
the same config section. This results in a only partially loaded
confguration where not all rulefiles are loaded and used.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-04-09 14:10:13 +02:00
Stefan Schantl
fa7663a1b5 ids.cgi: Remove newly added provider if the rules could not be
downloaded.

When adding a new provider and in case the rules file or tarball can not
be downloaded, the provider remains as configured.

To avoid that, the provider needs to be removed again.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 12:26:35 +01:00
Stefan Schantl
8114440752 convert-ids-modification-files: New converter.
This converter is responsible to convert the old oinkmaster modification
files into the new files and format.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 12:12:55 +01:00
Stefan Schantl
432b8ed21e ids.cgi: Drop last fragments from old modify sids backend.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:54:19 +01:00
Stefan Schantl
849fc8ea15 ids-functions.pl: Drop oinkmaster related functions and declarations.
They are not longer needed and safely can be dropped.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:45:17 +01:00
Stefan Schantl
443ad51d1c ids.cgi: Allow to configure IDS/IPS mode individually for each provider.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:37:21 +01:00
Stefan Schantl
4c98be8bd2 ids.cgi: Use new provider modifications backend.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:37:02 +01:00
Stefan Schantl
9f353f8518 ids.cgi: Use new backend to store the ruleset modifications of a
provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:36:44 +01:00
Stefan Schantl
2deba6bf4a ids-functions.pl: Use "enabled/disabled" to mark if a rule should be
altered.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:36:24 +01:00
Stefan Schantl
794469483f ids-functions.pl: Replace call of external oinkmaster.pl to newly
introduced process_ruleset function.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:35:48 +01:00
Stefan Schantl
5a6c7bbe85 ids-functions.pl: Add process_ruleset() function.
This function is going to replace the part which currently the
oinkmaster.pl script does.

It will read in the extracted ruleset, remove duplicates and alter the
rules to alert or drop in case they match. Also rules will be enabled or
disabled if the used requested this.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:34:30 +01:00
Stefan Schantl
518cbdd389 ids-functions.pl: Add get_provider_ruleset_modifications_file().
This function will obosolete the old oinkmaster modifications files.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:34:11 +01:00
Stefan Schantl
e246285af4 ids-functions.pl: Add private function to obtain the sid and rev of a
rule.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:33:47 +01:00
Stefan Schantl
e0eb5bc737 ids-functions.pl: Add get_providers_mode() function.
This function is used to gather the modes of the configured providers
and return them as hash.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:33:28 +01:00
Stefan Schantl
ff780d8b3f update-ids-ruleset: Fix typo in return code.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:27:01 +01:00
Stefan Schantl
74019d3044 update-ids-ruleset: Skip providers which are not enabled.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:23:44 +01:00
Stefan Schantl
9a3f9c2b23 update-ids-ruleset: Log and abort if to less free disk space is
available.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:22:50 +01:00
Stefan Schantl
c9c3eadbbf update-ids-ruleset: Add logging for various events.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:22:08 +01:00
Stefan Schantl
d1f7542659 update-ids-ruleset: Add function to iherit with the syslog daemon.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:19:31 +01:00
Stefan Schantl
65e3aef583 ids-functionsn.pl: Remove logging calls when checking free diskspace.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:17:06 +01:00
Stefan Schantl
52a557a848 ids-functions.pl: Remove logging calls from downloader.
The download script should not directly do the logging stuff.

It simply should download the files for the requested provider and
return an error code on fail.

The logging should be done at another place.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:14:40 +01:00
Stefan Schantl
e26edcc1c7 ids-functions.pl: Provide better return codes, if the downloader fails.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-25 06:03:40 +01:00
Stefan Schantl
0f2c5211f6 ids-functions.pl: Limit downloader to only one provider.
Remove the option and required code to download the rulesets
for all configured and enabled providers by just calling the downloader
function.

This cause a lot of troubles and if required, directly should be handled
by the processing script.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 21:23:31 +01:00
Stefan Schantl
235e3e92a3 ids-functions.pl: Add get_subscription_code() function.
This function can be used to obtain the subscription code of a given
configured provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 21:17:08 +01:00
Stefan Schantl
061391e776 ids-functions.pl: Use If-Modified-Since header to reduce file downloads.
When using the "If-Modified-Since" header, the server can be requested
if a modified version of the file can be served.

In case that is true, the file will be sent and stored by the downloader
function. If the file has not been touched since the last time, the
server will respond with the code "304" (Not modified).

This tells us, that the current stored file is the latest one (still up-to-date)
and we safely can skip the download attempt for this provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 20:51:56 +01:00
Stefan Schantl
f264adda35 ids-functions.pl: Re-order download request handler creation.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 20:29:21 +01:00
Stefan Schantl
7d8956083b ids-functions.pl: Early load required perl modules.
This will help us to determine if all required perl modules and their
dependencies are avail and load-able.

It also prevents us from doubble loading modules and makes development
and maintainance more easy.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 20:18:58 +01:00
Stefan Schantl
304ce130fd ids-functions.pl: Remove temporary file, if the download failed.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 20:04:30 +01:00
Stefan Schantl
e71804fb82 ids-functions.pl: Allow "3" download attempts for each provider before fail.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 20:04:03 +01:00
Stefan Schantl
de1199e2a3 ids-functions.pl: Drop downloader code for sourcefire based ruleset.
Even if the servers do not support HEAD requests, the remote filesize
(content_length) can be obtained from the connection headers.

This generic method works for all servers and therefore we do not need
the code for handle sourcefire servers in a different way anymore.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 20:01:30 +01:00
Stefan Schantl
abf148cebd rules.pl: Fix creating rules for location based groups.
The former used hash value only contains the country code when
a rule for a single country should be created.

In case a location group is used the hash value refers to the group name,
which does not work here.

The required country code is part of the processed string and can be omitted
from here. This works well for single codes and location groups, because those
are processed in a loop.

Fixes #12809.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Acked-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-23 17:12:37 +00:00
Michael Tremer
77e3829dc1 linux-firmware: Compress firmware on disk
This patch enabled that we can compress any firmware files on disk. This
will save some space since /lib/firmware is becoming larger with every
release.

From formerly 828MiB, this is now using ~349MiB which is a saving of
about 480MiB on disk. This is helping us a lot fighting to contain the
distribution to 2GB on /.

Some other firmware that is installed in other packages is not
compressed with this patch which is a bit sad, but potentially not worth
the effort.

In order to ship this change with a Core Update, it might be intuitive
to remove /lib/firmware first and then extract the new update with all
new files. However, I do not know if this all will compress as well as
before since now the files are already individually compressed. It might
be a challenge to ship this.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-23 16:11:14 +00:00
Peter Müller
247e97800d firewall: Fix placement of HOSTILE chains
They were mistakenly placed after the IPS chains in commit
7b529f5417, but should be placed after the
connection tracking and before the IPS.

Fixes: #12815

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Reviewed-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-23 16:07:36 +00:00
Peter Müller
e68cfdb140 Core Update 166: Ship bind
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-23 09:42:57 +00:00
Matthias Fischer
5019469505 bind: Update to 9.16.27
For details see:
https://downloads.isc.org/isc/bind9/9.16.27/doc/arm/html/notes.html#notes-for-bind-9-16-27

"Security Fixes

    The rules for acceptance of records into the cache have been
    tightened to prevent the possibility of poisoning if forwarders send
    records outside the configured bailiwick. (CVE-2021-25220)

    ISC would like to thank Xiang Li, Baojun Liu, and Chaoyi Lu from
    Network and Information Security Lab, Tsinghua University, and
    Changgen Zou from Qi An Xin Group Corp. for bringing this
    vulnerability to our attention. [GL #2950]

    TCP connections with keep-response-order enabled could leave the TCP
    sockets in the CLOSE_WAIT state when the client did not properly
    shut down the connection. (CVE-2022-0396) [GL #3112]

Feature Changes

    DEBUG(1)-level messages were added when starting and ending the BIND
    9 task-exclusive mode that stops normal DNS operation (e.g. for
    reconfiguration, interface scans, and other events that require
    exclusive access to a shared resource). [GL #3137]

Bug Fixes

    The max-transfer-time-out and max-transfer-idle-out options were not
    implemented when the BIND 9 networking stack was refactored in 9.16.
    The missing functionality has been re-implemented and outgoing zone
    transfers now time out properly when not progressing. [GL #1897]

    TCP connections could hang indefinitely if the other party did not
    read sent data, causing the TCP write buffers to fill. This has been
    fixed by adding a “write” timer. Connections that are hung while
    writing now time out after the tcp-idle-timeout period has elapsed.
    [GL #3132]

    The statistics counter representing the current number of clients
    awaiting recursive resolution results (RecursClients) could
    be miscalculated in certain resolution scenarios, potentially
    causing the value of the counter to drop below zero. This has been
    fixed. [GL #3147]"

Signed-off-by: Matthias Fischer <matthias.fischer@ipfire.org>
Reviewed-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-23 09:42:41 +00:00
Peter Müller
b889a5004a Core Update 166: Ship rrdtool
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-23 09:42:17 +00:00
Matthias Fischer
24005371e1 rrdtool: Update to 1.8.0
For details see:
https://github.com/oetiker/rrdtool-1.x/releases/tag/v1.8.0

"Bugfixes

    python bindings: properly convert double values of rrd info
    failed to expand 'Py_UNUSED', Invalid usage when expanding 'Py_UNUSED'
    document --showtime in xport help output
    fix --use-nan-for-all-missing-data
    update rrdruby.pod
    add missing rrdruby.pod and rrdpython.pod to dist
    Set first_weekday to 0 (Sunday), when HAVE__NL_TIME_WEEK_1STDAY is not defined
    fix median calculation for all NaN inputs
    fix potential leak in xport during failure
    fix many warnings raised by Cppcheck
    fix many compiler warnings from latest gcc
    ensure proper initialization in rrd_daemon
    cleanup testsuite
    better testing
    avoid invalid read in rrd_client
    add symbols from rrdc to librrd
    Fix duplicate write_changes_to_disk() calls when HAVE_LIBRADOS is true and HAVE_MMAP is false
    documentation updates
    for SMIN example in docs
    fix for pyton3 compatibility
    freemem only for valid status <Christian Kr"oger>
    fix double meaning of time 0 as uninitialized value
    fix for zfs not supporting fallocate. this makes resize work on zfs
    add rrdrados.pod to dist
    fetch - do not call rrd_freemem on uninitialized pointers
    use separate pango fontmap per thread
    switch to python 3
    do not leak filename when opening a broken file
    fix leaks in rrdcached
    avoid segfault when flushing cache
    escape json in legend entries
    fix leak in xport
    make rrdcgi param parsing more robust
    fix race in journal_write"

Signed-off-by: Matthias Fischer <matthias.fischer@ipfire.org>
Reviewed-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-23 09:41:57 +00:00
Peter Müller
afd8dc466b Nmap: Update to 7.92
Please refer to https://nmap.org/changelog#7.92 for the changelog of
this version; it is too long to include it here.

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Reviewed-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-22 17:35:52 +00:00
Peter Müller
b5abd5ccd3 Core Update 166: There is no need for /dev/null in that GPG call
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-22 17:35:11 +00:00
Peter Müller
90f03efea5 Core Update 166: Introduce new 2022 Pakfire key
Cc: Michael Tremer <michael.tremer@ipfire.org>
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-22 17:34:30 +00:00
Peter Müller
fa0a7c038c Core Update 166: Perform spring clean of orphaned files
On an IPFire installation that has been around since Core Update 110
(released April 28, 2017), these files have been identified as being
orphaned, comparing to a fresh installation of Core Update 164.

To avoid such a list of files agglomerating, persons responsible for a
Core Update should check whether an updated package contains deleted
files in its rootfile. If so, they should be deleted on existing systems
via update.sh as well.

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-22 17:33:51 +00:00
Michael Tremer
28f659f75c strongswan: Remove redundant iptables rules
We used to create some iptables rules that permitted traffic to the
firewall from IPsec peers. This however doesn't work due to changes in
iana-etc and it looks like those rules are entirely absolete now.

This patch removes them which should not cause any functional changes.

Fixes: #12808
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Acked-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-22 13:42:03 +00:00
Peter Müller
2188812cb0 credits: Run ./make.sh update-contributors
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-22 07:55:16 +00:00
Peter Müller
5a33fd9bc6 Nano: Braindead me overlooked the rootfiles were misplaced
This was introduced in b1c25d7a84, and
subsequently broke the nightly builds, and I failed to spot this in the
first place. :-/

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-22 07:51:17 +00:00
Peter Müller
7751dcccdd Core Update 166: Ship misc-progs
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-21 21:38:37 +00:00
Michael Tremer
136d91f346 misc-progs: Set a reasonable default PATH variable
We have some scripts in /usr/local/bin which cannot be found by any
misc-progs which is fixed by this patch.

Fixes: #12811
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Reviewed-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-21 21:36:51 +00:00