Commit Graph

9684 Commits

Author SHA1 Message Date
Stefan Schantl
2e558477da convert-ids-backend-files: Convert MONITOR_TRAFFIC_ONLY settings.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-04-16 14:39:09 +02:00
Stefan Schantl
a2c56ead73 ids-functions.pl: Remove read_enabled_disabled_sids_file() function.
Not longer needed and therefore dead code.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-04-15 06:02:49 +02:00
Stefan Schantl
a15c9b16b4 IDS: Move autoupdate logic to cron.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-04-15 05:59:33 +02:00
Stefan Schantl
c2eac6fcd4 convert-ids-backend-files: Move already downloaded files to new
location.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-04-15 05:52:01 +02:00
Stefan Schantl
b570d35c0a ids-functions.pl: Change location for downloaded rulesfiles to
"/var/cache/suricata/".

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-04-15 05:19:20 +02:00
Stefan Schantl
4f513522fe ids-functions.pl: Do not use a hard-code temporary download location.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-04-15 05:18:37 +02:00
Stefan Schantl
c215cfd887 convert-ids-backend-files: Remove old backend related files.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-04-15 05:13:23 +02:00
Stefan Schantl
70b1672d94 convert-ids-backend-files: Remove converted files.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-04-15 05:12:56 +02:00
Stefan Schantl
9f7702544a convert-ids-backend-files: Regenerate ruleset and used rulesets file.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-04-15 05:10:45 +02:00
Stefan Schantl
c00609ce56 convert-ids-backend-files: Successor of the
convert-ids-modifications-files converter.

This converter also will convert the used rulesfiles file for the
providers.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-04-14 06:18:56 +02:00
Stefan Schantl
990d111d70 ids-functions.pl: Add support for Etags.
Etags are used to itentify if an ressource has been changed
by sending a special request and an Etag value to the server.

If the ressource has changed the server will serve the new content
otherwise it will return the 304 (Not-Modified) code.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-04-11 05:48:17 +02:00
Stefan Schantl
8d6714edc8 ids-functions.pl: Change backend to use one file to load the used
rulefiles.

Suricata seems to struggle when using multiple and/or nested includes in
the same config section. This results in a only partially loaded
confguration where not all rulefiles are loaded and used.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-04-09 14:10:13 +02:00
Stefan Schantl
8114440752 convert-ids-modification-files: New converter.
This converter is responsible to convert the old oinkmaster modification
files into the new files and format.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 12:12:55 +01:00
Stefan Schantl
849fc8ea15 ids-functions.pl: Drop oinkmaster related functions and declarations.
They are not longer needed and safely can be dropped.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:45:17 +01:00
Stefan Schantl
2deba6bf4a ids-functions.pl: Use "enabled/disabled" to mark if a rule should be
altered.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:36:24 +01:00
Stefan Schantl
794469483f ids-functions.pl: Replace call of external oinkmaster.pl to newly
introduced process_ruleset function.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:35:48 +01:00
Stefan Schantl
5a6c7bbe85 ids-functions.pl: Add process_ruleset() function.
This function is going to replace the part which currently the
oinkmaster.pl script does.

It will read in the extracted ruleset, remove duplicates and alter the
rules to alert or drop in case they match. Also rules will be enabled or
disabled if the used requested this.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:34:30 +01:00
Stefan Schantl
518cbdd389 ids-functions.pl: Add get_provider_ruleset_modifications_file().
This function will obosolete the old oinkmaster modifications files.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:34:11 +01:00
Stefan Schantl
e246285af4 ids-functions.pl: Add private function to obtain the sid and rev of a
rule.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:33:47 +01:00
Stefan Schantl
e0eb5bc737 ids-functions.pl: Add get_providers_mode() function.
This function is used to gather the modes of the configured providers
and return them as hash.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:33:28 +01:00
Stefan Schantl
65e3aef583 ids-functionsn.pl: Remove logging calls when checking free diskspace.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:17:06 +01:00
Stefan Schantl
52a557a848 ids-functions.pl: Remove logging calls from downloader.
The download script should not directly do the logging stuff.

It simply should download the files for the requested provider and
return an error code on fail.

The logging should be done at another place.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-26 11:14:40 +01:00
Stefan Schantl
e26edcc1c7 ids-functions.pl: Provide better return codes, if the downloader fails.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-25 06:03:40 +01:00
Stefan Schantl
0f2c5211f6 ids-functions.pl: Limit downloader to only one provider.
Remove the option and required code to download the rulesets
for all configured and enabled providers by just calling the downloader
function.

This cause a lot of troubles and if required, directly should be handled
by the processing script.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 21:23:31 +01:00
Stefan Schantl
235e3e92a3 ids-functions.pl: Add get_subscription_code() function.
This function can be used to obtain the subscription code of a given
configured provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 21:17:08 +01:00
Stefan Schantl
061391e776 ids-functions.pl: Use If-Modified-Since header to reduce file downloads.
When using the "If-Modified-Since" header, the server can be requested
if a modified version of the file can be served.

In case that is true, the file will be sent and stored by the downloader
function. If the file has not been touched since the last time, the
server will respond with the code "304" (Not modified).

This tells us, that the current stored file is the latest one (still up-to-date)
and we safely can skip the download attempt for this provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 20:51:56 +01:00
Stefan Schantl
f264adda35 ids-functions.pl: Re-order download request handler creation.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 20:29:21 +01:00
Stefan Schantl
7d8956083b ids-functions.pl: Early load required perl modules.
This will help us to determine if all required perl modules and their
dependencies are avail and load-able.

It also prevents us from doubble loading modules and makes development
and maintainance more easy.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 20:18:58 +01:00
Stefan Schantl
304ce130fd ids-functions.pl: Remove temporary file, if the download failed.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 20:04:30 +01:00
Stefan Schantl
e71804fb82 ids-functions.pl: Allow "3" download attempts for each provider before fail.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 20:04:03 +01:00
Stefan Schantl
de1199e2a3 ids-functions.pl: Drop downloader code for sourcefire based ruleset.
Even if the servers do not support HEAD requests, the remote filesize
(content_length) can be obtained from the connection headers.

This generic method works for all servers and therefore we do not need
the code for handle sourcefire servers in a different way anymore.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
2022-03-24 20:01:30 +01:00
Stefan Schantl
abf148cebd rules.pl: Fix creating rules for location based groups.
The former used hash value only contains the country code when
a rule for a single country should be created.

In case a location group is used the hash value refers to the group name,
which does not work here.

The required country code is part of the processed string and can be omitted
from here. This works well for single codes and location groups, because those
are processed in a loop.

Fixes #12809.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Acked-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-23 17:12:37 +00:00
Michael Tremer
77e3829dc1 linux-firmware: Compress firmware on disk
This patch enabled that we can compress any firmware files on disk. This
will save some space since /lib/firmware is becoming larger with every
release.

From formerly 828MiB, this is now using ~349MiB which is a saving of
about 480MiB on disk. This is helping us a lot fighting to contain the
distribution to 2GB on /.

Some other firmware that is installed in other packages is not
compressed with this patch which is a bit sad, but potentially not worth
the effort.

In order to ship this change with a Core Update, it might be intuitive
to remove /lib/firmware first and then extract the new update with all
new files. However, I do not know if this all will compress as well as
before since now the files are already individually compressed. It might
be a challenge to ship this.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-23 16:11:14 +00:00
Peter Müller
e68cfdb140 Core Update 166: Ship bind
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-23 09:42:57 +00:00
Matthias Fischer
5019469505 bind: Update to 9.16.27
For details see:
https://downloads.isc.org/isc/bind9/9.16.27/doc/arm/html/notes.html#notes-for-bind-9-16-27

"Security Fixes

    The rules for acceptance of records into the cache have been
    tightened to prevent the possibility of poisoning if forwarders send
    records outside the configured bailiwick. (CVE-2021-25220)

    ISC would like to thank Xiang Li, Baojun Liu, and Chaoyi Lu from
    Network and Information Security Lab, Tsinghua University, and
    Changgen Zou from Qi An Xin Group Corp. for bringing this
    vulnerability to our attention. [GL #2950]

    TCP connections with keep-response-order enabled could leave the TCP
    sockets in the CLOSE_WAIT state when the client did not properly
    shut down the connection. (CVE-2022-0396) [GL #3112]

Feature Changes

    DEBUG(1)-level messages were added when starting and ending the BIND
    9 task-exclusive mode that stops normal DNS operation (e.g. for
    reconfiguration, interface scans, and other events that require
    exclusive access to a shared resource). [GL #3137]

Bug Fixes

    The max-transfer-time-out and max-transfer-idle-out options were not
    implemented when the BIND 9 networking stack was refactored in 9.16.
    The missing functionality has been re-implemented and outgoing zone
    transfers now time out properly when not progressing. [GL #1897]

    TCP connections could hang indefinitely if the other party did not
    read sent data, causing the TCP write buffers to fill. This has been
    fixed by adding a “write” timer. Connections that are hung while
    writing now time out after the tcp-idle-timeout period has elapsed.
    [GL #3132]

    The statistics counter representing the current number of clients
    awaiting recursive resolution results (RecursClients) could
    be miscalculated in certain resolution scenarios, potentially
    causing the value of the counter to drop below zero. This has been
    fixed. [GL #3147]"

Signed-off-by: Matthias Fischer <matthias.fischer@ipfire.org>
Reviewed-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-23 09:42:41 +00:00
Peter Müller
b889a5004a Core Update 166: Ship rrdtool
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-23 09:42:17 +00:00
Matthias Fischer
24005371e1 rrdtool: Update to 1.8.0
For details see:
https://github.com/oetiker/rrdtool-1.x/releases/tag/v1.8.0

"Bugfixes

    python bindings: properly convert double values of rrd info
    failed to expand 'Py_UNUSED', Invalid usage when expanding 'Py_UNUSED'
    document --showtime in xport help output
    fix --use-nan-for-all-missing-data
    update rrdruby.pod
    add missing rrdruby.pod and rrdpython.pod to dist
    Set first_weekday to 0 (Sunday), when HAVE__NL_TIME_WEEK_1STDAY is not defined
    fix median calculation for all NaN inputs
    fix potential leak in xport during failure
    fix many warnings raised by Cppcheck
    fix many compiler warnings from latest gcc
    ensure proper initialization in rrd_daemon
    cleanup testsuite
    better testing
    avoid invalid read in rrd_client
    add symbols from rrdc to librrd
    Fix duplicate write_changes_to_disk() calls when HAVE_LIBRADOS is true and HAVE_MMAP is false
    documentation updates
    for SMIN example in docs
    fix for pyton3 compatibility
    freemem only for valid status <Christian Kr"oger>
    fix double meaning of time 0 as uninitialized value
    fix for zfs not supporting fallocate. this makes resize work on zfs
    add rrdrados.pod to dist
    fetch - do not call rrd_freemem on uninitialized pointers
    use separate pango fontmap per thread
    switch to python 3
    do not leak filename when opening a broken file
    fix leaks in rrdcached
    avoid segfault when flushing cache
    escape json in legend entries
    fix leak in xport
    make rrdcgi param parsing more robust
    fix race in journal_write"

Signed-off-by: Matthias Fischer <matthias.fischer@ipfire.org>
Reviewed-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-23 09:41:57 +00:00
Peter Müller
afd8dc466b Nmap: Update to 7.92
Please refer to https://nmap.org/changelog#7.92 for the changelog of
this version; it is too long to include it here.

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Reviewed-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-22 17:35:52 +00:00
Peter Müller
b5abd5ccd3 Core Update 166: There is no need for /dev/null in that GPG call
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-22 17:35:11 +00:00
Peter Müller
90f03efea5 Core Update 166: Introduce new 2022 Pakfire key
Cc: Michael Tremer <michael.tremer@ipfire.org>
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-22 17:34:30 +00:00
Peter Müller
fa0a7c038c Core Update 166: Perform spring clean of orphaned files
On an IPFire installation that has been around since Core Update 110
(released April 28, 2017), these files have been identified as being
orphaned, comparing to a fresh installation of Core Update 164.

To avoid such a list of files agglomerating, persons responsible for a
Core Update should check whether an updated package contains deleted
files in its rootfile. If so, they should be deleted on existing systems
via update.sh as well.

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-22 17:33:51 +00:00
Peter Müller
5a33fd9bc6 Nano: Braindead me overlooked the rootfiles were misplaced
This was introduced in b1c25d7a84, and
subsequently broke the nightly builds, and I failed to spot this in the
first place. :-/

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-22 07:51:17 +00:00
Peter Müller
7751dcccdd Core Update 166: Ship misc-progs
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-21 21:38:37 +00:00
Peter Müller
f1269489e8 Core Update 166: Drop Pakfire metadata for Nano, which is now part of the core system
Cc: Michael Tremer <michael.tremer@ipfire.org>
Cc: Jon Murphy <jon.murphy@ipfire.org>
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-21 19:10:06 +00:00
Peter Müller
451cf01677 Core Update 166: Ship and restart strongSwan
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
2022-03-21 19:04:11 +00:00
Jon Murphy
b1c25d7a84 Nano: Move nano editor from packages to core system
- this will not change the default editor `vim`

Signed-off-by: Jon Murphy <jon.murphy@ipfire.org>
Acked-by: Peter Müller <peter.mueller@ipfire.org>
Acked-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-21 19:01:43 +00:00
Peter Müller
219a998412 Core Update 166: Drop old 2007 Pakfire key, and remove it from existing installations
It is not necessary to have this key present on IPFire systems anymore,
since it has not been in use for years, and we can expect systems to be
sufficiently up-to-date, so they no longer need to rely on old updates
or add-ons signed with this key.

Also, given the current key was generated in 2018, we should consider a
Pakfire key rollover soon.

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Acked-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-21 11:46:36 +00:00
Michael Tremer
5e5ee28234 core166: Ship /etc/inittab.d
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-21 08:51:51 +00:00
Adolf Belka
a1e9afcbaf sysvinit: Update rootfile to fix bug 12797
- In sysvinit-2.97 (Dec2020) a patch was added which allows init to load configuration data
   from files stored in /etc/inittab.d/
   This modification would have come in with Core Update 155.
- When sysvinit was updated from 2.88dsf to 2.98 the /etc/inittab.d/ directory in the
   rootfile was commented out.
  Sysvinit looks to see if there are any files in inittab.d but as that directory does not
   exist then the message "No inittab.d directory found" is written during the boot and is
   also shown in the log summary.
- This patch uncomments the directory so that it will be present. No files will be placed
   in it but its prescence will stop the error message being shown during boot.

Fixes: Bug #12797
Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-21 08:51:22 +00:00
Michael Tremer
bfde1c5af3 core166: Drop openvpn-ccd-convert script from all installations
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
2022-03-21 08:50:56 +00:00