ids.cgi: Allow whitespaces when parsing the rules files.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

convert-ids-multiple-providers: Fix setting ownership for the main
oinkmaster provider includes file.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Fix writing for used provider rulefiles.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

convert-ids-multiple-providers: Proper open the oinkmaster providers
sids file for writing.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

convert-ids-multiple-providers: Fix typo which tried to load the wrong
lib.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

convert-ids-multiple-providers: Remove old used rulefiles file.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

convert-ids-multiple-providers: Always remove old enabled / disabled
sids files.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

convert-ids-multiple-providers: Restart suricata afterwards.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

backup.pl: Launch convert-ids-multiple-providers if neccessary.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

IDS: Add convert-ids-multiple-provider script.

This converter does all the magic to convert any suricata
based IPFire version to work with the new multiple providers
IDS.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Do not expect a space before the sid when parsing rulefiles.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

convert-snort: Adjust converter to work with new IDS.

Only in case if somebody tries to import such an old backup.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Remove as deprecated marked variables.

They are not needed anymore.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ruleset-sources: Update download URL for Talos rulesets.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ruleset-sources: Add additional providers.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Sort elements in providers dropdown menu.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Only write existing provider specific used rulesfiles
files into main include yaml file.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

IDS: Move read_enabled_disabled_sids_file() function to ids-functions.pl.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

backup: Adjust includes file to include new IDS files into backups.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

Revert "ids-functions.pl: Remove config files on rulesdir cleanup."

Not all config files are shipped by the rulesets. For example the
"threshold.conf" and the "referneces.conf" are not include in each
ruleset.

Therefore it is not a common way to delete all config files. It is
much safer to simple keep them and overwrite existing ones by the
generated ones.

This reverts commit a71c3c9dcc60541aa4504d0f1fb0a78c0d58ed5e.

ids-functions.pl: Move code to handle plain rules files to
extractruleset() function.

Now everithing which is extracting or moving stored ruleset files is
easily accessing via one function which takes care about.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Only read providers used rulefiles file if it exists.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Fix accidently commited debug file path.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

update-ids-ruleset: Port script to work with multiple providers.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

langs-de.pl: Fix grammar.

Even as a native speaker, it seems german sometimes is a very difficult language......

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Add code to handle the reset of a provider to it's defaults.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Make backend code for forced ruleset update working again.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Add section for additional provider actions.

This section only will be displayed when an existing provider will be
edited and allows to reset a provider back to it's defaults or to force
a ruleset update.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Fix display issue with colum backgound colour in provider list.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Handle oinkmaster provider includes when deleting a provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Fix function call of get_used_provider_rulesfile_file().

The function is locatated in the IDS module and therefore needs to be
called from there.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Add/Remove provider file include in oinkmaster providers
include file when toggeling a provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Check if the file exists bevore adding it to the
oinkmaster provider includes file.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Introduce alter_oinkmaster_provider_includes_file().

This function can be used to directly modify the desired file.

It takes two arguments:
 * An action which could be "add" or "remove"
 * A provider handle, which should be added or removed.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Use get_oinkmaster_provider_modified_sids_file() function.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Introduce get_oinkmaster_provider_modified_sids_file()
function.

This function simply returns the gernerated path and filename for the
provider specific modified sids file.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

IDS: Redesign backend for enabled/disabled sids in rulefiles.

The enabled or disabled sids now will be written to an own
provider exclusive configuration file which dynamically will
be included by oinkmaster if needed.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Bring back usage of whitelist.rules and local.rules
files.

They now automatically will be included as static includes if the files
are present.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Use get_ruleset_date() from ids-functions.pl.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Introduce get_ruleset_date() function.

This function is used to get the creation date of the stored rules files
of a given provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Prevent from chainging the provider when editing an existing
one.

This commit locks the dropdown menu for selecting a provider, in case
an existing one should be edited.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Add hardcoded error message to language files.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Fix check and message when trying to enable suricata without
any enabled or no provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Lock the CGI when a provder will be deleted.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Drop old code to handle the settings of the ruleset section.

This entirely has been replaced by the providers section and the code to
handle the actions of this section.

Therefore this code is not longer needed.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Finish code to handle the removal of a provider from the list.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Finish code to handle toggeling a provider enabled/disabled.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

IDS: Redesign backend for used provider rulesfiles.

The selected rulesfiles of a provider now will be written to an own
provider exclusive yaml file, which will be included dynamically when
the provider is enabled or not.

This allows very easy handling to enable or disable a provider, in this
case the file which keeps the enabled providers rulesets only needs to
be included in the main file or even not.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Add action if a new provider is added.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Introduce move_tmp_ruleset() function.

This function is used to move an extracted temporary ruleset to
the rules location.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Fix another typo.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Remove accidently commited debug code.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Rework oinkmaster() to use get_enabled_providers
function.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Introduce get_enabled_providers() function.

This function simply returns an array with all enabled ruleset
providers.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Fix typo.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Use get_used_rulesfiles function from ids-functions.pl.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Introduce get_used_rulesfiles() function.

This function simply returns an array which contains the used rulesfiles
files.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Introduce drop_dl_rulesfile().

This tiny function is used, to delete the stored rulesfile in case a
provider will be deleted.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Rework function for modify-sid file to be more
generic.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Add cleanup_tmp_directory() function.

As the name of the function already says, it is responsible to
delete all temporary files after ruleset generation.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Remove config files on rulesdir cleanup.

They every time oinkmaster is called will be generated.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Rework oinkmaster() function.

Rework the function to work with the latest changes and multiple
providers.

The function now does the following:

* Extract the stored rules tarballs for all enabled providers.
* Copy rules files for enabled providers which provide plain files.
* Still calls oinkmaster to set up the rules and modify them.
* Calls the merge functions for classification and sid to msg files.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Assign temporary rules and conf path to variables.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Introduce merge_sid_msg() function.

This function is used to merge the sid to message mapping files
from various providers.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Fix typo.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Introduce merge_classifications() function.

This function is used to merge the individual classification files
provided by the providers.

The result will be written to the classification.config which will be
used by the IDS.

Fixes #11884.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Introduce extraceruleset() function.

This function is used to extract the required config and rules files
from the stored rules tarball for a given ruleset provider.

* The files will be extracted to a temporary directory layout in
  "/tmp/ids_tmp".

* Names of config files will be adjusted in case multiple providers
  offers the same config files, which is very common.

* The name of the single rulefiles will be adjusted to start with
  the vendors name to allow assigning them very easily to a single
  ruleset provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Always delete temporary file.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Fix typo.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ruleset-sources: Rename file to plain.

This is used if a provider offers a plain rulefile instead an archive.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Allow downloadruleset() function to deal with
multiple ruleset providers.

When calling the function now a single ruleset provider handle
can be specified to only download this ruleset or by adding "all" or
leaving the handle blank a download of all configured rulesets can be
triggered.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Introduce private _get_dl_rulesfile() function.

This function can be used to generate/get the absolute file and path
for a given ruleset provider.

The files will be stored in the usual "/var/tmp" folder with a new
file format based on the dl_file type and the provider.

Examples could be:
* /var/ipfire/idsrules-emerging.tar.gz
* /var/ipfire/idsrules-registered.tar.gz
* /var/ipfire/idsrules-somprovider.rules

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Run in perl strict mode.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ruleset-sources: Fix website url for community ruleset.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ruleset-sources: Replace subscription code placeholder.

Replace the <oinkcode> placeholder by the more generic
<subscription_code>.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Add code to handle enable/disable a provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Add code to handle enable/disable autoupdate for a provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Fix type in method.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Remove accidently commited commented code snipped.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Correctly use "enabled" for checked checkboxes.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Read-in providers settings file when neccessary.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Add code to add/edit a ruleset provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

Update language files.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Introduce providers_settings_file.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Rework rulesetsettings section.

* The page and section now supports multiple ruleset providers at once.
* Adding / Editing a ruleset provider has been moved to a own sub-page.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ruleset-sources: Fix website details for emergingthreats provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Move configuration of ruleset autoupdate intervall to IDS main
section.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Stop showing ruleset date on customize rulest sub-page.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Add "Back" button to customize ruleset sub-page.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Move / Splitt main page and customize ruleset subpage.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Only read-in ruleset if neccessary.

This process takes some time, especially on huge rulesets.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Add button to customize the ruleset.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Introduce and use get_provider_name() function.

This function is used to grab the name of a provider by the given
handle.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Move the section to customize the IDS ruleset to a function.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Remove unused rulesetsources hashes.

They have been superseeded by the new ruleset sources file.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Require ruleset-sources file for provider details.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Make CGI work with new ruleset-sources file.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Add get_ruleset_providers() function.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Make downloader work with new ruleset-sources file
format.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ruleset-sources: Rework file format and data.

The file now contains a lot more of data and easily can be extended
to provide more and new providers.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>