ids-functions.pl: Create default rules file file.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

suricata: Set correct ownership for default rules file.

The file has to be write-able for the WUI.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

convert-ids-multiple-providers: Set correct ownership for default rules
file.

Otherwise the file would belong to root and is not write-able by the
WUI.

Fixes #12759.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

convert-ids-multiple-providers: Do not try to set ownership for file
which is not longer used.

Fixes #12758.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ruleset-sources: Update download URL for Talos rulesets.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Log the download attempt of a ruleset.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

Revert "ids-functions.pl: Check if the generated stored rulesfile of a provider"

This commit causes massive troubles in the downloader and therefore
needs to be reverted.

This reverts commit 577e3304953c708bce7f6c067bf34c3f585d3a0f.

ids-functions.pl: Check if given filename exists bevore call stat on it.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Proper return flase if the downloaded rulestarball
cannot be stored.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

suricata.yaml: Set collection of stat to off.

Suricata will print a warning on startup if the collection of stats
is enabled but no stats logger, which will print them out is enabled.

Acctually we do not use any stats so this safely can be disabled.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

suricata.yaml: Set default log level to Info.

This will prevent suricata from displaying a warning on startup and
anyway would be the log level which suricata switches in such a case.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

suricata.yaml: Add config options for modbus, dnp3 and enip protocols.

All of them are disabled by default, but may be needed in some
environments and so easily can be enabled there.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Drop accidently commited debug output.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Set bypass flag for whitelisted hosts.

When adding a host to the whitelist set the bypass flag to
immediate take the load from the IDS.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Dynamically generate file of default suricata rules.

The "/var/ipfire/suricata/suricata-default-rules.yaml" file, now
dynamicall will be generated, based on the enabled application layer
protocols.

Only existing rulefiles for enabled app layer protocols will be loaded.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Add get_suricata_enable_app_layer_protos().

This function call suricata to obtain a list of enabled application
layer protocols (application/protocol parsers).

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

suricata: Fix ownership of the classification.config file.

The file has to be write-able for the nobody user.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Remove config files when cleaning up the rules
directory.

If there are one, they safly can be removed because the *.config files
now live in a different folder.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Adjust classification file for new path.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

suricata: Load *.config files from default location

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

ids-functions.pl: Do not call stat if no file has been given.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Check if the generated stored rulesfile of a provider
exists before returning the filename.

This will prevent from using and processing non existing files.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

update-ids-ruleset: Early exit script if lockfile exists.

This prevents from running the script while the WUI is performing
operations at the same time or to launch multiple instances of the
script.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Use experimental smartmatch.

This will prevent from spawning the http error log with warnings.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Do not expect a space after the msg tag has been closed while
processing rules.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Proper return N/A if no ruleset date could be
determined.

If no timestamp could be grabbed for rulestarball of a given provider,
return N/A.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Always write used providers rulefiles file.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Add the provider handle if the forced update of a provider
fails.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Return N/A if not date for a ruleset could be
determined.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Add some more sanity checks when adding a new provider.

* Check if the system is online.
* Check if enough free disk space is available.
* Abort whith an error message if the ruleset could not be
  downloaded.

In error case the provider now will be removed again from the file which
keeps the configured providers. Sadly it needs to be added first because
otherwise the downloader could not read the required values from it.....

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Use newly intruduced functions when removing a provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Introduce remove_provider().

This function is used to remove a configured provider by it's ID.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Introduce get_provider_handle().

This function is used to get the configured provider handle by a a given ID.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Regenerate ruleset if a provider get re-enabled.

Otherwise it could happen, that there are no rules files for this
specific provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Fix check when changing the IDS to monitor mode or drop mode.

The test condition was wrong here and therefore oinkmaster never has
been executed when this setting has been changed.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Allow whitespaces when parsing the rules files.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

convert-ids-multiple-providers: Fix setting ownership for the main
oinkmaster provider includes file.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Fix writing for used provider rulefiles.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

convert-ids-multiple-providers: Proper open the oinkmaster providers
sids file for writing.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

convert-ids-multiple-providers: Fix typo which tried to load the wrong
lib.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

convert-ids-multiple-providers: Remove old used rulefiles file.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

convert-ids-multiple-providers: Always remove old enabled / disabled
sids files.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

convert-ids-multiple-providers: Restart suricata afterwards.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

backup.pl: Launch convert-ids-multiple-providers if neccessary.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

IDS: Add convert-ids-multiple-provider script.

This converter does all the magic to convert any suricata
based IPFire version to work with the new multiple providers
IDS.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Do not expect a space before the sid when parsing rulefiles.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

convert-snort: Adjust converter to work with new IDS.

Only in case if somebody tries to import such an old backup.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Remove as deprecated marked variables.

They are not needed anymore.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ruleset-sources: Update download URL for Talos rulesets.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ruleset-sources: Add additional providers.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Sort elements in providers dropdown menu.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Only write existing provider specific used rulesfiles
files into main include yaml file.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

IDS: Move read_enabled_disabled_sids_file() function to ids-functions.pl.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

backup: Adjust includes file to include new IDS files into backups.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

Revert "ids-functions.pl: Remove config files on rulesdir cleanup."

Not all config files are shipped by the rulesets. For example the
"threshold.conf" and the "referneces.conf" are not include in each
ruleset.

Therefore it is not a common way to delete all config files. It is
much safer to simple keep them and overwrite existing ones by the
generated ones.

This reverts commit a71c3c9dcc60541aa4504d0f1fb0a78c0d58ed5e.

ids-functions.pl: Move code to handle plain rules files to
extractruleset() function.

Now everithing which is extracting or moving stored ruleset files is
easily accessing via one function which takes care about.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Only read providers used rulefiles file if it exists.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Fix accidently commited debug file path.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

update-ids-ruleset: Port script to work with multiple providers.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

langs-de.pl: Fix grammar.

Even as a native speaker, it seems german sometimes is a very difficult language......

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Add code to handle the reset of a provider to it's defaults.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Make backend code for forced ruleset update working again.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Add section for additional provider actions.

This section only will be displayed when an existing provider will be
edited and allows to reset a provider back to it's defaults or to force
a ruleset update.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Fix display issue with colum backgound colour in provider list.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Handle oinkmaster provider includes when deleting a provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Fix function call of get_used_provider_rulesfile_file().

The function is locatated in the IDS module and therefore needs to be
called from there.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Add/Remove provider file include in oinkmaster providers
include file when toggeling a provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Check if the file exists bevore adding it to the
oinkmaster provider includes file.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Introduce alter_oinkmaster_provider_includes_file().

This function can be used to directly modify the desired file.

It takes two arguments:
 * An action which could be "add" or "remove"
 * A provider handle, which should be added or removed.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Use get_oinkmaster_provider_modified_sids_file() function.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Introduce get_oinkmaster_provider_modified_sids_file()
function.

This function simply returns the gernerated path and filename for the
provider specific modified sids file.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

IDS: Redesign backend for enabled/disabled sids in rulefiles.

The enabled or disabled sids now will be written to an own
provider exclusive configuration file which dynamically will
be included by oinkmaster if needed.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Bring back usage of whitelist.rules and local.rules
files.

They now automatically will be included as static includes if the files
are present.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Use get_ruleset_date() from ids-functions.pl.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Introduce get_ruleset_date() function.

This function is used to get the creation date of the stored rules files
of a given provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Prevent from chainging the provider when editing an existing
one.

This commit locks the dropdown menu for selecting a provider, in case
an existing one should be edited.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Add hardcoded error message to language files.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Fix check and message when trying to enable suricata without
any enabled or no provider.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Lock the CGI when a provder will be deleted.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Drop old code to handle the settings of the ruleset section.

This entirely has been replaced by the providers section and the code to
handle the actions of this section.

Therefore this code is not longer needed.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Finish code to handle the removal of a provider from the list.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Finish code to handle toggeling a provider enabled/disabled.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

IDS: Redesign backend for used provider rulesfiles.

The selected rulesfiles of a provider now will be written to an own
provider exclusive yaml file, which will be included dynamically when
the provider is enabled or not.

This allows very easy handling to enable or disable a provider, in this
case the file which keeps the enabled providers rulesets only needs to
be included in the main file or even not.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Add action if a new provider is added.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Introduce move_tmp_ruleset() function.

This function is used to move an extracted temporary ruleset to
the rules location.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Fix another typo.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Remove accidently commited debug code.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Rework oinkmaster() to use get_enabled_providers
function.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Introduce get_enabled_providers() function.

This function simply returns an array with all enabled ruleset
providers.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Fix typo.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids.cgi: Use get_used_rulesfiles function from ids-functions.pl.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Introduce get_used_rulesfiles() function.

This function simply returns an array which contains the used rulesfiles
files.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Introduce drop_dl_rulesfile().

This tiny function is used, to delete the stored rulesfile in case a
provider will be deleted.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Rework function for modify-sid file to be more
generic.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Add cleanup_tmp_directory() function.

As the name of the function already says, it is responsible to
delete all temporary files after ruleset generation.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Remove config files on rulesdir cleanup.

They every time oinkmaster is called will be generated.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Rework oinkmaster() function.

Rework the function to work with the latest changes and multiple
providers.

The function now does the following:

* Extract the stored rules tarballs for all enabled providers.
* Copy rules files for enabled providers which provide plain files.
* Still calls oinkmaster to set up the rules and modify them.
* Calls the merge functions for classification and sid to msg files.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Assign temporary rules and conf path to variables.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Introduce merge_sid_msg() function.

This function is used to merge the sid to message mapping files
from various providers.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>

ids-functions.pl: Fix typo.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>