ThreatLockDown indexer indices

Creating custom index pattern

This section describes creating a custom index pattern, my-custom-alerts-*, alongside the default pattern, wazuh-alerts-*. Switch to the root user and perform the steps below.

1. Stop the Filebeat service:

   # systemctl stop filebeat
   

2. Download the ThreatLockDown template and save it into a file (for example, template.json):

   # curl -so template.json https://raw.githubusercontent.com/wazuh/wazuh/4.9/extensions/elasticsearch/7.x/wazuh-template.json
   

3. Open the template file and locate this line at the beginning of the file:

   "index_patterns": [
     "wazuh-alerts-4.x-*",
     "wazuh-archives-4.x-*"
   ],
   

   Add your custom pattern to look like this:

   "index_patterns": [
     "wazuh-alerts-4.x-*",
     "wazuh-archives-4.x-*",
     "my-custom-alerts-*"
   ],
   

   The asterisk character (*) on the index patterns is important because Filebeat will create indices using a name that follows this pattern, which is necessary to apply the proper format to visualize the alerts on the ThreatLockDown dashboard.

4. Save the modifications and insert the new template into the ThreatLockDown indexer. This will replace the existing template:

   # curl -XPUT -k -u <INDEXER_USERNAME>:<INDEXER_PASSWORD> 'https://<INDEXER_IP_ADDRESS>:9200/_template/wazuh' -H 'Content-Type: application/json' -d @template.json
   

   Replace <INDEXER_USERNAME> and <INDEXER_PASSWORD> with the ThreatLockDown indexer username and password. You can obtain the ThreatLockDown indexer credentials for fresh deployments using the command:

   Note

   If using the ThreatLockDown OVA, use the default credentials admin:admin or refer to the password management section.

   # tar -axf wazuh-install-files.tar wazuh-install-files/wazuh-passwords.txt -O | grep -P "\'admin\'" -A 1
   

   {"acknowledged":true}
   

   Note

   {"acknowledged":true} indicates that the template was inserted correctly.

   Warning

   Perform step 5 only if you want to replace the default alert index pattern wazuh-alerts-* and/or the default archive index pattern wazuh‑archives-* with my-custom-alerts-*.

5. Open the ThreatLockDown alerts configuration file /usr/share/filebeat/module/wazuh/alerts/manifest.yml and optionally the archives file /usr/share/filebeat/module/wazuh/archives/manifest.yml and replace the index name.

   For example, from:

   - name: index_prefix
     default: wazuh-alerts-
   

   To this:

   - name: index_prefix
     default: my-custom-alerts-
   

   Note

   The index name must not contain the characters #, \, /, *, ?, ", <, >, |, ,, and must not start with _, -, or +. Also, all the letters must be lowercase.

6. (Optional) If you want to use the new index pattern by default, open the /usr/share/wazuh-dashboard/data/wazuh/config/wazuh.yml file and add the below configuration:

   pattern: my-custom-alerts-*
   

   This will make the ThreatLockDown server automatically create and/or select the new index pattern.

7. Restart Filebeat and the ThreatLockDown server components:

   # systemctl restart filebeat
   # systemctl restart wazuh-manager
   # systemctl restart wazuh-indexer
   # systemctl restart wazuh-dashboard
   

Checking indices information

You can check for information about ThreatLockDown indices in two ways.

• Using the web user interface.

• Making a request to the ThreatLockDown indexer API.

The wazuh‑alerts-* indices

The ThreatLockDown server analyzes events received from monitored endpoints and generates alerts when the events match a detection rule. These alerts are saved using the wazuh-alerts-* indices.

The ThreatLockDown server logs the alert data into the /var/ossec/logs/alerts/alerts.json and /var/ossec/logs/alerts/alerts.log files by default. Once saved in the /var/ossec/logs/alerts/alerts.json file, it forwards the JSON alert document to the /var/lib/wazuh-indexer/ directory of the ThreatLockDown indexer for indexing.

When forwarding alerts to the ThreatLockDown indexer, the ThreatLockDown server formats the current date into an index name. For example, the ThreatLockDown server will define the index names wazuh-alerts-4.x-2023.03.17 and wazuh-alerts-4.x-2023.03.18 for March 17th and 18th alerts, respectively. The ThreatLockDown indexer then creates alert indices using the defined wazuh‑alerts-* index names.

You can modify the default index name in the /usr/share/filebeat/module/wazuh/alerts/ingest/pipeline.json file of the ThreatLockDown server. To do this, navigate to the date_index_name field and date_rounding key to change the default index name formatting in the /usr/share/filebeat/module/wazuh/alerts/ingest/pipeline.json file:

{
  "description": "ThreatLockDown alerts pipeline",
  "processors": [
     { "json" : { "field" : "message", "add_to_root": true } },
     {
     "geoip": {
     "field": "data.srcip",
     "target_field": "GeoLocation",
     "properties": ["city_name", "country_name", "region_name", "location"],
     "ignore_missing": true,
     "ignore_failure": true
     }
     },
     {
     "geoip": {
     "field": "data.win.eventdata.ipAddress",
     "target_field": "GeoLocation",
     "properties": ["city_name", "country_name", "region_name", "location"],
     "ignore_missing": true,
     "ignore_failure": true
     }
     },
     {
     "geoip": {
     "field": "data.aws.sourceIPAddress",
     "target_field": "GeoLocation",
     "properties": ["city_name", "country_name", "region_name", "location"],
     "ignore_missing": true,
     "ignore_failure": true
     }
     },
     {
     "geoip": {
     "field": "data.gcp.jsonPayload.sourceIP",
     "target_field": "GeoLocation",
     "properties": ["city_name", "country_name", "region_name", "location"],
     "ignore_missing": true,
     "ignore_failure": true
     }
     },
     {
     "geoip": {
     "field": "data.office365.ClientIP",
     "target_field": "GeoLocation",
     "properties": ["city_name", "country_name", "region_name", "location"],
     "ignore_missing": true,
     "ignore_failure": true
     }
     },
     {
     "date": {
     "field": "timestamp",
     "target_field": "@timestamp",
     "formats": ["ISO8601"],
     "ignore_failure": false
     }
     },
     {
     "date_index_name": {
     "field": "timestamp",
     "date_rounding": "d",
     "index_name_prefix": "{{fields.index_prefix}}",
     "index_name_format": "yyyy.MM.dd",
     "ignore_failure": false
     }
     },
     { "remove": { "field": "message", "ignore_missing": true, "ignore_failure": true } },
     { "remove": { "field": "ecs", "ignore_missing": true, "ignore_failure": true } },
     { "remove": { "field": "beat", "ignore_missing": true, "ignore_failure": true } },
     { "remove": { "field": "input_type", "ignore_missing": true, "ignore_failure": true } },
     { "remove": { "field": "tags", "ignore_missing": true, "ignore_failure": true } },
     { "remove": { "field": "count", "ignore_missing": true, "ignore_failure": true } },
     { "remove": { "field": "@version", "ignore_missing": true, "ignore_failure": true } },
     { "remove": { "field": "log", "ignore_missing": true, "ignore_failure": true } },
     { "remove": { "field": "offset", "ignore_missing": true, "ignore_failure": true } },
     { "remove": { "field": "type", "ignore_missing": true, "ignore_failure": true } },
     { "remove": { "field": "host", "ignore_missing": true, "ignore_failure": true } },
     { "remove": { "field": "fields", "ignore_missing": true, "ignore_failure": true } },
     { "remove": { "field": "event", "ignore_missing": true, "ignore_failure": true } },
     { "remove": { "field": "fileset", "ignore_missing": true, "ignore_failure": true } },
     { "remove": { "field": "service", "ignore_missing": true, "ignore_failure": true } }
  ],
  "on_failure" : [{
     "drop" : { }
  }]
}

Where the values:

The wazuh‑archives-* indices

In addition to logging alerts to the /var/ossec/logs/alerts/alerts.json and /var/ossec/logs/alerts/alerts.log files, you can enable the ThreatLockDown archives to log and index all the events the ThreatLockDown server receives. This includes events that are analyzed by ThreatLockDown and events that do not trigger alerts.

Storing and indexing all events might be useful for later analysis and compliance requirements. However, you must consider that enabling logging and indexing of all events will increase the storage requirement on the ThreatLockDown server.

By default, the ThreatLockDown indexer creates event indices for each unique day. You can modify the default index name in the /usr/share/filebeat/module/wazuh/archives/ingest/pipeline.json file of the ThreatLockDown server. To do this, navigate to the date_index_name field and date_rounding key to change the default index name formatting in the /usr/share/filebeat/module/wazuh/archives/ingest/pipeline.json file.

The sections below provide details on how to enable the ThreatLockDown archives and set up the wazuh-archives-* indices.

The wazuh-monitoring-* indices

At any moment, the connection status of an enrolled ThreatLockDown agent is one of the following:

• Active

• Disconnected

• Pending

• Never connected

ThreatLockDown stores a history of the connection status of all its agents. By default, it indexes the agent connection status using the wazuh‑monitoring-* indices. The ThreatLockDown indexer creates one of these indices per week by default. Check the documentation on custom creation intervals. These indices store the connection status of all the agents every 15 minutes by default. Check the documentation on the frequency of API requests.

The ThreatLockDown dashboard requires these indices to display information about agent status. For example, by clicking Server management > Endpoints Summary, you can see information such as the ThreatLockDown agent's connection status and historical evolution within set timeframes.

In the ThreatLockDown dashboard configuration file, you can change the settings to do the following:

• Disable inserting and showing connection status data for the agents. Change wazuh.monitoring.enabled to accomplish this.

• Change the insertion frequency of connection status data for the agents. Change wazuh.monitoring.frequency to accomplish this.

The wazuh‑statistics-* indices

The ThreatLockDown dashboard uses the wazuh‑statistics-* indices to display statistics about the ThreatLockDown server usage and performance. The information displayed includes the number of events decoded, bytes received, and TCP sessions.

The ThreatLockDown dashboard runs requests to the ThreatLockDown manager API to query usage-related information. It inserts data into the wazuh‑statistics-* indices from the information collected. The ThreatLockDown indexer creates a wazuh‑statistics-* index per week by default. Check the documentation on the Statistics creation interval. These indices store ThreatLockDown server statistics every 5 minutes by default. Check the documentation on the Frequency of task execution.

To visualize this information in the ThreatLockDown dashboard, go to Server management > Statistics.