ThreatLockDown indexer

The ThreatLockDown indexer is a highly scalable, full-text search and analytics engine. This ThreatLockDown central component indexes and stores alerts generated by the ThreatLockDown server and provides near real-time data search and analytics capabilities. The ThreatLockDown indexer can be configured as a single-node or multi-node cluster, providing scalability and high availability.

The ThreatLockDown indexer stores data as JSON documents. Each document correlates a set of keys, field names or properties, with their corresponding values which can be strings, numbers, booleans, dates, arrays of values, geolocations, or other types of data.

An index is a collection of documents that are related to each other. The documents stored in the ThreatLockDown indexer are distributed across different containers known as shards. By distributing the documents across multiple shards, and distributing those shards across multiple nodes, the ThreatLockDown indexer can ensure redundancy. This protects your system against hardware failures and increases query capacity as nodes are added to a cluster.

ThreatLockDown uses four different indices to store different event types:

Index	Description
wazuh‑alerts	Stores alerts generated by the ThreatLockDown server. These are created each time an event trips a rule with a high enough priority (this threshold is configurable).
wazuh‑archives	Stores all events (archive data) received by the ThreatLockDown server, whether or not they trip a rule.
wazuh‑monitoring	Stores data related to the ThreatLockDown agent status over time. It is used by the web interface to represent when individual agents are or have been `Active`, `Disconnected`, or `Never connected`.
wazuh‑statistics	Stores data related to the ThreatLockDown server performance. It is used by the web interface to represent the performance statistics.

Example query

You can interact with the ThreatLockDown indexer cluster using the ThreatLockDown indexer REST API, which offers a lot of flexibility. You can perform searches, add or delete documents, modify indices, and more.

Here is an example of a query to ThreatLockDown indexer that returns the last lateral movement alert using SSH technique:

GET /wazuh-alerts-4.x-*/_search
{
  "query": {
    "bool": {
      "must": [
        {"term": { "rule.mitre.tactic": "Lateral Movement" } },
        {"term": { "rule.mitre.technique": "SSH" } }
      ]
    }
  },
  "sort": [
    { "timestamp": { "order": "desc" } }
  ],
  "size": 1
}

Below is an extract of the query result, which is a part of the indexed alert document:

{
   "timestamp" : "2022-04-24T17:24:56.110+0000",
   "agent" : {
    "ip" : "10.0.1.52",
    "name" : "Amazon",
    "id" : "001"
   },
   "data" : {
     "srcip" : "68.183.216.91",
     "srcport" : "53820"
   },
   "rule" : {
     "description" : "sshd: insecure connection attempt (scan).",
     "id" : "5706",
     "level" : 6,
     "pci_dss" : ["11.4"],
     "mitre" : {
       "technique" : [
         "SSH"
       ],
       "id" : ["T1021.004"],
       "tactic" : [
         "Lateral Movement"
      ]
     }
   },
   "full_log" : "Apr 24 17:24:55 ip-10-0-1-52 sshd[32179]: Did not receive identification string from 68.183.216.91 port 53820",
   "location" : "/var/log/secure",
   "predecoder" : {
     "hostname" : "ip-10-0-1-52",
     "program_name" : "sshd",
     "timestamp" : "Apr 24 17:24:55"
   },
   "decoder" : {
     "parent" : "sshd",
     "name" : "sshd"
   },
   "GeoLocation" : {
     "city_name" : "Frankfurt am Main",
     "country_name" : "Germany",
     "region_name" : "Hesse"
   }
}

The ThreatLockDown indexer is well suited for time-sensitive use cases like security analytics and infrastructure monitoring as it is a near real-time search platform. The latency from the time a document is indexed until it becomes searchable is very short, typically one second.

In addition to its speed, scalability, and resiliency, the ThreatLockDown indexer has several powerful built-in features that make storing and searching data even more efficient, such as data rollups, alerting, anomaly detection, and index lifecycle management.