Using ThreatLockDown to monitor GCP services

ThreatLockDown helps to increase the security of a GCP infrastructure by collecting and analyzing log data.

In the Prerequisites section there is information on:

  • The required dependencies and credentials to access the services.

The Supported services section contains detailed instructions to:

  • Configure and set up all the supported services.

  • Configure ThreatLockDown to collect the logs.

Collecting and analyzing log data to increase the security of a GCP infrastructure can be achieved by using:

  • The Pub/Sub integration module.

  • The Storage integration module.

Google Cloud Storage

ThreatLockDown can process logs stored in a Google Cloud Storage bucket using the gcp-buckets module, a highly configurable module that collects any log stored in the bucket and processes the events using the defined threat detection rules and decoders.

Google Pub/Sub

ThreatLockDown can use the Google Cloud Pub/Sub messaging and ingestion service. It is widely used for event-driven systems and streaming analytics. It allows sending and receiving messages between applications. The ThreatLockDown module uses it to fetch different kinds of events (Data access, Admin activity, System events, DNS queries, etc.) from the Google Cloud infrastructure. Once events are collected, ThreatLockDown processes them using its threat detection rules.

The ThreatLockDown module for GCP gcp-pubsub provides the capability to monitor GCP-based services.

Data flow

The data flow between the ThreatLockDown module and Cloud Pub/Sub looks as follows:

  1. A publisher application creates a topic in the Cloud Pub/Sub service and sends messages to the topic. In this example, the publisher applications will be sending Activity logs and Cloud DNS events.

  2. Each published message is retained until it is acknowledged by the ThreatLockDown module consuming messages from that subscription.

  3. The ThreatLockDown module pulls the messages using its subscription to the Cloud Pub/Sub service.

  4. The ThreatLockDown module receives all messages from its subscription and acknowledges each one to the Cloud Pub/Sub service.

  5. Finally, the messages are removed from the subscription’s message queue.

Performance

The ThreatLockDown module for GCP Pub/Sub uses a Python library to pull the messages from the topic. Its implementation has design limitations when integrated with the ThreatLockDown module.

The module's performance benefits from horizontal scaling. To achieve the performance boost, it is necessary to configure the module for the same subscription on different machines (either managers or agents).

In addition, it is possible to use multiple threads to pull a larger number of messages using the num_threads parameter in the module configuration.