Google Cloud Platform

Anomali Security Analytics can ingest Google Cloud Platform (GCP) Firewall, GCP VPC, and any other GCP log data. The following article explains the steps you must complete to ingest GCP logs into Anomali.

Google Service Account

Step 1: Set Up GCP Resources

Create a Google Cloud project. See Google's guide to Creating and managing projects for more information.
Create an IAM service account to be used with the project. See Google's guide to Creating and managing service accounts for more information.
Within the service account configuration:
- On the Keys tab, generate a service account key file in JSON format. See Google's guide to Create and manage service account keys for more information.
  
  The key file is downloaded automatically as soon as you generate it. Securely save the key for Step 2: Add GCP as a new log source. Anomali will use the service account key file to authenticate its requests to Google Cloud Storage.
- On the Details tab, copy the email address associated with the service account, so that you can paste it when you set up service account access to the Google Cloud Storage bucket.
Create a Google Cloud Storage Bucket to be used within the project. See Google's guide to Create buckets for more information.

Within the bucket configuration:
- Select Edit access and click Add Principal to display a form you can use to grant access to the service account. See Google's guide to Use IAM with buckets for more information.
- In the New principals field, paste the service account and select it.
- Select Storage Object Viewer as the role.
Create a Cloud Function, as follows: See Google's guide to Cloud Functions triggers for more information.
1. Select 2nd gen environment and click Add Eventarc Trigger.
2. Select Cloud Storage as the event provider. Select google.cloud.storage.object.v1.finalized as the event, and select the storage bucket from Step 1: Set Up GCP Resources. You can use the Default compute service account for the Eventarc trigger.
3. Click Save Trigger.
4. Edit the function code, as follows:
5. Paste the following code into the function:
```
import functions_framework
import requests

@functions_framework.cloud_event
def send_gcp_storage_event(cloud_event):
    response = requests.post(
        url="https://api.threatstream.com/api/v1/xdr/cloud_logsource/gcp_storage/event/",
        json={
            "event_id": cloud_event["id"],
            "event_type": cloud_event["type"],
            "data": cloud_event.data
        },
        headers={
            'Content-type': 'application/json',
            'Authorization': 'apikey <email>:<api_key>',
        }
    )
    response.raise_for_status()
```
6. Replace the values for the Authorization API key with the email address and API key for the Anomali service account.
Route logs to the Google Cloud Storage bucket. See Google's guide to Route Logs to Supported Destinations for more information.

Step 2: Add GCP Firewall or VPC or Auto as new log source

To add GCP Firewall or VPC as a new log source on the Anomali platform:

Navigate to Search > Log Source Management.
Click New to add a new log source.
Enter a name for the log source.
(Optional) Enter a description for the log source.
Select GCP Firewall or GCP VPC or GCP Auto as the source type.
Select API as the ingestion method.
Select API as the ingest method.
Provide the Basic Configuration details:
1. Google User: Enter the provisioned storage account name. Refer to Step 1: Set Up GCP Resources.
2. GCP Service Account Credentials: Click Browse and then locate, select, and upload the key file that contains the GCP service account credentials. Refer toStep 1: Set Up GCP Resources.
3. GCP Storage Bucket Name: Enter the bucket name from Step 1: Set Up GCP Resources.
(Optional) Next to the Basic Configuration section, click on the Advanced Options link to provide additional information about the event log data:
1. Enter one or more parameters that provide additional context about the data. For example, "File format" indicates the format of the event log data being shipped, or "File extension" that indicates the extension of the logs.
2. Enter the corresponding value of the parameter. For example, CSV is a file format. Similarly, ".gzip" is a file extension.
Click Save to add the log source.

Step 3: Configure permissions

You can set permissions to decide which roles can view, add, or edit log sources. To do so, configure the following parameters:

Owner: Select an owner of the view from the list of users. This is the email address of the owner.
Type: Select the type of permission assigned to the view:
1. Private: Only the owner has read and write privileges, that is, no other user can access or edit the view.
2. Shared: Allows you to set the read-write privileges for a role that is mapped to different users, granting read-write permissions to all users assigned that role. You can also assign read or write permissions to everyone in your organization.

Step 4: Check log source status

Once added, the status of the log source becomes Create Requested. See Log Source Statuses to understand the different types statuses assigned to log sources.

Step 5: Receive confirmation email

You will receive a notification email from Anomali once you have created a new log source request. The Log Source Management page will show the status as Create Requested.

Once the log source has been provisioned and is ready for use, Anomali will send you another email and the Log Source Management page status for the log source will change to Active.

AWS S3

If your logs are already in S3, you can proceed directly to Step 2: Add GCP Firewall or VPC or Auto as new log source. If not, Anomali recommends following Step 1: Ship data to S3 using NXLog to use NXLog to first ship the logs to S3. After S3 receives the data, it sends updates to an Amazon SQS queue you designate, enabling Anomali to automatically pull the logs from S3 whenever it receives a new notification from the queue.

Onboarding Steps

Step 1: Ship data to S3 using NXLog

See Using NXLog to Ship Data to set up NXLog to send data into S3.

Step 2: Set up SQS queue

Once your data is in S3, you can send it to Anomali using Amazon SQS. See Setting up Amazon SQS queue for more information.

Step 3: Enable AWS S3 notifications

Enable AWS S3 notifications for Amazon SQS to receive change notifications when you add logs. See Amazon's guide to configuring AWS S3 notification for SQS for more information.

Step 4: Add GCP VPC or Firewall or Auto as new log source

On the Anomali platform, add GCP VPC or Firewall or Auto as a new log source, as follows:

Navigate to Search > Log Source Management.
Click New to add a new log source.
Enter a name for the log source.
(Optional) Enter a description for the log source.
Select it as the log source.
Select AWS S3 as the ingest method.
Ensure S3 notification are enabled. See Step 2: Enable AWS S3 notifications for more information.
Provide the Basic Configuration details:
1. AWS SQS Queue URL: Enter the URL of your AWS SQS queue. See Setting up Amazon SQS queue for more information.
2. AWS Secret Access Key: Provide the AWS secret access key.
3. AWS Access Key ID: Enter the AWS Access ID.
4. File Selector: Provide the path to the event log data files. Anomali recommends the following practices for the path:

Sourcetype must be the path prefix, as follows:
<sourcetype>/year=2025/month=04/day=05/path/to/file.
For example: gcp_vpc/year=2025/month=03/day=22/events/event_data_logs/.
Sourcetype name must not contain any spaces: For example, "gcp_vpc" or "gcp_firewall" are correct, however "gcp firewall" is incorrect.

(Optional) Next to the Basic Configuration section, click on the Advanced Options link to provide additional information about the event log data:
1. Enter one or more parameters that provide additional context about the data. For example, "File format" indicates the format of the event log data being shipped, or "File extension" that indicates the extension of the logs.
2. Enter the corresponding value of the parameter. For example, CSV is a file format. Similarly, ".gzip" is a file extension.
Click Save to add the log source.

Step 5: Configure permissions

You can set permissions to decide which roles can view, add, or edit log sources. To do so, configure the following parameters:

Owner: Select an owner of the view from the list of users. This is the email address of the owner.
Type: Select the type of permission assigned to the view:
1. Private: Only the owner has read and write privileges, that is, no other user can access or edit the view.
2. Shared: Allows you to set the read-write privileges for a role that is mapped to different users, granting read-write permissions to all users assigned that role. You can also assign read or write permissions to everyone in your organization.

Step 6: Check log source status

Once added, the status of the log source becomes Create Requested. See Log Source Statuses to understand the different types statuses assigned to log sources.

Step 7: Receive confirmation email

You will receive a notification email from Anomali once you have created a new log source request. The Log Source Management page will show the status as Create Requested.

Once the log source has been provisioned and is ready for use, Anomali will send you another email and the Log Source Management page status for the log source will change to Active.