Feedback:

Send email

superapi

Use the superapi operator to perform GET, POST, PUT, PATCH, and DELETE requests to internal ThreatStream API endpoints directly from the AQL Search interface and without additional authentication. The operator enables automation of data retrieval, updates, and bulk operations on ThreatStream. It supports flexible input specifications using fields from your pipeline, and allows you to explicitly define output fields and their types for correct parsing.

Before you begin

Before using the superapi operator, consider the following:

Fetching live data

To fetch data on the fly, always use the anomali_api_proxy placeholder table with the superapi operator.

Copy 
anomali_api_proxy | superapi

Resource consumption

Running queries with a large number of response rows or extended time ranges can lead to excessive resource consumption on the API server. To avoid service disruption, use the superapi operator cautiously and limit both the scope and frequency of your requests.

Response size limit

The superapi operator tracks the cumulative size of all HTTP responses within a single query.

When the total exceeds 50 MB, the operator stops making additional API calls. Rows processed before the limit is reached return normally. Rows that would have been processed after the limit is reached receive a warning in their output instead of API results.

Consider the following example:

Copy 
example_actors_list
| fields name as actor_name, id
| superapi endpoint='/api/v1/actor/<<id>>/tipreport/?limit=1000' input_fields='id'
| calc objects=json_extract(response_data, 'objects', ARRAY)
| mvexpand objects
| calc name=json_extract(objects_expanded, 'name')
| calc modified_ts=json_extract(objects_expanded, 'modified_ts')
| calc created_ts=json_extract(objects_expanded, 'created_ts')
| fields name, actor_name, id, modified_ts, created_ts
| where actor_name contains_ci ("APT27","APT28","APT34","APT41")
| sort modified_ts desc

In this query, the first line starts with a list of actors. Each row has an actor_name and an id. The superapi line then makes one HTTP API call per actor row, replacing <<id>> with the actor ID in that row.

For example, if the input table has the following rows:

Row actor_name id API call
1 APT27 101 /api/v1/actor/101/tipreport/?limit=1000
2 APT28 102 /api/v1/actor/102/tipreport/?limit=1000
3 APT34 103 /api/v1/actor/103/tipreport/?limit=1000

Because the endpoint uses limit=1000, each call can return up to 1,000 threat bulletins for that actor. If those reports are large, each response can be several MB, and could therefore exceed the 50 MB limit.

Note: Because API calls are executed in parallel, the order in which rows are processed is non-deterministic. Once the limit is reached, the specific rows that are skipped may vary between runs.

When the limit is reached, Anomali Search displays the following warning:

Copy 
API response size limit reached. Some results may be incomplete due to large API responses.

Skipped rows contain the following in their output:

Copy 
{"warning": "API response size limit reached. Row skipped."}

Additionally, caching responses ensures completed requests are not reprocessed again for non-idempotent requests (POST/PATCH), preventing duplicate records.

Recommended Best Practice

If you see the limit warning, reduce the scope of your query using one or more of the following approaches:

  • Limit input rows. Add | limit N before | superapi to cap the number of API calls made. For example:

    Copy 
    | makeresults count=100
    | streamstats count AS counter
    | limit 20
    | superapi endpoint='/api/v1/intelligence/?limit=1000&offset=<<counter>>' input_fields='counter'

  • Use a smaller page size. Lower the limit= parameter in your endpoint URL to reduce the size of each individual response. For example, use limit=100 instead of limit=1000.

  • Filter before the API call. Add | where conditions before | superapi to reduce the number of rows that need API enrichment.

Syntax

Copy 
superapi
    endpoint = <string>
    method = <string>
    payload | body_content= <string>
    output = <string>
    output_types = <string>

 

endpoint

Specifies the endpoint used to search on ThreatStream.

For a list of available ThreatStream API endpoints, refer to the ThreatStream API Reference Guide, which can be downloaded on the Downloads page.

 
method

Specifies the method to perform the request to ThreatStream.

Default: get
payload | body_content

The payload or body_content keywords are used to specify the payload to be sent in the request.

When using the payload keyword, the value must be a valid JSON-formatted string. This payload will be included in the body of the request.

The body_content keyword allows you to include input data in the request body using fields from the pipeline. You can reference these fields using the syntax <<field>>.

For example,

Copy 
body_content=' { "model_type" : "<<model_type>>", "name": "<<search_name>>"}'
output

Specifies a comma-separated list of fields to extract from the response. These fields are included in the output of the operator.

Key features:

  • Full path support: You can extract fields using their full path in the response structure.

    For example, if the response contains the following structure:

    Copy 
    {
      "objects": [
        {
          "id": 1,
          "name": "test"
        }
      ]
    }

    Then the following fields are equivalent:

    • output='id'

    • output='objects.{}.id'

  • Alias Support: You can rename output fields using the as keyword.

    For example,

    output='id as identifier,name'

  • Extracting Fields from Nested Paths: When the response contains nested objects, you can extract information from each key in the hierarchy.

    For example, if the response contains the following structure:

    Copy 
    {
        "results": {
            "threat_model": {
                "created": {
                    "actor": {
                        "2025-03-01": 1,
                        "2025-03-03": 41,
                        "2025-03-04": 126
                    },
                    "infrastructure": {
                        "2025-03-01": 57,
                        "2025-03-02": 49
                    }
                }
            }
        }
    }

You can extract every key in the hierarchy and get the count at the end of it in different output fields.

The syntax for this is the following:

  • Path of the desired hierarchy

  • as keyword to delimit the hierarchy and aliases

  • Aliases for each field in the hierarchy.

The final output to get the data in the hierarchy is as follows:

output=threat_model.{}.{}.{} as

status;type;event_time;cnt

This output generates 4 fields:

  • status

  • type

  • event_time

  • cnt

output_types

Specifies a comma-separated list of field types corresponding to each output field defined in the output parameter.

If the number of output_types specified doesn’t match the number of output fields or the type specified is not supported, then an error is displayed.

Supported TS API output types:

  • string

  • integer

  • number

  • timestamp

  • boolean

  • list

Examples

Example 1: Retrieve the first 20 threat intelligence entries with a confidence greater than 80 and a severity of “low".

Copy 
anomali_api_proxy | superapi
    endpoint = "/api/v2/intelligence?limit=20&&confidence__gt=80&severity=low"
    method = get
    output = "objects.{}.id as id, objects.{}.uuid as uuid, objects.{}.value as value, objects.{}.confidence as confidence, objects.{}.severity as severity, objects.{}.created_ts as created_ts"
    output_types = "number,string,string,number,string,timestamp"

Example 2: Retrieve observables with private tags.

Copy 
anomali_api_proxy | superapi
    endpoint = "/api/v1/intelligence/?limit=1000&is_public=false&tags.tlp__isnull=false"
    method = get
    output = "objects.{}.id as id, objects.{}.value as value, objects.{}.itype as itype, objects.{}.confidence as confidence, objects.{}.severity as severity, objects.{}.tags.{}.name as tag_name, objects.{}.tags.{}.tlp as tag_tlp, objects.{}.created_ts as created_ts"
    output_types = "number,string,string,number,string,string,string,timestamp"

Example 3: Combine lookup table results from multiple API endpoints.

Copy 
ocsf | superapi endpoint = '/api/v1/xdr/event/lookup/test_table_iris' output='result.modified_ts as modified_ts, result.modified_by_user as modified_by' | calc source='test_table_iris' | appendtable [| superapi endpoint = '/api/v1/xdr/event/lookup/test_table_iris_passthrough' output='result.modified_ts as modified_ts, result.modified_by_user as modified_by' | calc source='test_table_iris_passthrough']

Example 4: Calls the XDR IOC exclusion list API endpoint and maps the response fields into the result set for each OCSF event.

Copy 
ocsf | superapi endpoint='/api/v1/xdr/excludelist/ioc' output='{}.created_by as created_by,{}.created_ts as created_ts,{}.modified_by as modified_by,{}.modified_ts as modified_ts,{}.username as username,{}.value as value'