Related topics:
Feedback:

Using the Search RESTful API

AQL Search is accessible through a REST API. The API allows you to create a new search, monitor the status of the search job, and successfully retrieve the search results.

Notes:

  • Only the API endpoints described in this document are supported.

  • Only 10 Create New Search API requests per minute per organization are allowed.

About the API

The API offers bi-directional interaction with Search.

Accessing the API

To access the API related to Search, use the following base URL:

https://api.threatstream.com/api/v1/xdr/<resource>/

where

<resource> is an API endpoint; for example, /search/jobs

Authenticating to Search

Making requests through the API requires authenticating to ThreatStream using your username (the email address associated with your ThreatStream account) and your dedicated API Key. You can find your username and API Key on the My Profile tab within ThreatStream settings.

Specifying your username and API Key in the header of the request is the most secure method of authentication. Anomali recommends using this authentication method.

curl 'https://api.threatstream.com/api/v1/xdr/<resource>/' -H 'Authorization: apikey <username>:<api_key>'

Once you authenticate, each API call uses your API Key to successfully make API requests.

Create a New Search

To trigger a new search and create a new search job.

Note:

Only 10 new search API requests per minute per organization are allowed.

Request

/search/jobs/

HTTP method: POST

Attributes: See table below.

Attributes

Attribute Type Description

query

 string

Required. Any search query. For information on how to create a query, see Understanding the Search Bar.
source string

(Optional) source of the request, set as third_party_<your_app_name>.

For example, source="third_party_mynewapp".

Default: source is set to "third_party", if you don't provide a value for your_app_name.

time_range

 string

Required. Time range (in UNIX timestamp milliseconds) and the timezone in which records are searched. For example, you can issue a search in the records created within the last 15 minutes. For example:

"time_range": {
 "from": 1679777426000,
 "to": 1680123026000,
 "timezone":"America/Los_Angeles"
}

Response

Format: JSON

Attributes: See table below.

Attributes

Attribute Type Description
job_id string Job ID of the search job.

Examples

Request:

curl example:

curl -X 'POST' 'https://api.threatstream.com/api/v1/xdr/search/jobs/' \
-H 'Authorization: apikey <username>:<api_key>' \
-H 'accept: application/json' -H 'Content-Type: application/json' \
-d '{"query": 'eventlog | where sourcetype contains \"crowdstrike\" or sourcetype contains \"edr\" \
| aggr count as cnt by src | sort cnt desc | limit 100',"source": "third_party_mynewapp",\
"time_range": {"from": 1679777426000,"to": 1680123026000,"timezone": "America/Los_Angeles"}}'

Python example:

import requests
import json

response = requests.post(
	'https://api.threatstream.com/api/v1/xdr/search/jobs/',
	headers={
		'Accept': 'application/json',
		'Content-Type': 'application/json',
		'Authorization': 'apikey <username>:<api_key>'
	},    data=json.dumps({
		"source": "third_party_mynewapp",
		"query": 'eventlog | where sourcetype contains "crowd-strike" or \
					sourcetype contains "edr" | \
					aggr count as cnt by src | \
					sort cnt desc | limit 100',
		"time_range": {
			"from": 1679777426000,
			"to": 1680123026000,
			"timezone": "America/Los_Angeles"
		}
	})
)

if response.status_code == 200:
	try:
		response_data = response.json()
		job_id = response_data.get('job_id')
		print("Job ID: ",job_id)
	except ValueError:
		print("Response is not in JSON format:",response.text)
else:
	print("Error:", response.status_code, response.text)

Response:

{
	"job_id": 6f0f5c333a8b48e1b2659b4578c1b51f
}

Successful response status code: 200

Failed response error codes: 400, 500

Get Search Status

To get the status of a search using its job ID.

Request

/search/jobs/{job_id}/

HTTP method: GET

Attributes: See table below.

Attributes

Attribute Type Description

job_id

 string Required. Job ID of the search job.

Response

Returns a JSON object that contains the following information:

  • Histogram information

  • Aggregation status

  • Progress status

  • Scan count

  • Job status

    • QUEUED

    • CANCELLED

    • RUNNING

    • DONE

    • FAILED

Format: JSON

Attributes: See table below.

Attributes

Attribute Type Description
bucket_length integer Width of each histogram bin depending on the search duration.
end_time integer End timestamp of the time interval of the search query, in UNIX timestamp milliseconds. For example, if the search query interval lies between 0-100, then start_time=0 and end_time=100.
histogram_buckets list

Number of histogram buckets used to bucket the search result. Attributes:

  • count—Number of search results that matched the search query in a particular 10 minute histogram bin.

  • length—Width of each histogram bin depending on the search duratoin.

  • start_time—Timestamp of a particular histogram bin.
is_aggregated

boolean

Whether the search results are aggregated.

TRUE—Query is an aggregation operation. For example:

eventlog | where sourcetype contains \"crowdstrike\" or sourcetype contains \"edr\" | aggr count as cnt by src | sort cnt desc | limit 100"

FALSE—Query is not an aggregation operation. For example:

eventlog | fields src_ip, dest_ip

num_of_bucket integer Number of buckets in which you receive all the search results. Default is 60.
progress numeric Indicates the search progress. Numeric float value between 0 and 1.
scanned integer Number of records scanned by the search query.
start_time integer Start timestamp of the time interval of the search query, in UNIX timestamp milliseconds. For example, if the search query interval lies between 0-100, then start_time=0 and end_time=100.
status string

Status of the search. Possible options: 

  • QUEUED: Search job is waiting for execution.

  • CANCELLED: Query was canceled while it was running.

  • RUNNING: Search in progress.

  • DONE: Search query is complete and the search results are retrieved.

  • FAILED: When no result is returned due to many reasons, such as "Timeout", "Server error", "Job Id not found", or "Unexpected error".

Note: If more than 120 seconds lapse in the QUEUED status, you will encounter a FAILED status. You can reissue a new search.
total integer Total number of records matched by the search query.

Examples

Request:

curl example:

curl -X 'GET' 'https://api.threatstream.com/api/v1/xdr/search/jobs/6f5f5c716a8b48e1b2659b5918c1b51f/' \
-H 'Authorization: apikey <username>:<api_key>' \
-H 'accept: application/json'

Python example:

import requests
response = requests.get(
	"https://api.threatstream.com/api/v1/xdr/search/jobs/6f5f5c716a8b48e1b2659b5918c1b51f/",
	headers={
		'Accept': 'application/json',
		'Authorization': 'apikey <username>:<api_key>'
	}
)

Response:

{
	"bucket_length": 1200000, 
	"end_time": 1727650800000,
	"histogram_buckets": [
		{
			"count": 0, 
			"length": 1200000, 
			"start_time": 1727613600000
		}, 
		{
			"count": 0, 
			"length": 1200000, 
			"start_time": 1727614800000
		}
	], 
	"is_aggregated": false,
	"num_of_bucket": 60,
	"progress": 1.0, 
	"scanned": 35004,
	"start_time": 1727613600000,
	"status": "DONE", 
	"total": 35004
}

Successful response status code: 200

Failed response error codes: 404, 500

Get Search Results

To retrieve the results of a search job based on its job ID.

Request

/search/jobs/{job_id}/results/

HTTP method: GET

Attributes: See table below.

Attributes

s
AttributeTypeDescription

job_id

string

Required. Job ID of the search job.

offsetinteger

Optional. Offset of the search results. For example, if offset=10 and fetch_size=30, then this API will return results indexed 10 to 30.

Default: 0.

fetch_sizeinteger

Optional. Number of rows returned.

Maximum rows: 1000.

Default: 25.

Response

Format: JSON

Attributes: See table below.

Attributes

Attribute Type Description
count integer Number of records returned.
fields list

Search fields that you provided in the search query in the POST /search/jobs/ API call.

For example, if you provide event_time, event_meta_id, timestamp as part of your search query in the POST /search/jobs/ request, fields in the response would contain the event time, event ID, and the timestamp, as follows:

fields = ["event_time", "event_meta_id", "timestamp"]

has_next boolean

Whether there are more results pages than the current page.

TRUE—More results present beyond the current page.

FALSE—The current page is the last page with all the results.
is_aggregated boolean

Whether the search results are aggregated.

TRUE—Query is an aggregation operation. For example:

eventlog | where sourcetype contains \"crowdstrike\" or sourcetype contains \"edr\" | aggr count as cnt by src | sort cnt desc | limit 100"

FALSE—Query is not an aggregation operation. For example:

eventlog | fields src_ip, dest_ip

records list List of records containing the fields included in the fields response attribute.
result_row_count integer Total number of records retrieved by the search. Use offset and fetch_size to decide how many retrieved rows you access in one go.
search_end_time integer End timestamp of the search, in UNIX timestamp milliseconds.
search_start_time integer Start timestamp of the search, in UNIX timestamp milliseconds.
status string

Status of the search.

Possible options: 

  • QUEUED: Search job is waiting for execution.

  • CANCELLED: Query was canceled while it was running.

  • RUNNING: Search in progress.

  • DONE: Search query is complete and the search results are retrieved.

  • FAILED: When no result is returned due to many reasons, such as "Timeout", "Server error", "Job ID not found", or "Unexpected error".

Note: If more than 120 seconds lapse in the QUEUED status, you will encounter a FAILED status. You can reissue a new search.
types array Data type of the attributes in each retrieved search record.

Examples

Request:

curl example:

curl -X 'GET' 'https://api.threatstream.com/api/v1/xdr/search/jobs/6f5f5c716a8b48e1b2659b5918c1b51f/results/?offset=0&fetch_size=25' \
-H 'Authorization: apikey <username>:<api_key>' \
-H 'accept: application/json'

Python example:

import requests
response = requests.get(
	'https://api.threatstream.com/api/v1/xdr/search/jobs/6f5f5c716a8b48e1b2659b5918c1b51f/results/?offset=0&fetch_size=25',
	headers={
		'Accept': ‘application/json',
		'Authorization': 'apikey <username>:<api_key>'
	},
	params={
		'offset': 0,
		'fetch_size': 25
	}
)

Response:

{
	"count": 10, 
	"fields": [
		"event_time", "sourcetype", "dcid", 
		"src", "action", "dest", "url", "user", 
		"event_meta_id", "_raw_id", "day", "hour", "minute"
	],
	"has_next": false, 
	"is_aggregated": false, 
	"records": [
		[1727647847687, "myexamplesourcetype", "78", 
		null, "allow", null, "https://www.abc-example.com/feed", null, 
		8587425684, null, "29", "22", "10-0"],
		[1727647468096, "aws_cloudtrail", "1", 
		"192.1.0.0", "created", "", "", "MyAdminTeam", 
		8690012349, null, "29", "22", "00-0"],
	], 
	"result_row_count": 10,
	"search_end_time": 1727990933868, 
	"search_start_time": 1727990924712, 
	"status": "DONE", 
	"types": ["timestamp", "string", "string", 
		"string", "string", "string", "string", "string", 
		"number", "number", "string", "string", "string"]
}

Successful response status code: 200

Failed response error codes: 404, 500