streamstats

Feedback:

streamstats

Use the streamstats operator to generate cumulative statistics for each event in your data, streaming up to the current event. In other words, it adds new fields to your current results as it performs the specified statistics at row level, while accumulating the results from all previous events up to the current event in a specified time window.

This operator works with both eventlog and ocsf schemas:

For the eventlog table, events are ordered based on event_time and event_meta_id parameters.
For the ocsf table, events are ordered based on event_time and event_seq_id parameters to ensure proper sequential ordering across OCSF event classes.
For other tables, if no ordering is available, an internal row number is used to maintain ordering and statistics are calculated based on this.

When used with OCSF fields, streamstats supports nested object fields using dotted notation. Examples include metadata.product.name and actor.user.name.

Caution: streamstats performs expensive aggregation operations. Therefore, we recommend using this command on shorter time periods. Searches running over very large time ranges using streamstats can result in searches timing out.

Syntax

| streamstats <aggregation-list> [by <field-list>] [current=<boolean>] [window=<integer>] [limit=<integer>]

`<aggregation-list>`	Represents a comma-separated list of aggregations to be included in the result. For example: `count,sum(event_size)`. List of supported aggregations: `count` `sum` `avg` `min` `max` `range` `previous`: returns the previous value of the event in the specified window `next`: returns the next value of the event in the specified window `dc`: returns a count of distinct values within the specified window
`by <field-list>`	Specifies the list of fields to include in the group-by clause, Similar to the `aggr` operator, the `streamstats` operator allows you to create aggregations (statistics) at the group level. In this case, the aggregations are calculated for each group in the list of fields.
`current=<boolean>`	Specifies if the current row must be considered for the calculations. Setting it to `False` selects only the previous rows while excluding the current row. Default: `True`.
`window=<integer>`	Specifies the number of events to be used when calculating the statistics. This window size lets you: Consider only some rows for the calculation, and Configure how many previous events you wish to include for the calculation. Default: `0`, which means, all previous events are included in the calculation. Note: When using `dc`, `window` is required and must be between `1` and `50`. An unbounded window (`window=0`) is not supported with `dc`.
`limit=<integer>`	Specifies the limit of the number of events to be processed. Default: `1000000`.

Usage

You can use the streamstats operator to make running calculations on your events. For example, you can count the number of events by any field or to get the previous value of the field to make any further calculations.

Examples

Example 1: Eventlog with running statistics

Copy

| streamstats count as cnt, sum(bytes_in), next(bytes_in), previous(bytes_in) by app_type

Example 2: Multiple streamstats operations

Copy

| streamstats count as cnt, avg(bytes_in) by app_type | streamstats count as cnt_total

Example 3: Windowed streamstats

Copy

| streamstats count as cnt, next(bytes_in) by sourcetype window=5 current=false

Example 4: OCSF activity tracking with previous value

Note: For OCSF schema fields, array indexes start at array[1] and not array[0].

Copy

ocsf | streamstats previous(activity_name) as pr_activity | fields event_time, activity_name, metadata.product.name, pr_activity | sort event_time

Example 5: Distinct count over a sliding window

Copy

| streamstats dc(src) by sourcetype window=10