Feedback:

outlier

Use the outlier operator to modify numeric outputs for given input fields that contain outliers.

Syntax

|outlier [field_list=<field-list>] [outlier_action={"transform|remove"}] [mark={True|False}] [param=<number>] [uselower={True|False}]

field_list=<field-list>

Optional. Specify a comma-separated or space-separated list of numeric fields that you need to modify.

outlier_action={"transform|remove"}

or

outlier_action={"tf|rm"}

Optional. Possible options:

  • "transform", or "tf": Applies a transform to bound the outlier values.

  • "remove", or "rm": Removes all fields outside the threshold.

Default: "transform"
mark={True|False}

Optional. Specify a boolean value to add a suffix of 000 to the output of a transformed outlier.

Default: False

Note: The output field will be a string type rather than a numeric type, so avoid using this field in graphical representations or further processing.
param=<number>

Optional. Specify a numeric value for calculating the outlier. The outlier is calculated as follows:

upper quartile + inter quartile range * param.

Default: 2.5

Note: The inter-quartile range is calculated from (upper quartile - lower quartile). If uselower is set to True, a similar calculation is used, but is as follows:

lower quartile - inter-quartile range * param.
uselower={True|False}

Optional. Specify a boolean value to determine whether the outlier logic should be applied to outlier values below the mean.

Default: False

Usage

You can use the outlier operator to transform or remove numeric values in specific fields. If you use the outlier operator without any attributes, it will try to transform all numeric fields.

The outlier operator processes datasets by filtering based on the inter-quartile range (IQR). The IQR is calculated as the difference between the 25th and 75th percentiles of numeric fields. Let's assume that a field's value in an event is less than 25th percentile - param*IQR or greater than 75th percentile + param*IQR. In that case, depending on the specified action parameter, either the field is transformed, or the event is removed.

Use anomalydetection to find fields that contain outliers.

Examples

Example 1: Transform all numeric fields to handle outliers with default parameters.

Copy 
|outlier

Example 2: Transform a field to handle outliers below the mean.

Copy 
|outlier val2 uselower=true

Example 3: Transform a field and mark outlier values with a suffix.

Copy 
|outlier val2 mark=true

Example 4: Transform multiple fields to handle lower outliers.

Copy 
|outlier val1 val2 uselower=true

Example 5: Remove outliers from results.

Copy 
|outlier val1 action=remove

Example 6: Counts events by activity type, then identifies statistical outliers in those counts, marking and transforming anomalous activity volumes.

Copy 
ocsf | aggr count as test by activity_name | outlier test outlier_action='transform' mark=true