Feedback:

anomalousvalue

Use the anomalousvalue operator to compute an anomaly score for each field of each event, relative to the values of this field across other events.

Syntax

|anomalousvalue <av-options> [action=annotate|filter|summary] [pthresh=<number>] [<field-list>]

<av-options>

Required. Specify one or more option to control which fields are considered for discriminating anomalies.

Possible options:

  • maxanofreq=<float>: Maximum anomalous frequency is expressed as a floating point number between 0 and 1. Omits a field from consideration if the field is too frequently anomalous. If the ratio of anomalous occurrences of the field to the total number of occurrences of the field is greater than the maxanofreq value, then the field is removed from consideration.

    Default: 0.05

  • minnormfreq=<float>: Minimum normal frequency is expressed as a floating point number between 0 and 1. Omits a field from consideration if the field is not anomalous frequently enough. If the ratio of anomalous occurrences of the field to the total number of occurrences of the field is smaller than p, then the field is removed from consideration.

    Default: 0.01

  • minsupcount=<int>: Minimum supported count must be a positive integer. Drops a field that has a small number of occurrences in the input result set. If the field appears fewer than N times in the input events, the field is removed from consideration.

    Default: 100

  • minsupfreq=<float>: Minimum supported frequency is expressed as a floating point number between 0 and 1. Drops a field that has a low frequency of occurrence. The minsupfreq argument checks the ratio of occurrences of the field to the total number of events. If this ratio is smaller than p, the field is removed from consideration.

    Default: 0.05
action=annotate|filter|summary

Specify whether to return the anomaly score (annotate), filter out events that are not anomalous values (filter), or return a summary of anomaly statistics (summary).

Possible options:

  • annotate: Adds new fields to the events containing anomalous values.

  • filter: Returns events with anomalous values. Events without anomalous values are removed. The events that are returned are annotated, as described for annotate.

  • summary: Returns a table summarizing the anomaly statistics for each generated field. The table includes the number of events that contain this field, the fraction of events that are anomalous, what type of test (categorical or numerical) have been performed, and so on.

Default: filter
pthresh=<number>

Optional. Specify a probability threshold (as a decimal) that has to be met for a value to be considered anomalous.
<field-list>

Optional. Specify a comma-separated or space-separated list of fields that you need to analyze.

Default: empty

If the field list is not specified, all fields are considered.

Usage

You can use the anomalousvalue operator to detect outliers or anomalous values in a dataset. This can be particularly useful for identifying unusual patterns or detecting potential issues in your data. The operator applies statistical methods to determine which values deviate significantly from the expected range based on historical data.

For numerical fields, anomalousvalue identifies or summarizes the values in the data that are anomalous either by frequency of occurrence or number of standard deviations from the mean. For fields that are determined to be anomalous, a new field is added according to the selected action.

Examples

Example 1: Filter events with anomalous values below a probability threshold.

Copy 
|anomalousvalue action=filter pthresh=0.01

Example 2: Detect and annotate anomalous values in the dataset.

Copy 
|anomalousvalue action=annotate