score

Feedback:

Send email

score

Use the score operator to execute statistical tests to verify the results produced by machine learning models.

Syntax

|score <scoring-method> <a_field_1, a_field_2, ... a_field_n> against <b_field_1, b_field_2, ... b_field_m> [normalize={true|false}]

<scoring-method>

Specify a scoring method:

accuracy_score

Use accuracy scoring to get the prediction accuracy between actual-labels and predicted-labels. This method is the sklearn.metrics.accuracy_score from scikit-learn. Refer to scikit-learn documentation for complete details:

https://scikit-learn.org/stable/modules/generated/sklearn.metrics.accuracy_score.html
confusion_matrix

Use the confusion matrix to evaluate the accuracy between actual-labels and predicted-labels. This method is the sklearn.metrics.confusion_matrix method from scikit-learn. Refer to scikit-learn documentation for complete details:

https://scikit-learn.org/stable/modules/generated/sklearn.metrics.confusion_matrix.html
mean_absolute_error

Use the mean absolute error scoring to calculate regression loss between actual_fields and predicted_fields. This method is the sklearn.metrics.mean_absolute_error. method from scikit-learn. Refer to scikit-learn documentation for complete details:

https://scikit-https://scikit-learn.org/stable/modules/generated/sklearn.metrics.mean_absolute_error.html#sklearn.metrics.mean_absolute_error
mean_squared_error

Use the mean squared error score to calculate regression loss between actual_fields and predicted_fields. This method is the sklearn.metrics.mean_squared_error method from scikit-learn. Refer to scikit-learn documentation for complete details:

https://scikit-https://scikit-learn.org/stable/modules/generated/sklearn.metrics.mean_squared_error.html#sklearn.metrics.mean_squared_error

<x_field_n> against <x_field_n>

Comma-separated list of fields to compare.

[normalize={true|false}]

The accuracy_score method supports an option to specify your data normalization preference:

true: Return the fraction of correctly classified samples.
false: Return the raw count of correctly classified samples.

The implicit default if this option is not specified is true.

Usage

You can use the score operator to execute statistical tests to verify the results produced by machine learning models. The score operator is a versatile tool that can be applied across various use cases to validate models and their statistical outcomes. Depending on the nature of your data and the specific problem you aim to address, you can select the most appropriate scoring method provided by the score command.

The accuracy_score method supports pairwise comparisons between two sets of fields.

The other supported methods are used to compare the actual value and the predicted value.

Examples

Example 1: Calculate accuracy score with raw count of correctly classified samples.

Copy

|score accuracy_score a1,a2 against p1,p2 normalize=false

Example 2: Calculate accuracy score as fraction of correctly classified samples.

Copy

|score accuracy_score a1,a2 against p1,p2 normalize=true

Example 3: Generate confusion matrix for actual and predicted values.

Copy

|score confusion_matrix actual_value against predicted_value

Example 4: Calculate mean absolute error between actual and predicted values.

Copy

|score mean_absolute_error actual_value against predicted_value

Example 5: Calculate mean squared error between actual and predicted values.

Copy

test_score_table | score mean_squared_error actual against predicted

Example 6: Filters and cleans network flow fields, fills nulls with defaults, sorts by most recent event, then calculates the mean absolute error between source and destination port numbers.

Copy

ocsf | where severity in ('alert', 'info') | fields severity, src_endpoint.interface_name, src_endpoint.port, dst_endpoint.interface_name, dst_endpoint.port, src_endpoint.ip, dst_endpoint.ip | fillnull src_endpoint.interface_name value='no source' | fillnull src_endpoint.ip, dst_endpoint.ip | fillnull dst_endpoint.interface_name value='empty_dest' | sort event_time desc | score mean_absolute_error src_endpoint.port against dst_endpoint.port