predict

Feedback:

Send email

predict

Use the predict operator to make predictions for time series data. The operator will generate predictions for in-sample data and forecasts for out-sample data.

Syntax

`<field>+`	Required. Specify a comma-separated or space-separated list of numeric fields to predict.
`<model>`	Required. Specify the model to be used for prediction. Possible options: `autoregressive`: It is an autoregressive AR-X(p) model using conditional maximum likelihood (OLS). When predicting using the autoregressive model, the number of lags to be included in the model needs to be calculated first (this is an automatic process performed by the operator). For example, depending on the data, the best lags can be included in the model: [1,4,12]. It means, data points: 1, 4, and 12 will be included in the model, therefore, data points from 1 to 12 will be missing for the predicted data. In this example, the calculated AR-X model is AR-X(12). AR-X(p) models can be up to AR-X(15), which means you need to provide at least ~30 data points to get results. If you want to get more accurate results, then you need to provide more data points. `sarimax`: It is an Seasonal AutoRegressive Integrated Moving Average with eXogenous regressors (SARIMAX) model. Similar to the autoregressive model, it can incorporate trend and seasonal variations when predicting. This model automatically selects seasonal orders based on data without the need to manually configure it. It is very powerful as combined with exogenous data can improve the predictive power of the model, therefore, improve the forecast accuracy.
`forecast_k=<int>`	Optional. Specify the number of points to be forecast. These points are beyond the data provided, therefore, if you want to forecast 12 points in a monthly data set, you can specify `forecast_k=12`. If you only want to generate a prediction for in-sample data, then you can specify `forecast_k=0`. Default: `5`
`conf_interval=<int>`	Optional. Specify the confidence interval for the forecasts made by the `predict` operator. It is a percentage representing the probability that the future values will fall within the specified range around the predicted values. Default: `95`
`holdback=<int>`	Optional. Specify a portion of data that should be withheld from the model training process. The data can be used to evaluate the model's performance. If you have 100 data points and set holdback to `10`, the last 10 data points will not be used in training the model. This applies only for the Autoregressive model. It will be ignored for the SARIMAX model. Default: `0`
`correlate=<field>`	Optional. Specify the field that can be used as exogenous data to be associated to the training model process. It must have the same number of observations as the list of fields to predict, otherwise, the training process will fail.
`trend=<bool>`	Optional. Specify whether you want to include a trend in your prediction. `false`: No trend. `true`: Constant and time trend. Default: `false`
`seasonal=<bool\|string>`	Optional. For the autoregressive model, it is a boolean value. Specify `true` if you want to include seasonal dummies in the model. Default: `false` For the SARIMAX model, it must be a string containing 4 digits: `(P,D,Q,s)`. `D` indicates the integration order of the process. `P` and `Q` indicate the autoregressive (AR) and moving average (MA) orders. `s` is an integer indicating the periodicity (number of periods in a season). Default: `‘0,0,0,0’`
`with <timestamp_field>`	Optional. Specify the time field to be used in the forecast. According to the last value of this field, the forecasts will be generated using the `forecast_k` parameter. It should be in the timestamp format in milliseconds.

Usage

You can use the predict operator to predict future behavioral data using different algorithms which can give different results. This way, you can easily find the best algorithm for the data.

Note: To be able to get results, you need to provide time series data with only one frequency, otherwise, the operator will not be able to determine the frequency to use (if there are two or more) and will not generate results.

The predict operator can be an invaluable tool for forecasting and mitigating cybersecurity threats. It can be very useful when trying to forecast future time series data, or even to validate the model by comparing the predicted values with the actual values.

For each predicted field, three additional fields will be included in the results:

<field>_predicted: Predicted point for the specific field.
<field>_lower: Lower limit for the confidence interval for the predicted point.
<field>_upper: Upper limit for the confidence interval for the predicted point.

Examples

Example 1: Predict cars field using autoregressive model with confidence interval and holdback.

Copy

test_predict_table | predict cars autoregressive conf_interval=95 forecast_k=5 holdback=5 seasonal=false with time_stamp | sort time_stamp asc

Example 2: Predict field using SARIMAX model.

Copy

test_predict_table | predict cars sarimax conf_interval=95 forecast_k=5 seasonal='1,1,1,12' with time_stamp | sort time_stamp asc

Example 3: Generates hourly event count predictions using an autoregressive model trained on the OCSF event time trend.

Copy

ocsf | timetrend count as cnt span=1h | predict cnt autoregressive