Agentic forecasting with skforecast-ai¶

What is skforecast-ai?¶

skforecast-ai is an AI forecasting assistant that pairs a deterministic engine, powered by skforecast, with an LLM reasoning layer. Simply provide a time series, and the assistant automatically profiles the data, selects a model using established best practices, and evaluates its performance. It returns both the final forecast and the runnable skforecast script that produced it.

It is organized around a single core object, the ForecastingAssistant, which consists of two complementary components:

• Deterministic Engine (Rule-based and Reproducible): Profiles the data, selects a forecaster and estimator, derives lags and preprocessing steps, runs backtesting, and produces the final forecast. Crucially, it outputs the exact standalone skforecast script that generated the results. Given the same inputs and configuration, this workflow is guaranteed to be reproducible.

• Reasoning Layer (LLM-powered): Accessed primarily via the ask() method, this layer interprets and explains the objects and results you pass to it: data profiles, modeling plans, validation choices, backtesting outputs, and forecasts. The LLM acts strictly as an interpreter; it does not rerun the workflow or silently change modeling recommendations behind the scenes. Agentic features, such as the LLM-guided refine_plan() or create_cv(), are separate, explicit steps where the LLM suggests adjustments that are then implemented transparently in deterministic code.

The animation below follows one series through the assistant: what stays on your machine, what the LLM sees, and how its suggestions are validated before they reach the plan.

Two ways to use skforecast-ai¶

skforecast-ai supports two distinct workflows using the same underlying forecasting engine:

• The Fast Path: Use this when you want a forecast or backtest result in a single call. The assistant profiles the data, builds the modeling plan, executes the workflow, and returns the results alongside the reproducible skforecast code.

• The Step-by-Step Path: Use this when you want granular control to inspect or adjust intermediate decisions. You can manually create a profile, build a plan, optionally refine it with the LLM, define a validation strategy, evaluate the model, and then generate the forecast.

A useful mental model is that forecasting and validation are separate branches. Once you have a profile and a plan, you can use forecast() to produce future predictions directly, or backtest() to evaluate the model's performance on historical data. You can also use compare() to evaluate several candidate configurations under the same cross-validation strategy and obtain a ranked leaderboard, so the best configuration is chosen from measured performance rather than intuition.

The ask() method is available in both workflows. It can explain a profile, a plan, a generated script, a validation setup, or a forecast, backtest or comparison result, or answer general forecasting questions. It never executes the workflow or modifies your parameters.

The following example walks through the fast path: the quickest way to go from raw data to a validated forecast with minimal setup. It is ideal when you want rapid results and trust the assistant to make sensible, baseline modeling decisions on your behalf. If you prefer to understand and control what happens under the hood step by step, visit the step-by-step path tutorial for a more detailed walkthrough.

Assistant initialization¶

The first step is to instantiate a ForecastingAssistant, which will be responsible for executing the entire workflow (profiling, planning, backtesting, and forecasting), as well as explaining the outputs and suggesting improvements.

To activate the optional LLM support, users must pass a string in the format 'provider:model_name' (for example, 'openai:gpt-5.5', 'google:gemini-3.8-flash', 'anthropic:claude-sonnet-5', or 'ollama:qwen3:8b'). For hosted providers, the corresponding API key must be available as an environment variable or passed explicitly when creating the assistant. Your dataset is never sent: the LLM receives only metadata and summary statistics, never the raw time series values. In this tutorial, we keep send_data_to_llm=False (the default), so passing a result to ask(), whose own predictions and metrics are sent, emits a DataSentToLLMWarning as a reminder.

In [1]:

# Data processing
# ==============================================================================
import os
import pandas as pd
from skforecast.datasets import fetch_dataset

# Plots
# ==============================================================================
import matplotlib.pyplot as plt
import plotly.graph_objects as go
import plotly.io as pio
import plotly.offline as poff
pio.templates.default = "seaborn"
poff.init_notebook_mode(connected=True)
plt.style.use('seaborn-v0_8-darkgrid')

# skforecast and skforecast-ai
# ==============================================================================
import skforecast
import skforecast_ai
from skforecast_ai import ForecastingAssistant
from skforecast.model_selection import TimeSeriesFold

color = '\033[1m\033[38;5;208m' 
print(f"{color}Version skforecast_ai: {skforecast_ai.__version__}")
print(f"{color}Version skforecast: {skforecast.__version__}")

# Data processing # ============================================================================== import os import pandas as pd from skforecast.datasets import fetch_dataset # Plots # ============================================================================== import matplotlib.pyplot as plt import plotly.graph_objects as go import plotly.io as pio import plotly.offline as poff pio.templates.default = "seaborn" poff.init_notebook_mode(connected=True) plt.style.use('seaborn-v0_8-darkgrid') # skforecast and skforecast-ai # ============================================================================== import skforecast import skforecast_ai from skforecast_ai import ForecastingAssistant from skforecast.model_selection import TimeSeriesFold color = '\033[1m\033[38;5;208m' print(f"{color}Version skforecast_ai: {skforecast_ai.__version__}") print(f"{color}Version skforecast: {skforecast.__version__}")

Version skforecast_ai: 0.4.2
Version skforecast: 0.26.0

✏️ Note

If you do not have access to an LLM assistant, you can still follow the full tutorial using only the deterministic methods. Profiling, planning, backtesting, and forecasting all run without an LLM. Only the ask() explanations and the LLM-guided variants of refine_plan() and create_cv() require a configured LLM; their deterministic counterparts (for example, refine_plan() with explicit overrides and prompt=None) work without one.

In [2]:

# LLM-enabled assistant
# ==============================================================================
LLM_MODEL = "google:gemini-3.8-flash"
api_key = os.getenv("GOOGLE_API_KEY")

assistant = ForecastingAssistant(
    llm=LLM_MODEL, api_key=api_key, send_data_to_llm=False
)

# Using aws bedrock
# ==============================================================================
# assistant = ForecastingAssistant(
#     llm='bedrock:eu.anthropic.claude-sonnet-4-6',
#     base_url="eu-west-1"
# )

# Assistant without reasoning layer
# ==============================================================================
# assistant = ForecastingAssistant()

# LLM-enabled assistant # ============================================================================== LLM_MODEL = "google:gemini-3.8-flash" api_key = os.getenv("GOOGLE_API_KEY") assistant = ForecastingAssistant( llm=LLM_MODEL, api_key=api_key, send_data_to_llm=False ) # Using aws bedrock # ============================================================================== # assistant = ForecastingAssistant( # llm='bedrock:eu.anthropic.claude-sonnet-4-6', # base_url="eu-west-1" # ) # Assistant without reasoning layer # ============================================================================== # assistant = ForecastingAssistant()

Before going on, check_llm() reports how the configuration resolves (provider, credentials, endpoint, installed extras) without calling the model, so a missing key shows up here instead of in the first ask(). See Configuring the LLM for the details.

In [3]:

# Check the LLM configuration
# ==============================================================================
assistant.check_llm()

# Check the LLM configuration # ============================================================================== assistant.check_llm()

Out[3]:

                          LLM Configuration Check                           
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Check        ┃ Value                                                     ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Status       │ ok                                                        │
├──────────────┼───────────────────────────────────────────────────────────┤
│ Provider     │ google                                                    │
├──────────────┼───────────────────────────────────────────────────────────┤
│ Model        │ gemini-3.8-flash                                          │
├──────────────┼───────────────────────────────────────────────────────────┤
│ Credentials  │ explicit api_key                                          │
│              │ Explicit api_key given; the environment is not consulted. │
├──────────────┼───────────────────────────────────────────────────────────┤
│ Base URL     │ None                                                      │
├──────────────┼───────────────────────────────────────────────────────────┤
│ Dependencies │ installed                                                 │
├──────────────┼───────────────────────────────────────────────────────────┤
│ Test call    │ skipped                                                   │
└──────────────┴───────────────────────────────────────────────────────────┘

⚠️ Your data stays private

Enabling an LLM never sends your time-series data to the model provider. The assistant passes only a summary of the dataset (frequency, date range, target statistics, missing values, significant lags) and the forecaster configuration, never the raw observations. Results passed to ask() are the exception: their own predictions and metrics are sent, because a question about a result cannot be answered from summary statistics alone, and a DataSentToLLMWarning reminds you of it. Pass send_data_to_llm=True to acknowledge it and silence the warning. See What is sent to the LLM for the full list.

Data¶

The data in this document represent the hourly usage of the bike share system in the city of Washington, D.C. during the years 2011 and 2012. In addition to the number of users per hour, information about weather conditions and holidays is available.

In [4]:

# Downloading data
# ==============================================================================
data = fetch_dataset('bike_sharing', raw=True)
data = data[['date_time', 'users', 'holiday', 'weather', 'temp']]
data.head()

# Downloading data # ============================================================================== data = fetch_dataset('bike_sharing', raw=True) data = data[['date_time', 'users', 'holiday', 'weather', 'temp']] data.head()

╭───────────────────────────────── bike_sharing ──────────────────────────────────╮
│ Description:                                                                    │
│ Hourly usage of the bike share system in the city of Washington D.C. during the │
│ years 2011 and 2012. In addition to the number of users per hour, information   │
│ about weather conditions and holidays is available.                             │
│                                                                                 │
│ Source:                                                                         │
│ Fanaee-T,Hadi. (2013). Bike Sharing Dataset. UCI Machine Learning Repository.   │
│ https://doi.org/10.24432/C5W894.                                                │
│                                                                                 │
│ URL:                                                                            │
│ https://raw.githubusercontent.com/skforecast/skforecast-                        │
│ datasets/main/data/bike_sharing_dataset_clean.csv                               │
│                                                                                 │
│ Shape: 17544 rows x 12 columns                                                  │
╰─────────────────────────────────────────────────────────────────────────────────╯

Out[4]:

	date_time	users	holiday	weather	temp
0	2011-01-01 00:00:00	16.0	0.0	clear	9.84
1	2011-01-01 01:00:00	40.0	0.0	clear	9.02
2	2011-01-01 02:00:00	32.0	0.0	clear	9.02
3	2011-01-01 03:00:00	13.0	0.0	clear	9.84
4	2011-01-01 04:00:00	1.0	0.0	clear	9.84

✏️ Note

skforecast-ai is ready to preprocess the data, but it is recommended that users apply their own preprocessing steps before using the assistant. This ensures the data is in the desired format and any necessary transformations have been applied before proceeding with the forecasting workflow.

In [5]:

# Interactive plot of time series
# ==============================================================================
fig = go.Figure()
fig.add_trace(
    go.Scatter(x=data['date_time'], y=data['users'], mode='lines', name='Train')
)
fig.update_layout(
    title  = 'Number of users',
    xaxis_title="Time",
    yaxis_title="Users",
    legend_title="Partition:",
    width=800,
    height=400,
    margin=dict(l=20, r=20, t=35, b=20),
    legend=dict(orientation="h", yanchor="top", y=1, xanchor="left", x=0.001)
)
fig.show()

# Interactive plot of time series # ============================================================================== fig = go.Figure() fig.add_trace( go.Scatter(x=data['date_time'], y=data['users'], mode='lines', name='Train') ) fig.update_layout( title = 'Number of users', xaxis_title="Time", yaxis_title="Users", legend_title="Partition:", width=800, height=400, margin=dict(l=20, r=20, t=35, b=20), legend=dict(orientation="h", yanchor="top", y=1, xanchor="left", x=0.001) ) fig.show()

For a deeper walkthrough of the exploratory analysis behind this dataset, see the skforecast example: Forecasting time series with skforecast, XGBoost, LightGBM and CatBoost.

Forecasting with the assistant¶

The forecast() method is the fastest way to generate predictions. In a single call, it executes the full pipeline (profile → plan → execute) and returns a ForecastResult holding the predictions, the evaluation metrics (when available), and the exact standalone skforecast script that produced them.

The behavior is controlled by a single switch, test_size, which selects one of two modes:

• Prediction mode (test_size = None, the default): The model is trained on the entire dataset and forecasts the next steps time points into the future. Because there is no ground truth to compare against, no metrics are returned. If the historical data contains exogenous variables, their future values must be explicitly supplied via the exog argument.

• Evaluation mode (test_size is set). The dataset is split into train and test sets, the model is trained on the training portion, and predictions for the test window are compared against the held-out actuals to compute metrics. In this mode, the test-set exogenous values are taken from the split, so exog must not be passed.

The main arguments are:

Argument	Type	Description
data	Series, DataFrame, str, Path	Input dataset or path to a CSV file. When a Series is passed, the target is taken from its name.
steps	int	Forecast horizon (number of steps ahead to predict).
target	str, list of str	Column to forecast. Optional when data is a Series. Pass a list of column names for wide-format multi-series.
date_column	str	Column holding the timestamps. When None, the index of data must already be a DatetimeIndex.
exog	DataFrame	Future exogenous values covering the horizon (at least steps rows). Used only in prediction mode, and required there when the plan uses the exogenous variables of the data (a plan with use_exog=False, such as the seasonal naive baseline, rejects it).
interval	list of float	Prediction interval as [lower, upper] quantiles (e.g. [0.1, 0.9] for an 80% interval). None disables intervals.
test_size	int, float, str, Timestamp	Size or start of the test set, selecting the mode above. int: last test_size observations; float in (0, 1): last fraction of observations; str/Timestamp: first timestamp of the test set. None runs prediction mode.

💡 Tip

You can also use custom configurations for the forecaster, estimator, estimator_kwargs, lags, and window_features by passing them as arguments to forecast(). Other decisions have their own argument: metric, use_exog, differentiation, calendar_features, target_transformer and dropna_from_series. This allows you to override the assistant's automatic choices and use your own preferred settings. For example, this code specifies a custom forecaster while still letting the assistant handle the rest of the workflow:

results = assistant.forecast(
              data        = data,
              target      = 'users',
              date_column = "date_time",
              steps       = 36,
              test_size   = 36,
              forecaster  = 'ForecasterFoundation'
          )

The following code demonstrates how to execute the assistant in evaluation mode. By explicitly defining the test_size parameter (in this case, test_size=36), the pipeline automatically reserves the final 36 observations as a test set.

During execution, the assistant profiles the dataset, incorporates the provided exogenous features (holiday, weather, and temp), and trains an appropriate model on the training split. Because the method is triggered in evaluation mode, the resulting ForecastResult object provides the out-of-sample predictions, an 80% prediction interval, and the performance metrics evaluated against the held-out ground truth.

In [6]:

# Execute the forecasting assistant in evaluation mode
# ==============================================================================
results = assistant.forecast(
              data        = data,
              target      = 'users',
              date_column = "date_time",
              steps       = 36,
              interval    = [0.1, 0.9],  # 80% prediction interval
              test_size   = 36           # Last 36 hours as test set (evaluation mode)
          )

# Execute the forecasting assistant in evaluation mode # ============================================================================== results = assistant.forecast( data = data, target = 'users', date_column = "date_time", steps = 36, interval = [0.1, 0.9], # 80% prediction interval test_size = 36 # Last 36 hours as test set (evaluation mode) )

In [7]:

# Inspect the performance metrics and the resulting predictions
# ==============================================================================
display(results.metrics)
display(results.predictions.head())

# Inspect the performance metrics and the resulting predictions # ============================================================================== display(results.metrics) display(results.predictions.head())

	series	MAE	MSE	MASE	MAPE
0	users	32.013374	2758.8615	0.497185	0.415688

	pred	lower_bound	upper_bound
2012-12-30 12:00:00	146.512223	111.097550	181.185404
2012-12-30 13:00:00	138.768717	96.950185	178.804105
2012-12-30 14:00:00	145.133323	92.701496	185.332563
2012-12-30 15:00:00	140.891323	89.525337	185.272888
2012-12-30 16:00:00	139.538959	86.994814	180.904345

The show_code() method allows users to retrieve the exact code used to generate the forecast.

In [8]:

# skforecast code that generated the results
# ==============================================================================
results.show_code()

# skforecast code that generated the results # ============================================================================== results.show_code()

Generated codeCopy

import pandas as pd                                                                       
from sklearn.metrics import mean_absolute_error, mean_squared_error,                      
mean_absolute_percentage_error                                                            
from skforecast.metrics import mean_absolute_scaled_error                                 
from lightgbm import LGBMRegressor                                                        
from skforecast.preprocessing import RollingFeatures, CalendarFeatures                    
from skforecast.recursive import ForecasterRecursive                                      
                                                                                          
# Load data                                                                               
data = pd.read_csv('data.csv')                                                            
                                                                                          
data['date_time'] = pd.to_datetime(data['date_time'])                                     
data = data.set_index('date_time')                                                        
data = data.asfreq('h')                                                                   
data = data.sort_index()                                                                  
                                                                                          
# Train/test split                                                                        
end_train = '2012-12-30 11:00:00'  # last training date, adjust to change the split point 
data_train = data.loc[:end_train]                                                         
data_test  = data.loc[data.index > end_train]                                             
exog_features = ['holiday', 'weather', 'temp']                                            
                                                                                          
print(                                                                                    
    f"Train dates : {data_train.index.min()} --- {data_train.index.max()}                 
(n={len(data_train)})"                                                                    
)                                                                                         
print(                                                                                    
    f"Test dates  : {data_test.index.min()} --- {data_test.index.max()}                   
(n={len(data_test)})"                                                                     
)                                                                                         
                                                                                          
window_features = RollingFeatures(                                                        
    stats        = ['mean', 'std', 'mean', 'mean'],                                       
    window_sizes = [3, 3, 24, 168],                                                       
)                                                                                         
                                                                                          
calendar_features = CalendarFeatures(                                                     
    features = ['hour', 'day_of_week', 'weekend', 'month'],                               
    encoding = None,                                                                      
)                                                                                         
                                                                                          
# Create forecaster                                                                       
forecaster = ForecasterRecursive(                                                         
    estimator            = LGBMRegressor(random_state=123, verbose=-1),                   
    lags                 = [1, 2, 3, 5, 8, 10, 15, 17, 19, 20, 21, 22, 23, 24, 25, 26, 32,
33, 119, 121, 135, 136, 142, 143, 145, 160, 166, 167, 169, 313, 335, 337],                
    window_features      = window_features,                                               
    calendar_features    = calendar_features,                                             
    categorical_features = 'auto',                                                        
    dropna_from_series   = False,                                                         
)                                                                                         
                                                                                          
# Fit                                                                                     
forecaster.fit(                                                                           
    y                         = data_train['users'],                                      
    exog                      = data_train[exog_features],                                
    store_in_sample_residuals = True,                                                     
)                                                                                         
                                                                                          
# Predict intervals                                                                       
steps = 36                                                                                
predictions = forecaster.predict_interval(                                                
    steps    = steps,                                                                     
    exog     = data_test[exog_features],                                                  
    method   = 'bootstrapping',                                                           
    interval = [0.1, 0.9],                                                                
)                                                                                         
print(predictions)                                                                        
                                                                                          
# Evaluate on test set                                                                    
actual = data_test['users'].iloc[:steps]                                                  
mae = mean_absolute_error(actual, predictions['pred'])                                    
mse = mean_squared_error(actual, predictions['pred'])                                     
mase = mean_absolute_scaled_error(                                                        
    y_true  = actual,                                                                     
    y_pred  = predictions['pred'],                                                        
    y_train = data_train['users'],                                                        
)                                                                                         
mape = mean_absolute_percentage_error(actual, predictions['pred'])                        
                                                                                          
print(f"MAE  : {mae:.4f}")                                                                
print(f"MSE  : {mse:.4f}")                                                                
print(f"MASE : {mase:.4f}")                                                               
print(f"MAPE : {mape:.4f}")                                                               
                                                                                          
# NOTE: This script uses a train/test split for demonstration purposes.                   
# For production forecasting, retrain with all available data                             
# and provide future exogenous values covering the forecast horizon.                      

To generate forecasts for true future periods (starting immediately after the last observation in the dataset), the assistant is executed in prediction mode. This mode is active by default when test_size=None. In this configuration, the pipeline trains the optimal model using the entire available dataset and projects the predictions forward by the specified number of steps. Because there is no holdout data to serve as ground truth, the resulting ForecastResult object does not compute performance metrics. Additionally, if the historical data relies on exogenous features, a DataFrame containing their future values spanning the entire forecast horizon must be explicitly supplied via the exog parameter.

In [9]:

# Forecast the next 36 hours using the entire dataset (prediction mode)
# ==============================================================================
# Simulate future values of exogenous variables for the next 36 hours
exog = data[['holiday', 'weather', 'temp']].tail(36).copy()
exog.index = pd.date_range(
    start=pd.to_datetime(data['date_time'].max()) + pd.Timedelta(1, unit='h'), periods=36, freq='h'
)

results_pred = assistant.forecast(
                   data        = data,
                   target      = 'users',
                   date_column = "date_time",
                   steps       = 36,
                   interval    = [0.1, 0.9],  # 80% prediction interval
                   test_size   = None,        # Use the entire dataset for training (prediction mode)
                   exog        = exog         # Future values of exogenous variables for the next 36 hours
               )

display(results_pred.predictions.head())

# Forecast the next 36 hours using the entire dataset (prediction mode) # ============================================================================== # Simulate future values of exogenous variables for the next 36 hours exog = data[['holiday', 'weather', 'temp']].tail(36).copy() exog.index = pd.date_range( start=pd.to_datetime(data['date_time'].max()) + pd.Timedelta(1, unit='h'), periods=36, freq='h' ) results_pred = assistant.forecast( data = data, target = 'users', date_column = "date_time", steps = 36, interval = [0.1, 0.9], # 80% prediction interval test_size = None, # Use the entire dataset for training (prediction mode) exog = exog # Future values of exogenous variables for the next 36 hours ) display(results_pred.predictions.head())

	pred	lower_bound	upper_bound
2013-01-01 00:00:00	27.442927	11.127479	47.557452
2013-01-01 01:00:00	13.318240	3.310496	27.242435
2013-01-01 02:00:00	9.208553	2.380079	19.120340
2013-01-01 03:00:00	5.329501	0.570137	10.013675
2013-01-01 04:00:00	5.979160	1.305255	10.265962

Both modes return a ForecastResult, a lightweight container that bundles everything the assistant used and produced, so you can inspect the outputs, audit the decisions, or lift the code straight into production.

Attribute	Type	Description
predictions	DataFrame	Forecasted values for the requested steps. When intervals (or quantiles) are requested, the bound columns are included alongside the point predictions.
metrics	DataFrame, None	Evaluation metrics (MAE, MSE, MASE, MAPE), one row per series. None in prediction mode, where there is no ground truth to score against.
code	str	The exact standalone skforecast script that produced the forecast, deterministic and ready to run on its own.
profile	ForecastingProfile	The data profile behind the forecast: metadata, summary statistics, detected frequency, significant lags, and the high-level modeling decisions.
plan	ForecastPlan	The detailed configuration that was executed: forecaster, estimator, lags, window features, preprocessing, and interval settings.

Displaying the object in a notebook renders a rich summary of all of the above; the raw script is also available through results.show_code().

In [10]:

# Full results object
# ==============================================================================
results_pred

# Full results object # ============================================================================== results_pred

Out[10]:

                          Dataset Profile                          
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Property       ┃ Value                                          ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Format         │ single                                         │
├────────────────┼────────────────────────────────────────────────┤
│ Series         │ 1                                              │
├────────────────┼────────────────────────────────────────────────┤
│ Observations   │ 17544                                          │
├────────────────┼────────────────────────────────────────────────┤
│ Frequency      │ h                                              │
├────────────────┼────────────────────────────────────────────────┤
│ Target         │ users                                          │
├────────────────┼────────────────────────────────────────────────┤
│ Exog columns   │ holiday, weather, temp  (categorical: weather) │
├────────────────┼────────────────────────────────────────────────┤
│ Missing values │ None                                           │
└────────────────┴────────────────────────────────────────────────┘

                                    Recommendation                                     
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Property              ┃ Value                                                       ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Task type             │ single_series                                               │
├───────────────────────┼─────────────────────────────────────────────────────────────┤
│ Forecaster            │ ForecasterRecursive                                         │
├───────────────────────┼─────────────────────────────────────────────────────────────┤
│ Forecaster candidates │ ForecasterRecursive, ForecasterDirect, ForecasterFoundation │
├───────────────────────┼─────────────────────────────────────────────────────────────┤
│ Estimator             │ LGBMRegressor                                               │
├───────────────────────┼─────────────────────────────────────────────────────────────┤
│ Estimator candidates  │ LGBMRegressor, XGBRegressor, Ridge                          │
└───────────────────────┴─────────────────────────────────────────────────────────────┘

╭───────────────────────────────── Profile Explanation ──────────────────────────────────╮
│                                                                                        │
│  A single-series ML forecaster (ForecasterRecursive) is recommended. Data: 17544       │
│  observations, 'h' frequency. Alternative forecasters: ['ForecasterDirect',            │
│  'ForecasterFoundation']. Estimator: LGBMRegressor. A gradient boosting model is       │
│  preferred for a dataset of this size (17544 observations). Alternative estimators:    │
│  ['XGBRegressor', 'Ridge']. 3 exogenous variables (1 categorical) available as         │
│  predictors.                                                                           │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯
                                      Forecast Plan                                       
┏━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Property          ┃ Value                                                              ┃
┡━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Task type         │ single_series                                                      │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Forecaster        │ ForecasterRecursive                                                │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Estimator         │ LGBMRegressor                                                      │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Steps             │ 36                                                                 │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Frequency         │ h                                                                  │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Lags              │ [1, 2, 3, 5, 8, 10, 15, 17, 19, 20, 21, 22, 23, 24, 25, 26, 32,    │
│                   │ 33, 119, 121, 135, 136, 142, 143, 145, 160, 166, 167, 169, 313,    │
│                   │ 335, 337]                                                          │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Window features   │ [{'stats': ['mean', 'std'], 'window_size': 3}, {'stats': ['mean'], │
│                   │ 'window_size': 24}, {'stats': ['mean'], 'window_size': 168}]       │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Calendar features │ ['hour', 'day_of_week', 'weekend', 'month'] (raw ordinal encoding) │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Use exog          │ True                                                               │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Interval          │ [0.1, 0.9]                                                         │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Interval method   │ bootstrapping                                                      │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Primary metric    │ mean_absolute_error                                                │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Preprocessing     │ 1 step                                                             │
└───────────────────┴────────────────────────────────────────────────────────────────────┘

                                   Preprocessing Steps                                    
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Step                    ┃ Reason                                                       ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ handle_categorical_exog │ Categorical exogenous variables detected: ['weather']. These │
│                         │ are handled automatically by skforecast                      │
│                         │ (categorical_features='auto').                               │
└─────────────────────────┴──────────────────────────────────────────────────────────────┘

╭─────────────────────────────────── Plan Explanation ───────────────────────────────────╮
│                                                                                        │
│  Plan: ForecasterRecursive + LGBMRegressor. Lags: [1, 2, 3, 5, 8, 10, 15, 17, 19, 20,  │
│  21, 22, 23, 24, 25, 26, 32, 33, 119, 121, 135, 136, 142, 143, 145, 160, 166, 167,     │
│  169, 313, 335, 337]. Window features: ['mean(window=3)', 'std(window=3)',             │
│  'mean(window=24)', 'mean(window=168)']. Calendar features: ['hour', 'day_of_week',    │
│  'weekend', 'month'] (raw ordinal encoding). Prediction intervals via bootstrapping.   │
│  Exogenous variables included. MAE is interpretable, robust to outliers, and works at  │
│  any scale.                                                                            │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯
                    Predictions (36 rows)                     
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃ Index               ┃     pred ┃ lower_bound ┃ upper_bound ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│ 2013-01-01 00:00:00 │  27.4429 │     11.1275 │     47.5575 │
├─────────────────────┼──────────┼─────────────┼─────────────┤
│ 2013-01-01 01:00:00 │  13.3182 │      3.3105 │     27.2424 │
├─────────────────────┼──────────┼─────────────┼─────────────┤
│ 2013-01-01 02:00:00 │   9.2086 │      2.3801 │     19.1203 │
├─────────────────────┼──────────┼─────────────┼─────────────┤
│ 2013-01-01 03:00:00 │   5.3295 │      0.5701 │     10.0137 │
├─────────────────────┼──────────┼─────────────┼─────────────┤
│ 2013-01-01 04:00:00 │   5.9792 │      1.3053 │     10.2660 │
├─────────────────────┼──────────┼─────────────┼─────────────┤
│ ...                 │      ... │         ... │         ... │
├─────────────────────┼──────────┼─────────────┼─────────────┤
│ 2013-01-02 07:00:00 │ 125.3912 │     77.8800 │    189.4169 │
├─────────────────────┼──────────┼─────────────┼─────────────┤
│ 2013-01-02 08:00:00 │ 253.6618 │    159.2711 │    365.0444 │
├─────────────────────┼──────────┼─────────────┼─────────────┤
│ 2013-01-02 09:00:00 │ 190.4495 │    133.4486 │    237.9817 │
├─────────────────────┼──────────┼─────────────┼─────────────┤
│ 2013-01-02 10:00:00 │ 132.0387 │    100.9730 │    173.2996 │
├─────────────────────┼──────────┼─────────────┼─────────────┤
│ 2013-01-02 11:00:00 │ 145.3529 │    110.8851 │    201.4766 │
└─────────────────────┴──────────┴─────────────┴─────────────┘

Generated codeCopy

import pandas as pd                                                                       
from lightgbm import LGBMRegressor                                                        
from skforecast.preprocessing import RollingFeatures, CalendarFeatures                    
from skforecast.recursive import ForecasterRecursive                                      
                                                                                          
# Load data                                                                               
data = pd.read_csv('data.csv')                                                            
                                                                                          
# Load future exogenous variables covering the forecast horizon                           
exog_future = pd.read_csv('exog_future.csv')                                              
                                                                                          
data['date_time'] = pd.to_datetime(data['date_time'])                                     
data = data.set_index('date_time')                                                        
data = data.asfreq('h')                                                                   
data = data.sort_index()                                                                  
                                                                                          
exog_future['date_time'] = pd.to_datetime(exog_future['date_time'])                       
exog_future = exog_future.set_index('date_time')                                          
exog_future = exog_future.asfreq('h')                                                     
exog_future = exog_future.sort_index()                                                    
                                                                                          
exog_features = ['holiday', 'weather', 'temp']                                            
                                                                                          
window_features = RollingFeatures(                                                        
    stats        = ['mean', 'std', 'mean', 'mean'],                                       
    window_sizes = [3, 3, 24, 168],                                                       
)                                                                                         
                                                                                          
calendar_features = CalendarFeatures(                                                     
    features = ['hour', 'day_of_week', 'weekend', 'month'],                               
    encoding = None,                                                                      
)                                                                                         
                                                                                          
# Create forecaster                                                                       
forecaster = ForecasterRecursive(                                                         
    estimator            = LGBMRegressor(random_state=123, verbose=-1),                   
    lags                 = [1, 2, 3, 5, 8, 10, 15, 17, 19, 20, 21, 22, 23, 24, 25, 26, 32,
33, 119, 121, 135, 136, 142, 143, 145, 160, 166, 167, 169, 313, 335, 337],                
    window_features      = window_features,                                               
    calendar_features    = calendar_features,                                             
    categorical_features = 'auto',                                                        
    dropna_from_series   = False,                                                         
)                                                                                         
                                                                                          
# Fit                                                                                     
forecaster.fit(                                                                           
    y                         = data['users'],                                            
    exog                      = data[exog_features],                                      
    store_in_sample_residuals = True,                                                     
)                                                                                         
                                                                                          
# Predict intervals                                                                       
steps = 36                                                                                
predictions = forecaster.predict_interval(                                                
    steps    = steps,                                                                     
    exog     = exog_future[exog_features],                                                
    method   = 'bootstrapping',                                                           
    interval = [0.1, 0.9],                                                                
)                                                                                         
print(predictions)                                                                        

Interpreting results¶

After executing the workflow, you can leverage the LLM reasoning layer to interpret the outputs. By passing the ForecastResult object to the ask() method as context, the assistant analyzes the generated predictions, performance metrics, and model configuration. This allows you to quickly extract actionable insights (such as contextualizing specific error metrics, identifying predicted trends, or diagnosing model behavior) directly in natural language.

Each call briefs the model with a set of skills, curated skforecast documents chosen from the context and the wording of the question, so the answer follows the current skforecast API. See Skills for the LLM for the list and how to choose them yourself.

In [11]:

# Asking the assistant about the forecast results
# ==============================================================================
explanation = assistant.ask(
    prompt = "Explain the results of the forecast, including the metrics and predictions.",
    context = results
)

# Asking the assistant about the forecast results # ============================================================================== explanation = assistant.ask( prompt = "Explain the results of the forecast, including the metrics and predictions.", context = results )

/var/folders/wt/8tvn563d5v55nspfbydgqb9r0000gp/T/ipykernel_70978/2239136354.py:3: DataSentToLLMWarning: `send_data_to_llm=False` does not apply to `context`: the predicted values it carries are sent to the LLM, because a question about a result cannot be answered from summary statistics alone. Your input data is not sent: a result holds only the model's output, never the data it was fitted on. To keep predictions local, ask without `context`.
  explanation = assistant.ask(

In [12]:

# Show the explanation generated by the LLM
# ==============================================================================
explanation.show_explanation()

# Show the explanation generated by the LLM # ============================================================================== explanation.show_explanation()

╭────────────────────────────────── Assistant Response ──────────────────────────────────╮
│                                                                                        │
│  The forecast evaluated a 36-step horizon for the users series using                   │
│  ForecasterRecursive with LGBMRegressor. Across the evaluation, the model achieved a   │
│  Mean Absolute Error (MAE) of 32.013374 and a Mean Absolute Scaled Error (MASE) of     │
│  0.497185. The resulting predictions span 36 hourly timestamps with an 80% prediction  │
│  interval generated via bootstrapping.                                                 │
│                                                                                        │
│  Evaluation Metrics                                                                    │
│                                                                                        │
│  The model evaluation produced the following metrics:                                  │
│                                                                                        │
│   • MAE: 32.013374                                                                     │
│   • MSE: 2758.8615                                                                     │
│   • MASE: 0.497185 (because MASE is below 1, it beats the one-step naive forecast      │
│     (last training value))                                                             │
│   • MAPE: 0.415688 (MAPE is a percentage and becomes unreliable as the target          │
│     approaches zero)                                                                   │
│                                                                                        │
│  Forecast Predictions                                                                  │
│                                                                                        │
│  The forecast covers 36 hourly steps from 2012-12-30 12:00:00 to 2012-12-31 23:00:00,  │
│  including point predictions and 80% prediction intervals ([0.1, 0.9]) generated with  │
│  bootstrapping.                                                                        │
│                                                                                        │
│  Summary across all 36 forecast steps:                                                 │
│                                                                                        │
│   • Point predictions (pred): minimum of 5.258729928204422, maximum of                 │
│     260.7711523801768, and mean of 103.19879814283314.                                 │
│   • Lower bound: minimum of 0.44880061218456635, maximum of 142.57459533864323, and    │
│     mean of 61.74293153975548.                                                         │
│   • Upper bound: minimum of 12.096545115409352, maximum of 352.6076131902337, and      │
│     mean of 137.95308075514998.                                                        │
│                                                                                        │
│  Sample values from the available output rows (26 intermediate rows omitted):          │
│                                                                                        │
│   • 2012-12-30 12:00:00: prediction of 146.512223, lower bound of 111.097550, upper    │
│     bound of 181.185404.                                                               │
│   • 2012-12-30 13:00:00: prediction of 138.768717, lower bound of 96.950185, upper     │
│     bound of 178.804105.                                                               │
│   • 2012-12-30 14:00:00: prediction of 145.133323, lower bound of 92.701496, upper     │
│     bound of 185.332563.                                                               │
│   • 2012-12-30 15:00:00: prediction of 140.891323, lower bound of 89.525337, upper     │
│     bound of 185.272888.                                                               │
│   • 2012-12-30 16:00:00: prediction of 139.538959, lower bound of 86.994814, upper     │
│     bound of 180.904345.                                                               │
│   • 2012-12-31 19:00:00: prediction of 141.776795, lower bound of 99.460841, upper     │
│     bound of 200.731831.                                                               │
│   • 2012-12-31 20:00:00: prediction of 113.839137, lower bound of 61.893890, upper     │
│     bound of 153.044573.                                                               │
│   • 2012-12-31 21:00:00: prediction of 76.905230, lower bound of 40.671154, upper      │
│     bound of 107.208312.                                                               │
│   • 2012-12-31 22:00:00: prediction of 56.118157, lower bound of 28.809600, upper      │
│     bound of 83.006316.                                                                │
│   • 2012-12-31 23:00:00: prediction of 39.970777, lower bound of 17.809739, upper      │
│     bound of 59.555964.                                                                │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯

Refining the modeling plan¶

Beyond explaining results, the assistant can also help you to improve the modeling strategy using the refine_plan() method. This method accepts an existing profile and plan and returns an updated ForecastPlan. It operates in two distinct modes:

• Deterministic mode (Explicit Overrides): When no prompt is provided, you can pass explicit configuration overrides (such as forecaster, estimator, estimator_kwargs, steps, interval, lags, or window_features). Only the explicitly specified fields are updated; the remaining configuration is deterministically re-derived from the original plan.

• LLM mode (Domain Knowledge Integration): When a prompt is provided, the assistant leverages the LLM to interpret your natural-language domain knowledge. The agent suggests domain-specific lags and window_features, which are then merged into the plan. To maintain strict interpretability, the LLM’s reasoning is appended to the returned plan's explanation attribute, allowing you to trace exactly why each feature was proposed. It applies to the machine learning forecasters: ForecasterStats, ForecasterFoundation and ForecasterEquivalentDate use no lags or window features, so for them the prompt is ignored with a warning and the LLM is not called.

The following example demonstrates the LLM mode. By supplying a prompt with context about the dataset, we instruct the assistant to engineer more appropriate lags and window features. We then execute forecast() using the refined plan and compare the new metrics against our original baseline run.

Keep in mind that a refined plan is a hypothesis, not a guaranteed improvement. The LLM proposes lags and window features from the domain knowledge you provide, but only a proper evaluation can confirm whether they actually help. Always compare the refined plan against the original before adopting it, and check that the suggested features are sensible for your data and use case rather than trusting them blindly.

In [13]:

# Ask the assistant to improve the forecast accuracy
# ==============================================================================
prompt  = (
    "I'm forecasting hourly bike rentals. Demand follows a clear daily rhythm with "
    "rush-hour peaks, and it changes between weekdays and weekends. It's also usually "
    "similar to what happened at the same time last week, and the last few hours give "
    "a good sense of the current trend. Please pick lags and rolling features that fit this."
)

refined_plan = assistant.refine_plan(
                   profile = results_pred.profile,
                   plan    = results_pred.plan,
                   prompt  = prompt
               )

# Ask the assistant to improve the forecast accuracy # ============================================================================== prompt = ( "I'm forecasting hourly bike rentals. Demand follows a clear daily rhythm with " "rush-hour peaks, and it changes between weekdays and weekends. It's also usually " "similar to what happened at the same time last week, and the last few hours give " "a good sense of the current trend. Please pick lags and rolling features that fit this." ) refined_plan = assistant.refine_plan( profile = results_pred.profile, plan = results_pred.plan, prompt = prompt )

In [14]:

# Refined plan proposed by the assistant
# ==============================================================================
refined_plan

# Refined plan proposed by the assistant # ============================================================================== refined_plan

Out[14]:

                                      Forecast Plan                                       
┏━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Property          ┃ Value                                                              ┃
┡━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Task type         │ single_series                                                      │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Forecaster        │ ForecasterRecursive                                                │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Estimator         │ LGBMRegressor                                                      │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Steps             │ 36                                                                 │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Frequency         │ h                                                                  │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Lags              │ [1, 2, 3, 4, 5, 6, 23, 24, 25, 47, 48, 49, 167, 168, 169]          │
│                   │ (LLM-suggested)                                                    │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Window features   │ [{'stats': ['mean', 'std'], 'window_size': 4}, {'stats': ['mean',  │
│                   │ 'max'], 'window_size': 24}, {'stats': ['mean'], 'window_size':     │
│                   │ 168}]  (LLM-suggested)                                             │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Calendar features │ ['hour', 'day_of_week', 'weekend', 'month'] (raw ordinal encoding) │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Use exog          │ True                                                               │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Interval          │ [0.1, 0.9]                                                         │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Interval method   │ bootstrapping                                                      │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Primary metric    │ mean_absolute_error                                                │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Preprocessing     │ 1 step                                                             │
└───────────────────┴────────────────────────────────────────────────────────────────────┘

                                   Preprocessing Steps                                    
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Step                    ┃ Reason                                                       ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ handle_categorical_exog │ Categorical exogenous variables detected: ['weather']. These │
│                         │ are handled automatically by skforecast                      │
│                         │ (categorical_features='auto').                               │
└─────────────────────────┴──────────────────────────────────────────────────────────────┘

╭─────────────────────────────────── Plan Explanation ───────────────────────────────────╮
│                                                                                        │
│  Plan: ForecasterRecursive + LGBMRegressor. Lags: [1, 2, 3, 4, 5, 6, 23, 24, 25, 47,   │
│  48, 49, 167, 168, 169]. Window features: ['mean(window=4)', 'std(window=4)',          │
│  'mean(window=24)', 'max(window=24)', 'mean(window=168)']. Calendar features:          │
│  ['hour', 'day_of_week', 'weekend', 'month'] (raw ordinal encoding). Prediction        │
│  intervals via bootstrapping. Exogenous variables included. MAE is interpretable,      │
│  robust to outliers, and works at any scale.                                           │
│                                                                                        │
│  LLM Refinement Reasoning: Selected lags and window features reflect the multi-scale   │
│  temporal dynamics of hourly bike rental demand:                                       │
│                                                                                        │
│   • Recent trend (1-6 hours): Immediate lags 1 to 6 and a 4-hour rolling window        │
│     ('mean', 'std') capture the evolving intra-day momentum and recent volatility.     │
│   • Daily rhythm & rush-hour patterns (24 and 48 hours): Lags 23, 24, 25 and 47, 48,   │
│     49 capture the strong 24-hour and 48-hour daily cycles along with adjacent         │
│     rush-hour shifts. A 24-hour rolling window ('mean', 'max') tracks the daily        │
│     average level and peak rush-hour capacity.                                         │
│   • Weekly cycle (168 hours): Lags 167, 168, and 169 capture the exact same time from  │
│     last week to differentiate weekday commute patterns from weekend recreation. A     │
│     168-hour rolling mean captures the overall weekly baseline demand.                 │
│                                                                                        │
│  Note: the LLM-suggested lags and window_features are hypotheses, not validated        │
│  improvements. Confirm any expected accuracy gain before relying on them.              │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯

In [15]:

# Run the forecast with the refined plan 
# ==============================================================================
results_refined = assistant.forecast(
                      data        = data,
                      target      = 'users',
                      date_column = "date_time",
                      steps       = 36,
                      test_size   = 36,           # Last 36 hours as test set (evaluation mode)
                      plan        = refined_plan  # Refined plan proposed by the assistant
                  )

# Run the forecast with the refined plan # ============================================================================== results_refined = assistant.forecast( data = data, target = 'users', date_column = "date_time", steps = 36, test_size = 36, # Last 36 hours as test set (evaluation mode) plan = refined_plan # Refined plan proposed by the assistant )

In [16]:

# Results of the forecast with the refined plan
# ==============================================================================
results_refined.metrics

# Results of the forecast with the refined plan # ============================================================================== results_refined.metrics

Out[16]:

	series	MAE	MSE	MASE	MAPE
0	users	50.866787	5302.616682	0.789989	0.540271

In [17]:

# Results of the forecast with the original plan
# ==============================================================================
results.metrics

# Results of the forecast with the original plan # ============================================================================== results.metrics

Out[17]:

	series	MAE	MSE	MASE	MAPE
0	users	32.013374	2758.8615	0.497185	0.415688

As noted above, a refined plan is a hypothesis, not a guaranteed improvement. The LLM may propose lags or window features that are not relevant to the series, or it may misread the domain knowledge you provided, so the refined plan can perform worse than the original.

More importantly, here we evaluate the change on a single 36-hour test window, which is far too small to draw a reliable conclusion. A single split can favor either plan by chance. To decide whether the refinement genuinely helps, evaluate both plans with a backtest over multiple folds (see the next section), which averages performance across many test windows and gives a far more trustworthy comparison.

Backtesting¶

In time series forecasting, backtesting is the process of evaluating a predictive model by retrospectively simulating its performance on historical data. It functions as a specialized form of temporal cross-validation, ensuring that data leakage is prevented by strictly respecting the chronological order of observations.

The reliability of backtesting metrics depends entirely on how closely the evaluation setup mirrors production conditions. To obtain trustworthy performance estimates, the backtesting strategy must accurately reflect the real-world prediction horizon, refit frequency, and data availability.

In skforecast-ai, this validation strategy is governed by the TimeSeriesFold object. It defines exactly how the historical data is partitioned into successive training and test windows. It controls critical parameters such as the forecast horizon (steps), the starting point for evaluation (initial_train_size), and whether the model is periodically retrained as the window rolls forward (refit). Because these choices dictate the interpretation of the resulting metrics, correctly configuring the TimeSeriesFold is the most critical decision in the evaluation phase. For a comprehensive overview of these mechanics, refer to the skforecast Backtesting user guide.

skforecast-ai provides three distinct ways to define this validation strategy, ranging from full manual control to automated generation:

• Explicit Instantiation (Recommended): Manually construct a TimeSeriesFold and pass it directly to the backtest() method. If you already know the exact operational constraints of your production environment, this is the most explicit and reproducible approach, keeping the validation setup entirely under your control.

• LLM Mode (create_cv() with a prompt): Describe your operational use case in natural language (e.g., required horizon, retraining frequency, trusted historical depth). The LLM reasoning layer interprets this context and translates it into a strictly defined TimeSeriesFold, materializing the setup into deterministic code.

• Deterministic Mode (create_cv() without a prompt): Allow the assistant to automatically derive a sensible TimeSeriesFold from the existing profile and plan using rule-based defaults. You can still explicitly override individual parameters such as initial_train_size or refit as needed.

The animation below shows the LLM mode with a deployment scenario: how the scenario becomes TimeSeriesFold parameters (the horizon always comes from the plan), and how backtest() then evaluates the model fold by fold, the way it will be used.

The following example demonstrates the three approaches.

In [18]:

# Explicit Instantiation of the cv
# ==============================================================================
# Create your own TimeSeriesFold object
end_train = '2012-08-31 23:59:00'
cv = TimeSeriesFold(
         steps              = 36, 
         initial_train_size = end_train, 
         refit              = False
     )

results_backtest = assistant.backtest(
                       data        = data,
                       target      = 'users',
                       date_column = "date_time",
                       cv          = cv,          # TimeSeriesFold, or a CVResult from create_cv()
                       interval    = [0.1, 0.9],  # 80% prediction interval
                   )

# Explicit Instantiation of the cv # ============================================================================== # Create your own TimeSeriesFold object end_train = '2012-08-31 23:59:00' cv = TimeSeriesFold( steps = 36, initial_train_size = end_train, refit = False ) results_backtest = assistant.backtest( data = data, target = 'users', date_column = "date_time", cv = cv, # TimeSeriesFold, or a CVResult from create_cv() interval = [0.1, 0.9], # 80% prediction interval )

  0%|          | 0/82 [00:00<?, ?it/s]

In [19]:

# Show backtest results
# ==============================================================================
results_backtest.show_explanation()
display(results_backtest.metrics)
display(results_backtest.predictions.head())

# Show backtest results # ============================================================================== results_backtest.show_explanation() display(results_backtest.metrics) display(results_backtest.predictions.head())

╭───────────────────────────────── Backtest Explanation ─────────────────────────────────╮
│                                                                                        │
│  Initial training up to 2012-08-31 23:59:00, trained once (no refit), 36-step          │
│  horizon, 82 folds. Results: mean_absolute_error: 46.3122, mean_squared_error:         │
│  5495.8816, mean_absolute_scaled_error: 0.7496, mean_absolute_percentage_error:        │
│  0.4727.                                                                               │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯

	mean_absolute_error	mean_squared_error	mean_absolute_scaled_error	mean_absolute_percentage_error
0	46.312218	5495.881638	0.749598	0.472735

	fold	pred	lower_bound	upper_bound
2012-09-01 00:00:00	0	124.163668	99.669766	149.384335
2012-09-01 01:00:00	0	96.989454	68.523139	127.651105
2012-09-01 02:00:00	0	65.894182	32.969786	97.883129
2012-09-01 03:00:00	0	29.961028	10.347714	60.808462
2012-09-01 04:00:00	0	9.103018	2.980393	28.085362

After running the workflow, you can ask the assistant to explain the backtesting results. The ask() method can take a BacktestResult object as context and provide insights about the strategy, the metrics, or any other relevant information.

In [20]:

# Asking the assistant about the backtesting
# ==============================================================================
explanation = assistant.ask(
    prompt = "Explain the results of the backtesting, including the strategy, metrics, and predictions.",
    context = results_backtest
)

# Asking the assistant about the backtesting # ============================================================================== explanation = assistant.ask( prompt = "Explain the results of the backtesting, including the strategy, metrics, and predictions.", context = results_backtest )

/var/folders/wt/8tvn563d5v55nspfbydgqb9r0000gp/T/ipykernel_70978/244763278.py:3: DataSentToLLMWarning: `send_data_to_llm=False` does not apply to `context`: the predicted values it carries are sent to the LLM, because a question about a result cannot be answered from summary statistics alone. Your input data is not sent: a result holds only the model's output, never the data it was fitted on. To keep predictions local, ask without `context`.
  explanation = assistant.ask(

In [21]:

# Show the explanation generated by the LLM
# ==============================================================================
explanation.show_explanation()

# Show the explanation generated by the LLM # ============================================================================== explanation.show_explanation()

╭────────────────────────────────── Assistant Response ──────────────────────────────────╮
│                                                                                        │
│  The backtesting evaluated the ForecasterRecursive model using an LGBMRegressor        │
│  across 82 evaluation folds with a 36-step forecast horizon and no refitting. The      │
│  evaluation achieved a mean absolute error of 46.312218 and a mean absolute scaled     │
│  error of 0.749598, beating the one-step naive forecast (last training value). An 80%  │
│  prediction interval was generated for all 2928 forecast steps using bootstrapping.    │
│                                                                                        │
│  Backtesting Strategy                                                                  │
│                                                                                        │
│  The cross-validation strategy was configured as follows:                              │
│                                                                                        │
│   • Forecast horizon (steps): 36                                                       │
│   • Initial training size: 2012-08-31 23:59:00                                         │
│   • Refit: False (the model was fitted once, n_fits: 1)                                │
│   • Fixed train size: True                                                             │
│   • Gap: 0                                                                             │
│   • Fold stride: 36                                                                    │
│   • Skip folds: None                                                                   │
│   • Allow incomplete fold: True                                                        │
│   • Differentiation: None                                                              │
│   • Total folds: 82                                                                    │
│                                                                                        │
│  Evaluation Metrics                                                                    │
│                                                                                        │
│  The deterministic evaluation over the backtesting period produced the following       │
│  error metrics:                                                                        │
│                                                                                        │
│   • mean_absolute_error: 46.312218                                                     │
│   • mean_squared_error: 5495.881638                                                    │
│   • mean_absolute_scaled_error: 0.749598                                               │
│   • mean_absolute_percentage_error: 0.472735                                           │
│                                                                                        │
│  The mean absolute scaled error is 0.749598; because it is below 1, the model beats    │
│  the one-step naive forecast (last training value). The mean absolute percentage       │
│  error is 0.472735; note that MAPE is a percentage and becomes unreliable as the       │
│  target approaches zero.                                                               │
│                                                                                        │
│  Predictions Summary                                                                   │
│                                                                                        │
│  Backtesting produced 2928 total prediction rows across folds 0 through 81, including  │
│  point predictions and an 80% prediction interval ([0.1, 0.9]) generated via           │
│  bootstrapping. Across all 2928 rows, the summary statistics are:                      │
│                                                                                        │
│   • Point predictions (pred): min 1.9345238001281464, max 881.3578535350078, mean      │
│     235.999687947411                                                                   │
│   • Lower bound (lower_bound): min -2.1351855197660417, max 827.8249722414072, mean    │
│     193.18431772887837                                                                 │
│   • Upper bound (upper_bound): min 4.739552115733364, max 914.3772544676444, mean      │
│     283.5445453993996                                                                  │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯

Rather than manually configuring TimeSeriesFold parameters, you can describe your target backtesting strategy in natural language and allow the assistant's LLM layer to translate it into a rigorous cross-validation schema. Once you have generated a data profile and a modeling plan, simply pass a prompt detailing your operational constraints, such as the required forecast horizon, historical training depth, refitting frequency, or expected data gaps. The assistant processes this context and returns a fully configured TimeSeriesFold object, accompanied by an explicit explanation of its design choices. This transparency ensures you can thoroughly audit and verify the validation strategy before executing the backtest.

In [22]:

# Let the assistant create the cross-validation strategy for you
# ==============================================================================
# Profile and create a plan for your data
profile = assistant.profile(data, target="users", date_column="date_time")
plan = assistant.plan(profile, steps=36)

# Call create_cv with a prompt that describes your prediction strategy
prompt = (
    "I forecast bike demand 36 hours ahead. "
    "The model should be trained once on all data up to the end of August 2012, 23:59. "
    "Do not refit the model as the window rolls forward."
)
cv = assistant.create_cv(
         profile = profile,
         plan    = plan,
         prompt  = prompt
     )

# Let the assistant create the cross-validation strategy for you # ============================================================================== # Profile and create a plan for your data profile = assistant.profile(data, target="users", date_column="date_time") plan = assistant.plan(profile, steps=36) # Call create_cv with a prompt that describes your prediction strategy prompt = ( "I forecast bike demand 36 hours ahead. " "The model should be trained once on all data up to the end of August 2012, 23:59. " "Do not refit the model as the window rolls forward." ) cv = assistant.create_cv( profile = profile, plan = plan, prompt = prompt )

In [23]:

# CVResult object: the TimeSeriesFold, its explanation and the code that rebuilds it
# ==============================================================================
cv

# CVResult object: the TimeSeriesFold, its explanation and the code that rebuilds it # ============================================================================== cv

Out[23]:

╭───────────────────────────── Cross-Validation Explanation ─────────────────────────────╮
│                                                                                        │
│  The model is trained once on all historical data up to the end of August 2012         │
│  ('2012-08-31 23:00:00'). Training occurs only once without retraining across          │
│  subsequent evaluation folds (refit=False), evaluating 36-hour forecasts across the    │
│  remaining four months of data (yielding well over 2 folds). Initial training up to    │
│  2012-08-31 23:00:00, trained once (no refit), 36-step horizon, 82 folds.              │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯
        Cross-Validation Configuration         
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃ Parameter             ┃               Value ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ steps                 │                  36 │
├───────────────────────┼─────────────────────┤
│ initial_train_size    │ 2012-08-31 23:00:00 │
├───────────────────────┼─────────────────────┤
│ refit                 │               False │
├───────────────────────┼─────────────────────┤
│ fixed_train_size      │               False │
├───────────────────────┼─────────────────────┤
│ gap                   │                   0 │
├───────────────────────┼─────────────────────┤
│ fold_stride           │                  36 │
├───────────────────────┼─────────────────────┤
│ skip_folds            │                None │
├───────────────────────┼─────────────────────┤
│ allow_incomplete_fold │                True │
├───────────────────────┼─────────────────────┤
│ differentiation       │                None │
├───────────────────────┼─────────────────────┤
│ n_folds               │                  82 │
├───────────────────────┼─────────────────────┤
│ n_fits                │                   1 │
└───────────────────────┴─────────────────────┘

Generated codeCopy

from skforecast.model_selection import TimeSeriesFold                                     
                                                                                          
# Time series cross-validation configuration                                              
cv = TimeSeriesFold(                                                                      
    steps              = 36,                                                              
    initial_train_size = '2012-08-31 23:00:00',                                           
    refit              = False,                                                           
)                                                                                         

In [24]:

# LLM Reasoning
# ==============================================================================
import textwrap
print(textwrap.fill(cv.explanation, width=88))

# LLM Reasoning # ============================================================================== import textwrap print(textwrap.fill(cv.explanation, width=88))

The model is trained once on all historical data up to the end of August 2012
('2012-08-31 23:00:00'). Training occurs only once without retraining across subsequent
evaluation folds (refit=False), evaluating 36-hour forecasts across the remaining four
months of data (yielding well over 2 folds). Initial training up to 2012-08-31 23:00:00,
trained once (no refit), 36-step horizon, 82 folds.

Since the prompt correctly describes the intended use case, the cv object returned by create_cv() is equivalent to the one we built manually above (the same 82 folds; the timestamp is aligned to the hourly frequency). However, it was derived from a natural-language description rather than explicit parameters. The returned CVResult carries the splitter as cv.cv and, as cv.explanation, a description of the choices the assistant made. This allows you to verify that the resulting TimeSeriesFold matches your intended strategy.

The backtesting workflow returns a BacktestResult, a lightweight container that bundles everything the assistant used and produced, so you can inspect the outputs, audit the decisions, or lift the code straight into production.

Attribute	Type	Description
predictions	DataFrame	Full out-of-sample backtest predictions across all folds. When intervals (or quantiles) are requested, the bound columns are included alongside the point predictions.
metrics	DataFrame	Backtesting metrics (MAE, MSE, MASE, MAPE) returned by skforecast, one row per series, computed across all folds.
cv_config	dict	The resolved TimeSeriesFold parameters (steps, initial train size, refit, gap, etc.), kept for full traceability of the validation strategy.
code	str	The exact standalone skforecast script that reproduces the backtesting workflow, deterministic and ready to run on its own.
explanation	str	A human-readable summary of the backtesting configuration and what the results mean.
profile	ForecastingProfile	The data profile behind the backtest: metadata, summary statistics, detected frequency, significant lags, and the high-level modeling decisions.
plan	ForecastPlan	The detailed configuration that was executed: forecaster, estimator, lags, window features, preprocessing, and interval settings.

Displaying the object renders a rich summary of all of the above; the raw script is also available through results_backtest.show_code().

In [25]:

# Full results object
# ==============================================================================
results_backtest

# Full results object # ============================================================================== results_backtest

Out[25]:

╭───────────────────────────────── Backtest Explanation ─────────────────────────────────╮
│                                                                                        │
│  Initial training up to 2012-08-31 23:59:00, trained once (no refit), 36-step          │
│  horizon, 82 folds. Results: mean_absolute_error: 46.3122, mean_squared_error:         │
│  5495.8816, mean_absolute_scaled_error: 0.7496, mean_absolute_percentage_error:        │
│  0.4727.                                                                               │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯
        Cross-Validation Configuration         
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃ Parameter             ┃               Value ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ steps                 │                  36 │
├───────────────────────┼─────────────────────┤
│ initial_train_size    │ 2012-08-31 23:59:00 │
├───────────────────────┼─────────────────────┤
│ refit                 │               False │
├───────────────────────┼─────────────────────┤
│ fixed_train_size      │                True │
├───────────────────────┼─────────────────────┤
│ gap                   │                   0 │
├───────────────────────┼─────────────────────┤
│ fold_stride           │                  36 │
├───────────────────────┼─────────────────────┤
│ skip_folds            │                None │
├───────────────────────┼─────────────────────┤
│ allow_incomplete_fold │                True │
├───────────────────────┼─────────────────────┤
│ differentiation       │                None │
├───────────────────────┼─────────────────────┤
│ n_folds               │                  82 │
├───────────────────────┼─────────────────────┤
│ n_fits                │                   1 │
└───────────────────────┴─────────────────────┘
                                     Backtest Metrics                                     
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┓
┃ mean_absolute_error ┃ mean_squared_error ┃ mean_absolute_scale… ┃ mean_absolute_perce… ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━┩
│             46.3122 │          5495.8816 │               0.7496 │               0.4727 │
└─────────────────────┴────────────────────┴──────────────────────┴──────────────────────┘
                    Backtest Predictions (2928 rows)                    
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃ Index               ┃    fold ┃     pred ┃ lower_bound ┃ upper_bound ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│ 2012-09-01 00:00:00 │  0.0000 │ 124.1637 │     99.6698 │    149.3843 │
├─────────────────────┼─────────┼──────────┼─────────────┼─────────────┤
│ 2012-09-01 01:00:00 │  0.0000 │  96.9895 │     68.5231 │    127.6511 │
├─────────────────────┼─────────┼──────────┼─────────────┼─────────────┤
│ 2012-09-01 02:00:00 │  0.0000 │  65.8942 │     32.9698 │     97.8831 │
├─────────────────────┼─────────┼──────────┼─────────────┼─────────────┤
│ 2012-09-01 03:00:00 │  0.0000 │  29.9610 │     10.3477 │     60.8085 │
├─────────────────────┼─────────┼──────────┼─────────────┼─────────────┤
│ 2012-09-01 04:00:00 │  0.0000 │   9.1030 │      2.9804 │     28.0854 │
├─────────────────────┼─────────┼──────────┼─────────────┼─────────────┤
│ ...                 │     ... │      ... │         ... │         ... │
├─────────────────────┼─────────┼──────────┼─────────────┼─────────────┤
│ 2012-12-31 19:00:00 │ 81.0000 │ 145.8848 │     97.1293 │    180.5958 │
├─────────────────────┼─────────┼──────────┼─────────────┼─────────────┤
│ 2012-12-31 20:00:00 │ 81.0000 │ 103.3326 │     67.3546 │    140.2631 │
├─────────────────────┼─────────┼──────────┼─────────────┼─────────────┤
│ 2012-12-31 21:00:00 │ 81.0000 │  64.1944 │     33.2954 │     86.5947 │
├─────────────────────┼─────────┼──────────┼─────────────┼─────────────┤
│ 2012-12-31 22:00:00 │ 81.0000 │  46.9473 │     23.0095 │     64.8209 │
├─────────────────────┼─────────┼──────────┼─────────────┼─────────────┤
│ 2012-12-31 23:00:00 │ 81.0000 │  30.5361 │     12.3134 │     45.8523 │
└─────────────────────┴─────────┴──────────┴─────────────┴─────────────┘
                          Dataset Profile                          
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Property       ┃ Value                                          ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Format         │ single                                         │
├────────────────┼────────────────────────────────────────────────┤
│ Series         │ 1                                              │
├────────────────┼────────────────────────────────────────────────┤
│ Observations   │ 17544                                          │
├────────────────┼────────────────────────────────────────────────┤
│ Frequency      │ h                                              │
├────────────────┼────────────────────────────────────────────────┤
│ Target         │ users                                          │
├────────────────┼────────────────────────────────────────────────┤
│ Exog columns   │ holiday, weather, temp  (categorical: weather) │
├────────────────┼────────────────────────────────────────────────┤
│ Missing values │ None                                           │
└────────────────┴────────────────────────────────────────────────┘

                                    Recommendation                                     
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Property              ┃ Value                                                       ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Task type             │ single_series                                               │
├───────────────────────┼─────────────────────────────────────────────────────────────┤
│ Forecaster            │ ForecasterRecursive                                         │
├───────────────────────┼─────────────────────────────────────────────────────────────┤
│ Forecaster candidates │ ForecasterRecursive, ForecasterDirect, ForecasterFoundation │
├───────────────────────┼─────────────────────────────────────────────────────────────┤
│ Estimator             │ LGBMRegressor                                               │
├───────────────────────┼─────────────────────────────────────────────────────────────┤
│ Estimator candidates  │ LGBMRegressor, XGBRegressor, Ridge                          │
└───────────────────────┴─────────────────────────────────────────────────────────────┘

╭───────────────────────────────── Profile Explanation ──────────────────────────────────╮
│                                                                                        │
│  A single-series ML forecaster (ForecasterRecursive) is recommended. Data: 17544       │
│  observations, 'h' frequency. Alternative forecasters: ['ForecasterDirect',            │
│  'ForecasterFoundation']. Estimator: LGBMRegressor. A gradient boosting model is       │
│  preferred for a dataset of this size (17544 observations). Alternative estimators:    │
│  ['XGBRegressor', 'Ridge']. 3 exogenous variables (1 categorical) available as         │
│  predictors.                                                                           │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯
                                      Forecast Plan                                       
┏━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Property          ┃ Value                                                              ┃
┡━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Task type         │ single_series                                                      │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Forecaster        │ ForecasterRecursive                                                │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Estimator         │ LGBMRegressor                                                      │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Steps             │ 36                                                                 │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Frequency         │ h                                                                  │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Lags              │ [1, 2, 3, 5, 8, 10, 15, 17, 19, 20, 21, 22, 23, 24, 25, 26, 32,    │
│                   │ 33, 119, 121, 135, 136, 142, 143, 145, 160, 166, 167, 169, 313,    │
│                   │ 335, 337]                                                          │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Window features   │ [{'stats': ['mean', 'std'], 'window_size': 3}, {'stats': ['mean'], │
│                   │ 'window_size': 24}, {'stats': ['mean'], 'window_size': 168}]       │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Calendar features │ ['hour', 'day_of_week', 'weekend', 'month'] (raw ordinal encoding) │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Use exog          │ True                                                               │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Interval          │ [0.1, 0.9]                                                         │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Interval method   │ bootstrapping                                                      │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Primary metric    │ mean_absolute_error                                                │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Preprocessing     │ 1 step                                                             │
└───────────────────┴────────────────────────────────────────────────────────────────────┘

                                   Preprocessing Steps                                    
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Step                    ┃ Reason                                                       ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ handle_categorical_exog │ Categorical exogenous variables detected: ['weather']. These │
│                         │ are handled automatically by skforecast                      │
│                         │ (categorical_features='auto').                               │
└─────────────────────────┴──────────────────────────────────────────────────────────────┘

╭─────────────────────────────────── Plan Explanation ───────────────────────────────────╮
│                                                                                        │
│  Plan: ForecasterRecursive + LGBMRegressor. Lags: [1, 2, 3, 5, 8, 10, 15, 17, 19, 20,  │
│  21, 22, 23, 24, 25, 26, 32, 33, 119, 121, 135, 136, 142, 143, 145, 160, 166, 167,     │
│  169, 313, 335, 337]. Window features: ['mean(window=3)', 'std(window=3)',             │
│  'mean(window=24)', 'mean(window=168)']. Calendar features: ['hour', 'day_of_week',    │
│  'weekend', 'month'] (raw ordinal encoding). Prediction intervals via bootstrapping.   │
│  Exogenous variables included. MAE is interpretable, robust to outliers, and works at  │
│  any scale.                                                                            │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯

Generated codeCopy

import pandas as pd                                                                       
from lightgbm import LGBMRegressor                                                        
from skforecast.preprocessing import RollingFeatures, CalendarFeatures                    
from skforecast.recursive import ForecasterRecursive                                      
from skforecast.model_selection import TimeSeriesFold, backtesting_forecaster             
                                                                                          
# Load data                                                                               
data = pd.read_csv('data.csv')                                                            
                                                                                          
data['date_time'] = pd.to_datetime(data['date_time'])                                     
data = data.set_index('date_time')                                                        
data = data.asfreq('h')                                                                   
data = data.sort_index()                                                                  
                                                                                          
window_features = RollingFeatures(                                                        
    stats        = ['mean', 'std', 'mean', 'mean'],                                       
    window_sizes = [3, 3, 24, 168],                                                       
)                                                                                         
                                                                                          
calendar_features = CalendarFeatures(                                                     
    features = ['hour', 'day_of_week', 'weekend', 'month'],                               
    encoding = None,                                                                      
)                                                                                         
                                                                                          
# Create forecaster                                                                       
forecaster = ForecasterRecursive(                                                         
    estimator            = LGBMRegressor(random_state=123, verbose=-1),                   
    lags                 = [1, 2, 3, 5, 8, 10, 15, 17, 19, 20, 21, 22, 23, 24, 25, 26, 32,
33, 119, 121, 135, 136, 142, 143, 145, 160, 166, 167, 169, 313, 335, 337],                
    window_features      = window_features,                                               
    calendar_features    = calendar_features,                                             
    categorical_features = 'auto',                                                        
    dropna_from_series   = False,                                                         
)                                                                                         
                                                                                          
# Time series cross-validation configuration                                              
cv = TimeSeriesFold(                                                                      
    steps              = 36,                                                              
    initial_train_size = '2012-08-31 23:59:00',                                           
    refit              = False,                                                           
)                                                                                         
                                                                                          
# Run backtesting                                                                         
exog_features = ['holiday', 'weather', 'temp']                                            
                                                                                          
metrics, predictions = backtesting_forecaster(                                            
    forecaster        = forecaster,                                                       
    y                 = data['users'],                                                    
    exog              = data[exog_features],                                              
    cv                = cv,                                                               
    metric            = ['mean_absolute_error', 'mean_squared_error',                     
'mean_absolute_scaled_error', 'mean_absolute_percentage_error'],                          
    interval          = [0.1, 0.9],                                                       
    n_jobs            = 'auto',                                                           
    verbose           = False,                                                            
    show_progress     = True,                                                             
    suppress_warnings = True,                                                             
)                                                                                         
                                                                                          
print(metrics)                                                                            
print(predictions.head())                                                                 

Comparing forecaster configurations¶

Choosing a forecasting model should not rely on intuition alone. Two configurations that look equally reasonable can perform very differently once evaluated on real temporal data. The most reliable approach is to test every candidate under identical conditions and compare their metrics.

The compare() method does exactly that. It receives a list of candidate configurations, backtests each one using the same TimeSeriesFold strategy, and returns a leaderboard ranked by the selected metric.

Candidates can be provided in two ways:

• Automatic candidates (candidates=None): the assistant builds the comparison set from the profile.forecaster_candidates that belong to the same family as the recommended forecaster, so every row is scored on the same thing. For multi-series data, a multi-series forecaster is never ranked against a multivariate one: ForecasterRecursiveMultiSeries is compared with ForecasterFoundation. This is useful when exploring a new dataset without a predefined shortlist. When cv is the CVResult of create_cv() for a plan that is none of those candidates (a plan refined with another estimator or other lags, for example), that plan runs too, as the row 'Strategy plan', and the summary names it.

• Explicit candidates (recommended): pass a list of (name, config) tuples, where name labels the row in the leaderboard and config holds the same override keys understood by plan(): 'forecaster', 'estimator', 'estimator_kwargs', 'lags', 'window_features', 'use_exog', 'differentiation', 'calendar_features', 'target_transformer' and 'dropna_from_series'. This provides full control and makes the resulting table easier to interpret.

For a single series, compare() also adds a seasonal naive baseline (ForecasterEquivalentDate, which repeats the value of one seasonal period earlier, or the last value when the frequency has none) as one more row, ranked like the others, so the leaderboard shows whether a model beats the simplest reasonable forecast. Pass baseline=False to leave it out.

A failed candidate does not stop the comparison. Instead, a CandidateFailedWarning is issued, the row records the error and is placed last.

The animation below shows automatic candidates with the strategy of the backtesting example: the three configurations the profile proposes and the seasonal naive baseline run through the same folds, the leaderboard is a plain sort of the metric, and the winner goes straight into forecast(). Finally, ask() explains the ranking without changing it.

The following examples demonstrate both approaches.

💡 Tip

All candidates use the same cross-validation strategy, ensuring a fair comparison. However, the results are only meaningful if the cv setup reflects the real use case where the model will be deployed. For example, if the production system retrains weekly, the backtest should also refit weekly. If the model is expected to forecast 24 hours ahead, the backtest should use a 24-hour horizon.

The evaluation window must also be representative. A period that is too short or dominated by unusual events (holidays, outages, or exceptional peaks) may favor a candidate that performs poorly over time. Define the validation setup carefully before comparing models so the final ranking is reliable.

In [26]:

# Comparison of the forecaster candidates suggested by the profile
# ==============================================================================
end_train = '2012-08-31 23:59:00'
cv = TimeSeriesFold(
         steps              = 36,
         initial_train_size = end_train,
         refit              = False,
     )

results_compare = assistant.compare(
                      data        = data,
                      target      = 'users',
                      date_column = "date_time",
                      cv          = cv,   # Same TimeSeriesFold for every candidate
                      candidates  = None  # Candidates suggested by the assistant 
                  )

# Comparison of the forecaster candidates suggested by the profile # ============================================================================== end_train = '2012-08-31 23:59:00' cv = TimeSeriesFold( steps = 36, initial_train_size = end_train, refit = False, ) results_compare = assistant.compare( data = data, target = 'users', date_column = "date_time", cv = cv, # Same TimeSeriesFold for every candidate candidates = None # Candidates suggested by the assistant )

Comparing forecasters:   0%|          | 0/4 [00:00<?, ?it/s]

Loading weights:   0%|          | 0/92 [00:00<?, ?it/s]

In [27]:

# Ranked leaderboard
# ==============================================================================
results_compare.results

# Ranked leaderboard # ============================================================================== results_compare.results

Out[27]:

	rank	name	forecaster	estimator	mean_absolute_error	mean_squared_error	mean_absolute_scaled_error	mean_absolute_percentage_error
0	1	ForecasterFoundation	ForecasterFoundation	autogluon/chronos-2-small	38.436159	4231.325965	0.597467	0.611349
1	2	ForecasterRecursive	ForecasterRecursive	LGBMRegressor	46.312218	5495.881638	0.749598	0.472735
2	3	ForecasterDirect	ForecasterDirect	LGBMRegressor	49.298312	5936.622526	0.797930	0.499413
3	4	Baseline (seasonal naive)	ForecasterEquivalentDate	None	91.668716	22652.390710	1.503712	0.912133

In [28]:

# Deterministic summary of the comparison
# ==============================================================================
results_compare.show_explanation()

# Deterministic summary of the comparison # ============================================================================== results_compare.show_explanation()

╭──────────────────────────────── Comparison Explanation ────────────────────────────────╮
│                                                                                        │
│  Compared 4 configurations, ranked ascending by mean_absolute_error. Shared            │
│  cross-validation strategy: Initial training up to 2012-08-31 23:59:00, trained once   │
│  (no refit), 36-step horizon, 82 folds. ForecasterFoundation is not trained: the       │
│  window and refit settings do not apply to it, each fold forecasts from the            │
│  observations before it. Best: 'ForecasterFoundation' (ForecasterFoundation /          │
│  autogluon/chronos-2-small) = 38.4362, 17.0% ahead of 'ForecasterRecursive'            │
│  (46.3122). All 3 configurations beat the baseline 'Baseline (seasonal naive)'         │
│  (91.6687). The best, 'ForecasterFoundation', beats it by 58.1%.                       │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯

In practice, you will often already have a shortlist in mind: a fast baseline, a gradient boosting model, or a variant with a richer feature set. Passing explicit (name, config) tuples keeps the comparison focused and makes the resulting leaderboard easy to understand at a glance.

The config dictionary accepts the same overrides as plan(). Any omitted option falls back to the deterministic recommendation derived from the dataset profile, so candidates can remain concise. For example, {'forecaster': 'ForecasterDirect'} changes only the forecaster while keeping the recommended estimator, lags, and features.

⚠️ Computational cost

Each candidate is backtested independently across all folds, so runtime increases with both the number and complexity of the configurations. Comparing four candidates will take roughly four times as long as running one backtest.

Start with a small set of clearly different options, review the results, and refine from there. Testing many near-identical variants is costly and rarely useful.

In [29]:

# Comparison of an explicit shortlist of configurations
# ==============================================================================
candidates = [
    (
        "ridge_baseline",
        {
            "forecaster": "ForecasterRecursive",
            "estimator" : "Ridge",
            "lags"      : 24,
        }
    ),
    (
        "lgbm_daily_lags",
        {
            "forecaster": "ForecasterRecursive",
            "estimator" : "LGBMRegressor",
            "lags"      : 24,
        }
    ),
    (
        "lgbm_weekly_lags_rolling",
        {
            "forecaster"      : "ForecasterRecursive",
            "estimator"       : "LGBMRegressor",
            "lags"            : [1, 2, 3, 23, 24, 25, 167, 168, 169],
            "window_features" : [{"stats": ["mean", "std"], "window_size": 24}],
        }
    ),
    (
        "lgbm_direct",
        {
            "forecaster": "ForecasterDirect",
            "estimator" : "LGBMRegressor",
            "lags"      : 24,
        }
    ),
    (
        "foundation_model",
        {
            "forecaster": "ForecasterFoundation"
        }
    ),
]

results_compare = assistant.compare(
                      data        = data,
                      target      = 'users',
                      date_column = "date_time",
                      cv          = cv,
                      candidates  = candidates,  # Specific candidates to compare
                      metric      = ['mean_absolute_error', 'mean_absolute_scaled_error'],
                  )

# Comparison of an explicit shortlist of configurations # ============================================================================== candidates = [ ( "ridge_baseline", { "forecaster": "ForecasterRecursive", "estimator" : "Ridge", "lags" : 24, } ), ( "lgbm_daily_lags", { "forecaster": "ForecasterRecursive", "estimator" : "LGBMRegressor", "lags" : 24, } ), ( "lgbm_weekly_lags_rolling", { "forecaster" : "ForecasterRecursive", "estimator" : "LGBMRegressor", "lags" : [1, 2, 3, 23, 24, 25, 167, 168, 169], "window_features" : [{"stats": ["mean", "std"], "window_size": 24}], } ), ( "lgbm_direct", { "forecaster": "ForecasterDirect", "estimator" : "LGBMRegressor", "lags" : 24, } ), ( "foundation_model", { "forecaster": "ForecasterFoundation" } ), ] results_compare = assistant.compare( data = data, target = 'users', date_column = "date_time", cv = cv, candidates = candidates, # Specific candidates to compare metric = ['mean_absolute_error', 'mean_absolute_scaled_error'], )

Comparing forecasters:   0%|          | 0/6 [00:00<?, ?it/s]

Loading weights:   0%|          | 0/92 [00:00<?, ?it/s]

When several metrics are requested, all of them are shown as columns but only the first one drives the ranking.

In [30]:

# Ranked leaderboard, sorted by the first metric requested
# ==============================================================================
results_compare.results

# Ranked leaderboard, sorted by the first metric requested # ============================================================================== results_compare.results

Out[30]:

	rank	name	forecaster	estimator	mean_absolute_error	mean_absolute_scaled_error
0	1	foundation_model	ForecasterFoundation	autogluon/chronos-2-small	38.436159	0.597467
1	2	lgbm_direct	ForecasterDirect	LGBMRegressor	50.224908	0.819189
2	3	lgbm_weekly_lags_rolling	ForecasterRecursive	LGBMRegressor	50.557647	0.824568
3	4	lgbm_daily_lags	ForecasterRecursive	LGBMRegressor	55.517689	0.905517
4	5	Baseline (seasonal naive)	ForecasterEquivalentDate	None	91.668716	1.503712
5	6	ridge_baseline	ForecasterRecursive	Ridge	93.145620	1.519244

The comparison returns a ComparisonResult, which groups the shared setup, ranked leaderboard, and individual backtests in a single object. Use it to inspect the results, audit the comparison, or reuse the winning candidate’s code in production.

Attribute	Type	Description
results	DataFrame	Ranked leaderboard, one row per candidate, sorted best to worst. Columns: rank, name, forecaster, estimator, the metric columns, and error when at least one candidate failed.
candidates	dict	Mapping of candidate name to the full BacktestResult object.
failures	dict	Mapping of candidate name to a CandidateFailure describing why it failed. Empty when every candidate succeeds.
ranking_metric	str	Name of the metric used to sort results.
cv_config	dict	Resolved TimeSeriesFold parameters plus the resulting n_folds, applied identically to every candidate.
profile	ForecastingProfile	The shared data profile behind every candidate.
explanation	str	Deterministic, human-readable summary of the comparison.
best_name	str	Name of the top-ranked candidate.
best_candidate	BacktestResult	Top-ranked candidate as a complete BacktestResult.
baseline_name	str, None	Name of the seasonal naive baseline row, or None when no baseline was added (multi-series data, baseline=False, or a target with missing values or missing timestamps).

Displaying the object renders a rich summary of the explanation and the leaderboard.

In [31]:

# Rich summary of the comparison
# ==============================================================================
results_compare

# Rich summary of the comparison # ============================================================================== results_compare

Out[31]:

╭──────────────────────────────── Comparison Explanation ────────────────────────────────╮
│                                                                                        │
│  Compared 6 configurations, ranked ascending by mean_absolute_error. Shared            │
│  cross-validation strategy: Initial training up to 2012-08-31 23:59:00, trained once   │
│  (no refit), 36-step horizon, 82 folds. ForecasterFoundation is not trained: the       │
│  window and refit settings do not apply to it, each fold forecasts from the            │
│  observations before it. Best: 'foundation_model' (ForecasterFoundation /              │
│  autogluon/chronos-2-small) = 38.4362, 23.5% ahead of 'lgbm_direct' (50.2249). 4 of 5  │
│  configurations beat the baseline 'Baseline (seasonal naive)' (91.6687);               │
│  'ridge_baseline' does not. The best, 'foundation_model', beats it by 58.1%.           │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯
                                    Comparison Results                                    
┏━━━━━━━┳━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Index ┃ rank ┃         name ┃   forecaster ┃    estimator ┃ mean_absol… ┃ mean_absolu… ┃
┡━━━━━━━╇━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ 0     │    1 │ foundation_… │ ForecasterF… │ autogluon/c… │     38.4362 │       0.5975 │
├───────┼──────┼──────────────┼──────────────┼──────────────┼─────────────┼──────────────┤
│ 1     │    2 │  lgbm_direct │ ForecasterD… │ LGBMRegress… │     50.2249 │       0.8192 │
├───────┼──────┼──────────────┼──────────────┼──────────────┼─────────────┼──────────────┤
│ 2     │    3 │ lgbm_weekly… │ ForecasterR… │ LGBMRegress… │     50.5576 │       0.8246 │
├───────┼──────┼──────────────┼──────────────┼──────────────┼─────────────┼──────────────┤
│ 3     │    4 │ lgbm_daily_… │ ForecasterR… │ LGBMRegress… │     55.5177 │       0.9055 │
├───────┼──────┼──────────────┼──────────────┼──────────────┼─────────────┼──────────────┤
│ 4     │    5 │     Baseline │ ForecasterE… │          N/A │     91.6687 │       1.5037 │
│       │      │    (seasonal │              │              │             │              │
│       │      │       naive) │              │              │             │              │
├───────┼──────┼──────────────┼──────────────┼──────────────┼─────────────┼──────────────┤
│ 5     │    6 │ ridge_basel… │ ForecasterR… │        Ridge │     93.1456 │       1.5192 │
└───────┴──────┴──────────────┴──────────────┴──────────────┴─────────────┴──────────────┘

Because every candidate is a full BacktestResult, the details of any individual configuration remain available, including its metrics, its predictions and the standalone script that generated them.

In [32]:

# Inspect a specific candidate
# ==============================================================================
candidate = results_compare.candidates['foundation_model']

display(candidate.metrics)
display(candidate.predictions.head())
candidate.show_code()

# Inspect a specific candidate # ============================================================================== candidate = results_compare.candidates['foundation_model'] display(candidate.metrics) display(candidate.predictions.head()) candidate.show_code()

	mean_absolute_error	mean_absolute_scaled_error
0	38.436159	0.597467

	level	fold	pred
2012-09-01 00:00:00	users	0	148.059464
2012-09-01 01:00:00	users	0	102.715004
2012-09-01 02:00:00	users	0	66.084412
2012-09-01 03:00:00	users	0	40.878693
2012-09-01 04:00:00	users	0	29.577866

Generated codeCopy

import pandas as pd                                                                       
from skforecast.foundation import FoundationModel, ForecasterFoundation                   
from skforecast.model_selection import TimeSeriesFold, backtesting_foundation             
                                                                                          
# Load data                                                                               
data = pd.read_csv('data.csv')                                                            
                                                                                          
data['date_time'] = pd.to_datetime(data['date_time'])                                     
data = data.set_index('date_time')                                                        
data = data.asfreq('h')                                                                   
data = data.sort_index()                                                                  
                                                                                          
exog_features = ['holiday', 'weather', 'temp']                                            
                                                                                          
# Create foundation model (chronos-2-small)                                               
estimator = FoundationModel(                                                              
    model_id       = 'autogluon/chronos-2-small',                                         
    context_length = 8192,                                                                
)                                                                                         
                                                                                          
# Create forecaster                                                                       
forecaster = ForecasterFoundation(estimator=estimator)                                    
                                                                                          
# Time series cross-validation configuration                                              
cv = TimeSeriesFold(                                                                      
    steps              = 36,                                                              
    initial_train_size = '2012-08-31 23:59:00',                                           
    refit              = False,                                                           
)                                                                                         
                                                                                          
# Run backtesting                                                                         
metrics, predictions = backtesting_foundation(                                            
    forecaster        = forecaster,                                                       
    series            = data['users'],                                                    
    cv                = cv,                                                               
    metric            = ['mean_absolute_error', 'mean_absolute_scaled_error'],            
    exog              = data[exog_features],                                              
    verbose           = False,                                                            
    show_progress     = True,                                                             
    suppress_warnings = True,                                                             
)                                                                                         
                                                                                          
print(metrics)                                                                            
print(predictions.head())                                                                 

The most useful result of a comparison is often not the leaderboard, but best_candidate. It is a complete BacktestResult containing both the winning profile and plan, so it can be passed directly into the rest of the workflow without manually rebuilding the configuration.

In [33]:

# Winning configuration
# ==============================================================================
print(f"Best candidate: {results_compare.best_name}")
results_compare.best_candidate.plan

# Winning configuration # ============================================================================== print(f"Best candidate: {results_compare.best_name}") results_compare.best_candidate.plan

Best candidate: foundation_model

Out[33]:

                Forecast Plan                 
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Property       ┃ Value                     ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Task type      │ foundation                │
├────────────────┼───────────────────────────┤
│ Forecaster     │ ForecasterFoundation      │
├────────────────┼───────────────────────────┤
│ Estimator      │ autogluon/chronos-2-small │
├────────────────┼───────────────────────────┤
│ Steps          │ 36                        │
├────────────────┼───────────────────────────┤
│ Frequency      │ h                         │
├────────────────┼───────────────────────────┤
│ Use exog       │ True                      │
├────────────────┼───────────────────────────┤
│ Interval       │ None                      │
├────────────────┼───────────────────────────┤
│ Primary metric │ mean_absolute_error       │
├────────────────┼───────────────────────────┤
│ Preprocessing  │ 1 step                    │
└────────────────┴───────────────────────────┘

                                   Preprocessing Steps                                    
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Step                    ┃ Reason                                                       ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ handle_categorical_exog │ Categorical exogenous variables detected: ['weather'].       │
│                         │ 'autogluon/chronos-2-small' consumes categorical covariates  │
│                         │ natively, so no encoding is needed.                          │
└─────────────────────────┴──────────────────────────────────────────────────────────────┘

╭─────────────────────────────────── Plan Explanation ───────────────────────────────────╮
│                                                                                        │
│  Plan: ForecasterFoundation + autogluon/chronos-2-small. No lag or window features:    │
│  the foundation model forecasts directly from the raw context window. Exogenous        │
│  variables included. Primary metric: mean_absolute_error, as requested; also           │
│  computed: mean_absolute_scaled_error. The model reads the last 8192 observations of   │
│  the series as context; the series has 17544, so older observations are not used.      │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯

In [34]:

# Produce the final forecast with the winning configuration
# ==============================================================================

# Simulate future values of exogenous variables for the next 36 hours
exog = data[['holiday', 'weather', 'temp']].tail(36).copy()
exog.index = pd.date_range(
    start=pd.to_datetime(data['date_time'].max()) + pd.Timedelta(1, unit='h'), periods=36, freq='h'
)

results_pred = assistant.forecast(
                   data        = data,
                   target      = 'users',
                   date_column = "date_time",
                   steps       = 36,
                   profile     = results_compare.profile,
                   plan        = results_compare.best_candidate.plan,
                   interval    = [0.1, 0.9],  # 80% prediction interval
                   test_size   = None,        # Use the entire dataset for training (prediction mode)
                   exog        = exog         # Future values of exogenous variables for the next 36 hours
               )

# Reproducible skforecast script for the winning configuration
# ==============================================================================
results_pred.show_code()

# Produce the final forecast with the winning configuration # ============================================================================== # Simulate future values of exogenous variables for the next 36 hours exog = data[['holiday', 'weather', 'temp']].tail(36).copy() exog.index = pd.date_range( start=pd.to_datetime(data['date_time'].max()) + pd.Timedelta(1, unit='h'), periods=36, freq='h' ) results_pred = assistant.forecast( data = data, target = 'users', date_column = "date_time", steps = 36, profile = results_compare.profile, plan = results_compare.best_candidate.plan, interval = [0.1, 0.9], # 80% prediction interval test_size = None, # Use the entire dataset for training (prediction mode) exog = exog # Future values of exogenous variables for the next 36 hours ) # Reproducible skforecast script for the winning configuration # ============================================================================== results_pred.show_code()

Loading weights:   0%|          | 0/92 [00:00<?, ?it/s]

Generated codeCopy

import pandas as pd                                                                       
from skforecast.foundation import FoundationModel, ForecasterFoundation                   
                                                                                          
# Load data                                                                               
data = pd.read_csv('data.csv')                                                            
                                                                                          
# Load future exogenous variables covering the forecast horizon                           
exog_future = pd.read_csv('exog_future.csv')                                              
                                                                                          
data['date_time'] = pd.to_datetime(data['date_time'])                                     
data = data.set_index('date_time')                                                        
data = data.asfreq('h')                                                                   
data = data.sort_index()                                                                  
                                                                                          
exog_future['date_time'] = pd.to_datetime(exog_future['date_time'])                       
exog_future = exog_future.set_index('date_time')                                          
exog_future = exog_future.asfreq('h')                                                     
exog_future = exog_future.sort_index()                                                    
                                                                                          
series = data['users']                                                                    
exog = data[['holiday', 'weather', 'temp']]                                               
                                                                                          
# Create foundation model (chronos-2-small)                                               
estimator = FoundationModel(                                                              
    model_id       = 'autogluon/chronos-2-small',                                         
    context_length = 8192,                                                                
)                                                                                         
                                                                                          
# Create forecaster                                                                       
forecaster = ForecasterFoundation(estimator=estimator)                                    
                                                                                          
# Fit (stores context only, no training)                                                  
forecaster.fit(series=series, exog=exog)                                                  
                                                                                          
# Predict intervals (native quantiles)                                                    
steps = 36                                                                                
predictions = forecaster.predict_interval(                                                
    steps    = steps,                                                                     
    exog     = exog_future,                                                               
    interval = [0.1, 0.9],                                                                
)                                                                                         
print(predictions)                                                                        

Like any other result, a ComparisonResult can be passed to ask() as context to have the ranking explained. The LLM restates what the leaderboard measured, not why one candidate beat another, and it cannot change the outcome: all metrics and rankings are computed deterministically before it sees the result.

In [35]:

# Asking the assistant about the comparison
# ==============================================================================
explanation = assistant.ask(
    prompt = (
        "Explain the comparison results. Is the margin between the top candidates "
        "meaningful, or are they practically equivalent?"
    ),
    context = results_compare
)

# Asking the assistant about the comparison # ============================================================================== explanation = assistant.ask( prompt = ( "Explain the comparison results. Is the margin between the top candidates " "meaningful, or are they practically equivalent?" ), context = results_compare )

/var/folders/wt/8tvn563d5v55nspfbydgqb9r0000gp/T/ipykernel_70978/955099911.py:3: DataSentToLLMWarning: `send_data_to_llm=False` does not apply to `context`: the predicted values it carries are sent to the LLM, because a question about a result cannot be answered from summary statistics alone. Your input data is not sent: a result holds only the model's output, never the data it was fitted on. To keep predictions local, ask without `context`.
  explanation = assistant.ask(

In [36]:

# Show the explanation generated by the LLM
# ==============================================================================
explanation.show_explanation()

# Show the explanation generated by the LLM # ============================================================================== explanation.show_explanation()

╭────────────────────────────────── Assistant Response ──────────────────────────────────╮
│                                                                                        │
│  The comparison ranked six candidate configurations evaluated over 82 backtesting      │
│  folds with a 36-step horizon, with foundation_model achieving the lowest              │
│  mean_absolute_error of 38.436159, followed by lgbm_direct at 50.224908. Whether the   │
│  margin between these candidates is statistically meaningful or practically            │
│  equivalent cannot be assessed because per-fold spread and variance are not provided   │
│  in the context. The leaderboard reports aggregate ranking metrics as observed, but    │
│  does not provide distribution details across folds.                                   │
│                                                                                        │
│  Leaderboard Summary                                                                   │
│                                                                                        │
│   • foundation_model (ForecasterFoundation, autogluon/chronos-2-small): rank 1,        │
│     mean_absolute_error of 38.436159, mean_absolute_scaled_error of 0.597467.          │
│   • lgbm_direct (ForecasterDirect, LGBMRegressor): rank 2, mean_absolute_error of      │
│     50.224908, mean_absolute_scaled_error of 0.819189.                                 │
│   • lgbm_weekly_lags_rolling (ForecasterRecursive, LGBMRegressor): rank 3,             │
│     mean_absolute_error of 50.557647, mean_absolute_scaled_error of 0.824568.          │
│   • lgbm_daily_lags (ForecasterRecursive, LGBMRegressor): rank 4, mean_absolute_error  │
│     of 55.517689, mean_absolute_scaled_error of 0.905517.                              │
│   • Baseline (seasonal naive) (ForecasterEquivalentDate): rank 5, mean_absolute_error  │
│     of 91.668716, mean_absolute_scaled_error of 1.503712.                              │
│   • ridge_baseline (ForecasterRecursive, Ridge): rank 6, mean_absolute_error of        │
│     93.145620, mean_absolute_scaled_error of 1.519244.                                 │
│                                                                                        │
│  Assessing Margins Between Candidates                                                  │
│                                                                                        │
│   • Per-fold spread is not given, so whether the difference between foundation_model   │
│     and the remaining models is meaningful cannot be assessed.                         │
│   • Similarly, whether the small numeric difference between lgbm_direct (50.224908)    │
│     and lgbm_weekly_lags_rolling (50.557647) represents practical equivalence cannot   │
│     be confirmed without per-fold variation data.                                      │
│   • According to the deterministic summary, foundation_model is 23.5% ahead of         │
│     lgbm_direct and beats the Baseline (seasonal naive) row by 58.1%.                  │
│   • The leaderboard reports what occurred across the aggregate evaluation, but it      │
│     does not supply causal explanations for why one candidate ranked above another.    │
│                                                                                        │
│  Metric Interpretation                                                                 │
│                                                                                        │
│   • Candidates with a mean_absolute_scaled_error below 1 beat the one-step naive       │
│     forecast (last training value). This applies to foundation_model (0.597467),       │
│     lgbm_direct (0.819189), lgbm_weekly_lags_rolling (0.824568), and lgbm_daily_lags   │
│     (0.905517).                                                                        │
│   • Candidates with a mean_absolute_scaled_error above 1 do not beat the one-step      │
│     naive forecast (last training value). This applies to Baseline (seasonal naive)    │
│     (1.503712) and ridge_baseline (1.519244).                                          │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯

This tutorial covered the fast path of skforecast-ai: the most efficient route from raw time series data to a validated forecast with minimal manual configuration. This approach is highly effective for rapidly generating robust baseline models when you trust the assistant to implement sensible default strategies.

However, if your use case requires granular control to inspect, audit, or modify the intermediate objects (such as the data profile, the modeling plan, or the validation schema) before executing the next phase, we recommend exploring the step-by-step workflow for a comprehensive, stage-by-stage walkthrough of the underlying architecture.