Agentic forecasting with skforecast-ai¶

What is skforecast-ai?¶

skforecast-ai is an AI forecasting assistant that pairs a deterministic engine, powered by skforecast, with an LLM reasoning layer. Simply provide a time series, and the assistant automatically profiles the data, selects a model using established best practices, and evaluates its performance. It returns both the final forecast and the runnable skforecast script that produced it.

It is organized around a single core object, the ForecastingAssistant, which consists of two complementary components:

• Deterministic Engine (Rule-based and Reproducible): Profiles the data, selects a forecaster and estimator, derives lags and preprocessing steps, runs backtesting, and produces the final forecast. Crucially, it outputs the exact standalone skforecast script that generated the results. Given the same inputs and configuration, this workflow is guaranteed to be reproducible.

• Reasoning Layer (LLM-powered): Accessed primarily via the ask() method, this layer interprets and explains the objects and results you pass to it: data profiles, modeling plans, validation choices, backtesting outputs, and forecasts. The LLM acts strictly as an interpreter; it does not rerun the workflow or silently change modeling recommendations behind the scenes. Agentic features, such as the LLM-guided refine_plan() or create_cv(), are separate, explicit steps where the LLM suggests adjustments that are then implemented transparently in deterministic code.

Two ways to use skforecast-ai¶

skforecast-ai supports two distinct workflows using the same underlying forecasting engine:

• The Fast Path: Use this when you want a forecast or backtest result in a single call. The assistant profiles the data, builds the modeling plan, executes the workflow, and returns the results alongside the reproducible skforecast code.

• The Step-by-Step Path: Use this when you want granular control to inspect or adjust intermediate decisions. You can manually create a profile, build a plan, optionally refine it with the LLM, define a validation strategy, evaluate the model, and then generate the forecast.

A useful mental model is that forecasting and validation are separate branches. Once you have a profile and a plan, you can use forecast() to produce future predictions directly, or backtest() to evaluate the model's performance on historical data. You can also use compare() to evaluate several candidate configurations under the same cross-validation strategy and obtain a ranked leaderboard, so the best configuration is chosen from measured performance rather than intuition.

The ask() method is available in both workflows. It can explain a profile, plan, validation setup, backtest result, comparison result, or answer general forecasting questions, but it will never execute the workflow or modify your parameters without explicit instruction.

Fast path: one call

Profiling, planning and execution happen internally.

data

↓

Forecast

forecast()
or forecast_code()

↓

predictions + code

Backtesting (validation)

create_cv()
Deterministic, Agentic mode
or pass a skforecast TimeSeriesFold object

↓

backtest()
or backtest_code()

↓

metrics + predictions + code

Step-by-step path: full control

Build a profile and a plan from your data, then branch into forecasting and backtesting.

data

↓

profile()

↓

plan()
refine_plan(), optional (Deterministic or Agentic mode)

↓

Forecast

forecast()
or forecast_code()

↓

predictions + code

Backtesting (validation)

create_cv()
Deterministic, Agentic mode
or pass a skforecast TimeSeriesFold object

↓

backtest()
or backtest_code()

↓

metrics + predictions + code

Model selection: which forecaster should you use?

compare() answers the question every forecasting project starts with: Among several reasonable models, which one actually performs best on my data? Every candidate is evaluated using the same data and cross-validation strategy. Therefore, the differences you see come from the models, not from the setup.

Candidates A handful of configurations worth testing: different forecasters, estimators, lags or window features. Supply your own, or let the data profile propose them.

→

compare() Runs a full backtest for each candidate under identical conditions, and scores them with the metrics you care about.

→

A ranked answer A leaderboard sorted best to worst, the reproducible code behind every row, and the winner ready to be used for forecasting or further tuning.

The ranking is a plain sort of the metric column: fully deterministic and auditable. The LLM plays no part in choosing the winner.

LLM reasoning: available at any moment, in any workflow

Call ask() before, during or after either path. It can take a profile, a plan, a result (from forecast(), backtest() or compare()), or nothing at all (pure Q&A).

The following example walks through the fast path: the quickest way to go from raw data to a validated forecast with minimal setup. It is ideal when you want rapid results and trust the assistant to make sensible, baseline modeling decisions on your behalf. If you prefer to understand and control what happens under the hood step by step, visit the step-by-step path tutorial for a more detailed walkthrough.

Assistant initialization¶

The first step is to instantiate a ForecastingAssistant, which will be responsible for executing the entire workflow (profiling, planning, backtesting, and forecasting), as well as explaining the outputs and suggesting improvements.

To activate the optional LLM support, users must pass a string in the format 'provider:model_name' (for example, 'openai:gpt-5.5', 'google:gemini-3-flash-preview', 'anthropic:claude-sonnet-5', or 'ollama:qwen3:8b'). For hosted providers, the corresponding API key must be available as an environment variable or passed explicitly when creating the assistant. In this tutorial, we set send_data_to_llm=False. This ensures strict data privacy: the LLM receives only metadata and summary statistics, never the raw time series values.

In [1]:

# Data processing
# ==============================================================================
import os
import pandas as pd
from skforecast.datasets import fetch_dataset

# Plots
# ==============================================================================
import matplotlib.pyplot as plt
import plotly.graph_objects as go
import plotly.io as pio
import plotly.offline as poff
pio.templates.default = "seaborn"
poff.init_notebook_mode(connected=True)
plt.style.use('seaborn-v0_8-darkgrid')

# skforecast and skforecast-ai
# ==============================================================================
import skforecast
import skforecast_ai
from skforecast_ai import ForecastingAssistant
from skforecast.model_selection import TimeSeriesFold

color = '\033[1m\033[38;5;208m' 
print(f"{color}Version skforecast_ai: {skforecast_ai.__version__}")
print(f"{color}Version skforecast: {skforecast.__version__}")

# Data processing # ============================================================================== import os import pandas as pd from skforecast.datasets import fetch_dataset # Plots # ============================================================================== import matplotlib.pyplot as plt import plotly.graph_objects as go import plotly.io as pio import plotly.offline as poff pio.templates.default = "seaborn" poff.init_notebook_mode(connected=True) plt.style.use('seaborn-v0_8-darkgrid') # skforecast and skforecast-ai # ============================================================================== import skforecast import skforecast_ai from skforecast_ai import ForecastingAssistant from skforecast.model_selection import TimeSeriesFold color = '\033[1m\033[38;5;208m' print(f"{color}Version skforecast_ai: {skforecast_ai.__version__}") print(f"{color}Version skforecast: {skforecast.__version__}")

Version skforecast_ai: 0.2.0
Version skforecast: 0.23.0

✏️ Note

If you do not have access to an LLM assistant, you can still follow the full tutorial using only the deterministic methods. Profiling, planning, backtesting, and forecasting all run without an LLM. Only the ask() explanations and the LLM-guided variants of refine_plan() and create_cv() require a configured LLM; their deterministic counterparts (for example, refine_plan() with explicit overrides and prompt=None) work without one.

In [2]:

# LLM-enabled assistant
# ==============================================================================
LLM_MODEL = "google:gemini-3.5-flash"
api_key = os.getenv("GOOGLE_API_KEY")

assistant = ForecastingAssistant(
    llm=LLM_MODEL, api_key=api_key, send_data_to_llm=False
)

# Using aws bedrock
# ==============================================================================
# assistant = ForecastingAssistant(
#     llm='bedrock:eu.anthropic.claude-sonnet-4-6',
#     base_url="eu-west-1"
# )

# Assistant without reasoning layer
# ==============================================================================
# assistant = ForecastingAssistant()

# LLM-enabled assistant # ============================================================================== LLM_MODEL = "google:gemini-3.5-flash" api_key = os.getenv("GOOGLE_API_KEY") assistant = ForecastingAssistant( llm=LLM_MODEL, api_key=api_key, send_data_to_llm=False ) # Using aws bedrock # ============================================================================== # assistant = ForecastingAssistant( # llm='bedrock:eu.anthropic.claude-sonnet-4-6', # base_url="eu-west-1" # ) # Assistant without reasoning layer # ============================================================================== # assistant = ForecastingAssistant()

⚠️ Your data stays private

By default, enabling an LLM does not send your time-series data to the model provider. The assistant passes only summary statistics, detected frequency, seasonality flags and the forecaster configuration, never the raw observations. To explicitly allow it, pass send_data_to_llm=True.

Data¶

The data in this document represent the hourly usage of the bike share system in the city of Washington, D.C. during the years 2011 and 2012. In addition to the number of users per hour, information about weather conditions and holidays is available.

In [3]:

# Downloading data
# ==============================================================================
data = fetch_dataset('bike_sharing', raw=True)
data = data[['date_time', 'users', 'holiday', 'weather', 'temp']]
data.head()

# Downloading data # ============================================================================== data = fetch_dataset('bike_sharing', raw=True) data = data[['date_time', 'users', 'holiday', 'weather', 'temp']] data.head()

╭───────────────────────────────── bike_sharing ──────────────────────────────────╮
│ Description:                                                                    │
│ Hourly usage of the bike share system in the city of Washington D.C. during the │
│ years 2011 and 2012. In addition to the number of users per hour, information   │
│ about weather conditions and holidays is available.                             │
│                                                                                 │
│ Source:                                                                         │
│ Fanaee-T,Hadi. (2013). Bike Sharing Dataset. UCI Machine Learning Repository.   │
│ https://doi.org/10.24432/C5W894.                                                │
│                                                                                 │
│ URL:                                                                            │
│ https://raw.githubusercontent.com/skforecast/skforecast-                        │
│ datasets/main/data/bike_sharing_dataset_clean.csv                               │
│                                                                                 │
│ Shape: 17544 rows x 12 columns                                                  │
╰─────────────────────────────────────────────────────────────────────────────────╯

Out[3]:

	date_time	users	holiday	weather	temp
0	2011-01-01 00:00:00	16.0	0.0	clear	9.84
1	2011-01-01 01:00:00	40.0	0.0	clear	9.02
2	2011-01-01 02:00:00	32.0	0.0	clear	9.02
3	2011-01-01 03:00:00	13.0	0.0	clear	9.84
4	2011-01-01 04:00:00	1.0	0.0	clear	9.84

✏️ Note

skforecast-ai is ready to preprocess the data, but it is recommended that users apply their own preprocessing steps before using the assistant. This ensures the data is in the desired format and any necessary transformations have been applied before proceeding with the forecasting workflow.

In [4]:

# Interactive plot of time series
# ==============================================================================
fig = go.Figure()
fig.add_trace(
    go.Scatter(x=data['date_time'], y=data['users'], mode='lines', name='Train')
)
fig.update_layout(
    title  = 'Number of users',
    xaxis_title="Time",
    yaxis_title="Users",
    legend_title="Partition:",
    width=800,
    height=400,
    margin=dict(l=20, r=20, t=35, b=20),
    legend=dict(orientation="h", yanchor="top", y=1, xanchor="left", x=0.001)
)
fig.show()

# Interactive plot of time series # ============================================================================== fig = go.Figure() fig.add_trace( go.Scatter(x=data['date_time'], y=data['users'], mode='lines', name='Train') ) fig.update_layout( title = 'Number of users', xaxis_title="Time", yaxis_title="Users", legend_title="Partition:", width=800, height=400, margin=dict(l=20, r=20, t=35, b=20), legend=dict(orientation="h", yanchor="top", y=1, xanchor="left", x=0.001) ) fig.show()

For a deeper walkthrough of the exploratory analysis behind this dataset, see the skforecast example: Forecasting time series with skforecast, XGBoost, LightGBM and CatBoost.

Forecasting with the assistant¶

The forecast() method is the fastest way to generate predictions. In a single call, it executes the full pipeline (profile → plan → execute) and returns a ForecastResult holding the predictions, the evaluation metrics (when available), and the exact standalone skforecast script that produced them.

The behavior is controlled by a single switch, test_size, which selects one of two modes:

• Prediction mode (test_size = None, the default): The model is trained on the entire dataset and forecasts the next steps time points into the future. Because there is no ground truth to compare against, no metrics are returned. If the historical data contains exogenous variables, their future values must be explicitly supplied via the exog argument.

• Evaluation mode (test_size is set). The dataset is split into train and test sets, the model is trained on the training portion, and predictions for the test window are compared against the held-out actuals to compute metrics. In this mode, the test-set exogenous values are taken from the split, so exog must not be passed.

The main arguments are:

Argument	Type	Description
data	Series, DataFrame, str, Path	Input dataset or path to a CSV file. When a Series is passed, the target is taken from its name.
steps	int	Forecast horizon (number of steps ahead to predict).
target	str, list of str	Column to forecast. Optional when data is a Series. Pass a list of column names for wide-format multi-series.
date_column	str	Column holding the timestamps. When None, the index of data must already be a DatetimeIndex.
exog	DataFrame	Future exogenous values covering the horizon (at least steps rows). Used only in prediction mode, and required there when the data has exogenous variables.
interval	list of float	Prediction interval as [lower, upper] quantiles (e.g. [0.1, 0.9] for an 80% interval). None disables intervals.
test_size	int, float, str, Timestamp	Size or start of the test set, selecting the mode above. int: last test_size observations; float in (0, 1): last fraction of observations; str/Timestamp: first timestamp of the test set. None runs prediction mode.

💡 Tip

You can also use custom configurations for the forecaster, estimator, estimator_kwargs, lags, and window_features by passing them as arguments to forecast(). This allows you to override the assistant's automatic choices and use your own preferred settings. For example, this code specifies a custom forecaster while still letting the assistant handle the rest of the workflow:

results = assistant.forecast(
              data        = data,
              target      = 'users',
              date_column = "date_time",
              steps       = 36,
              forecaster  = 'ForecasterFoundation'
          )

The following code demonstrates how to execute the assistant in evaluation mode. By explicitly defining the test_size parameter (in this case, test_size=36), the pipeline automatically reserves the final 36 observations as a test set.

During execution, the assistant profiles the dataset, incorporates the provided exogenous features (holiday, weather, and temp), and trains an appropriate model on the training split. Because the method is triggered in evaluation mode, the resulting ForecastResult object provides the out-of-sample predictions, an 80% prediction interval, and the performance metrics evaluated against the held-out ground truth.

In [5]:

# Execute the forecasting assistant in evaluation mode
# ==============================================================================
results = assistant.forecast(
              data        = data,
              target      = 'users',
              date_column = "date_time",
              steps       = 36,
              interval    = [0.1, 0.9],  # 80% prediction interval
              test_size   = 36           # Last 36 hours as test set (evaluation mode)
          )

# Execute the forecasting assistant in evaluation mode # ============================================================================== results = assistant.forecast( data = data, target = 'users', date_column = "date_time", steps = 36, interval = [0.1, 0.9], # 80% prediction interval test_size = 36 # Last 36 hours as test set (evaluation mode) )

In [6]:

# Inspect the performance metrics and the resulting predictions
# ==============================================================================
display(results.metrics)
display(results.predictions.head())

# Inspect the performance metrics and the resulting predictions # ============================================================================== display(results.metrics) display(results.predictions.head())

	series	MAE	MSE	MASE	MAPE
0	users	32.013374	2758.8615	0.497185	0.415688

	pred	lower_bound	upper_bound
2012-12-30 12:00:00	146.512223	111.097550	181.185404
2012-12-30 13:00:00	138.768717	96.950185	178.804105
2012-12-30 14:00:00	145.133323	92.701496	185.332563
2012-12-30 15:00:00	140.891323	89.525337	185.272888
2012-12-30 16:00:00	139.538959	86.994814	180.904345

The show_code() method allows users to retrieve the exact code used to generate the forecast.

In [7]:

# skforecast code that generated the results
# ==============================================================================
results.show_code()

# skforecast code that generated the results # ============================================================================== results.show_code()

Generated codeCopy

import pandas as pd                                                                       
from sklearn.metrics import mean_absolute_error, mean_squared_error,                      
mean_absolute_percentage_error                                                            
from skforecast.metrics import mean_absolute_scaled_error                                 
from lightgbm import LGBMRegressor                                                        
from skforecast.preprocessing import RollingFeatures, CalendarFeatures                    
from skforecast.recursive import ForecasterRecursive                                      
                                                                                          
# Load data                                                                               
data = pd.read_csv('data.csv')                                                            
                                                                                          
data['date_time'] = pd.to_datetime(data['date_time'])                                     
data = data.set_index('date_time')                                                        
data = data.asfreq('h')                                                                   
data = data.sort_index()                                                                  
                                                                                          
# Train/test split                                                                        
end_train = '2012-12-30 11:00:00'  # last training date, adjust to change the split point 
data_train = data.loc[:end_train]                                                         
data_test  = data.loc[data.index > end_train]                                             
exog_features = ['holiday', 'weather', 'temp']                                            
                                                                                          
print(                                                                                    
    f"Train dates : {data_train.index.min()} --- {data_train.index.max()}                 
(n={len(data_train)})"                                                                    
)                                                                                         
print(                                                                                    
    f"Test dates  : {data_test.index.min()} --- {data_test.index.max()}                   
(n={len(data_test)})"                                                                     
)                                                                                         
                                                                                          
window_features = RollingFeatures(                                                        
    stats        = ['mean', 'std', 'mean', 'mean'],                                       
    window_sizes = [3, 3, 24, 168],                                                       
)                                                                                         
                                                                                          
calendar_features = CalendarFeatures(                                                     
    features = ['hour', 'day_of_week', 'weekend', 'month'],                               
    encoding = None,                                                                      
)                                                                                         
                                                                                          
# Create forecaster                                                                       
forecaster = ForecasterRecursive(                                                         
    estimator            = LGBMRegressor(random_state=123, verbose=-1),                   
    lags                 = [1, 2, 3, 5, 8, 10, 15, 17, 19, 20, 21, 22, 23, 24, 25, 26, 32,
33, 119, 121, 135, 136, 142, 143, 145, 160, 166, 167, 169, 313, 335, 337],                
    window_features      = window_features,                                               
    calendar_features    = calendar_features,                                             
    categorical_features = 'auto',                                                        
    dropna_from_series   = False,                                                         
)                                                                                         
                                                                                          
# Fit                                                                                     
forecaster.fit(                                                                           
    y                         = data_train['users'],                                      
    exog                      = data_train[exog_features],                                
    store_in_sample_residuals = True,                                                     
)                                                                                         
                                                                                          
# Predict intervals                                                                       
steps = 36                                                                                
predictions = forecaster.predict_interval(                                                
    steps    = steps,                                                                     
    exog     = data_test[exog_features],                                                  
    method   = 'bootstrapping',                                                           
    interval = [0.1, 0.9],                                                                
)                                                                                         
print(predictions)                                                                        
                                                                                          
# Evaluate on test set                                                                    
actual = data_test['users'].iloc[:steps]                                                  
mae = mean_absolute_error(actual, predictions['pred'])                                    
mse = mean_squared_error(actual, predictions['pred'])                                     
mase = mean_absolute_scaled_error(                                                        
    y_true  = actual,                                                                     
    y_pred  = predictions['pred'],                                                        
    y_train = data_train['users'],                                                        
)                                                                                         
mape = mean_absolute_percentage_error(actual, predictions['pred'])                        
                                                                                          
print(f"MAE  : {mae:.4f}")                                                                
print(f"MSE  : {mse:.4f}")                                                                
print(f"MASE : {mase:.4f}")                                                               
print(f"MAPE : {mape:.4f}")                                                               
                                                                                          
# NOTE: This script uses a train/test split for demonstration purposes.                   
# For production forecasting, retrain with all available data                             
# and provide future exogenous values covering the forecast horizon.                      

To generate forecasts for true future periods (starting immediately after the last observation in the dataset), the assistant is executed in prediction mode. This mode is active by default when test_size=None. In this configuration, the pipeline trains the optimal model using the entire available dataset and projects the predictions forward by the specified number of steps. Because there is no holdout data to serve as ground truth, the resulting ForecastResult object does not compute performance metrics. Additionally, if the historical data relies on exogenous features, a DataFrame containing their future values spanning the entire forecast horizon must be explicitly supplied via the exog parameter.

In [8]:

# Forecast the next 36 hours using the entire dataset (prediction mode)
# ==============================================================================
# Simulate future values of exogenous variables for the next 36 hours
exog = data[['holiday', 'weather', 'temp']].tail(36).copy()
exog.index = pd.date_range(
    start=pd.to_datetime(data['date_time'].max()) + pd.Timedelta(hours=1), periods=36, freq='h'
)

results_pred = assistant.forecast(
                   data        = data,
                   target      = 'users',
                   date_column = "date_time",
                   steps       = 36,
                   interval    = [0.1, 0.9],  # 80% prediction interval
                   test_size   = None,        # Use the entire dataset for training (prediction mode)
                   exog        = exog         # Future values of exogenous variables for the next 36 hours
               )

display(results_pred.predictions.head())

# Forecast the next 36 hours using the entire dataset (prediction mode) # ============================================================================== # Simulate future values of exogenous variables for the next 36 hours exog = data[['holiday', 'weather', 'temp']].tail(36).copy() exog.index = pd.date_range( start=pd.to_datetime(data['date_time'].max()) + pd.Timedelta(hours=1), periods=36, freq='h' ) results_pred = assistant.forecast( data = data, target = 'users', date_column = "date_time", steps = 36, interval = [0.1, 0.9], # 80% prediction interval test_size = None, # Use the entire dataset for training (prediction mode) exog = exog # Future values of exogenous variables for the next 36 hours ) display(results_pred.predictions.head())

	pred	lower_bound	upper_bound
2013-01-01 00:00:00	27.442927	11.127479	47.557452
2013-01-01 01:00:00	13.318240	3.310496	27.242435
2013-01-01 02:00:00	9.208553	2.380079	19.120340
2013-01-01 03:00:00	5.329501	0.570137	10.013675
2013-01-01 04:00:00	5.979160	1.305255	10.265962

Both modes return a ForecastResult, a lightweight container that bundles everything the assistant used and produced, so you can inspect the outputs, audit the decisions, or lift the code straight into production.

Attribute	Type	Description
predictions	DataFrame	Forecasted values for the requested steps. When intervals (or quantiles) are requested, the bound columns are included alongside the point predictions.
metrics	DataFrame, None	Evaluation metrics (MAE, MSE, MASE), one row per series. None in prediction mode, where there is no ground truth to score against.
code	str	The exact standalone skforecast script that produced the forecast, deterministic and ready to run on its own.
profile	ForecastingProfile	The data profile behind the forecast: metadata, summary statistics, detected frequency and seasonality, and the high-level modeling decisions.
plan	ForecastPlan	The detailed configuration that was executed: forecaster, estimator, lags, window features, preprocessing, and interval settings.

Displaying the object in a notebook renders a rich summary of all of the above; the raw script is also available through results.show_code().

In [9]:

# Full results object
# ==============================================================================
results_pred

# Full results object # ============================================================================== results_pred

Out[9]:

                          Dataset Profile                          
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Property       ┃ Value                                          ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Format         │ single                                         │
├────────────────┼────────────────────────────────────────────────┤
│ Series         │ 1                                              │
├────────────────┼────────────────────────────────────────────────┤
│ Observations   │ 17544                                          │
├────────────────┼────────────────────────────────────────────────┤
│ Frequency      │ h                                              │
├────────────────┼────────────────────────────────────────────────┤
│ Target         │ users                                          │
├────────────────┼────────────────────────────────────────────────┤
│ Exog columns   │ holiday, weather, temp  (categorical: weather) │
├────────────────┼────────────────────────────────────────────────┤
│ Missing values │ None                                           │
└────────────────┴────────────────────────────────────────────────┘

                                    Recommendation                                     
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Property              ┃ Value                                                       ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Task type             │ single_series                                               │
├───────────────────────┼─────────────────────────────────────────────────────────────┤
│ Forecaster            │ ForecasterRecursive                                         │
├───────────────────────┼─────────────────────────────────────────────────────────────┤
│ Forecaster candidates │ ForecasterRecursive, ForecasterDirect, ForecasterFoundation │
├───────────────────────┼─────────────────────────────────────────────────────────────┤
│ Estimator             │ LGBMRegressor                                               │
├───────────────────────┼─────────────────────────────────────────────────────────────┤
│ Estimator candidates  │ LGBMRegressor, XGBRegressor, Ridge                          │
└───────────────────────┴─────────────────────────────────────────────────────────────┘

╭───────────────────────────────── Profile Explanation ──────────────────────────────────╮
│                                                                                        │
│  A single-series ML forecaster (ForecasterRecursive) is recommended. Data: 17544       │
│  observations, 'h' frequency. Alternative forecasters: ['ForecasterDirect',            │
│  'ForecasterFoundation']. Estimator: LGBMRegressor. A gradient boosting model is       │
│  preferred for a dataset of this size (17544 observations). Alternative estimators:    │
│  ['XGBRegressor', 'Ridge']. 3 exogenous variables (1 categorical) available as         │
│  predictors.                                                                           │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯
                                      Forecast Plan                                       
┏━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Property          ┃ Value                                                              ┃
┡━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Task type         │ single_series                                                      │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Forecaster        │ ForecasterRecursive                                                │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Estimator         │ LGBMRegressor                                                      │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Steps             │ 36                                                                 │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Frequency         │ h                                                                  │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Lags              │ [1, 2, 3, 5, 8, 10, 15, 17, 19, 20, 21, 22, 23, 24, 25, 26, 32,    │
│                   │ 33, 119, 121, 135, 136, 142, 143, 145, 160, 166, 167, 169, 313,    │
│                   │ 335, 337]                                                          │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Window features   │ [{'stats': ['mean', 'std'], 'window_size': 3}, {'stats': ['mean'], │
│                   │ 'window_size': 24}, {'stats': ['mean'], 'window_size': 168}]       │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Calendar features │ ['hour', 'day_of_week', 'weekend', 'month'] (raw ordinal encoding) │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Use exog          │ True                                                               │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Interval          │ [0.1, 0.9]                                                         │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Interval method   │ bootstrapping                                                      │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Primary metric    │ mean_absolute_error                                                │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Preprocessing     │ 1 step                                                             │
└───────────────────┴────────────────────────────────────────────────────────────────────┘

                                   Preprocessing Steps                                    
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Step                    ┃ Reason                                                       ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ handle_categorical_exog │ Categorical exogenous variables detected: ['weather']. These │
│                         │ are handled automatically by skforecast                      │
│                         │ (categorical_features='auto').                               │
└─────────────────────────┴──────────────────────────────────────────────────────────────┘

╭─────────────────────────────────── Plan Explanation ───────────────────────────────────╮
│                                                                                        │
│  Plan: ForecasterRecursive + LGBMRegressor. Lags: [1, 2, 3, 5, 8, 10, 15, 17, 19, 20,  │
│  21, 22, 23, 24, 25, 26, 32, 33, 119, 121, 135, 136, 142, 143, 145, 160, 166, 167,     │
│  169, 313, 335, 337]. Window features: ['mean(window=3)', 'std(window=3)',             │
│  'mean(window=24)', 'mean(window=168)']. Calendar features: ['hour', 'day_of_week',    │
│  'weekend', 'month'] (raw ordinal encoding). Prediction intervals via bootstrapping.   │
│  NaN rows kept (NaN-tolerant estimator). Exogenous variables included. MAE is          │
│  interpretable, robust to outliers, and works at any scale.                            │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯
                    Predictions (36 rows)                     
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃ Index               ┃     pred ┃ lower_bound ┃ upper_bound ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│ 2013-01-01 00:00:00 │  27.4429 │     11.1275 │     47.5575 │
├─────────────────────┼──────────┼─────────────┼─────────────┤
│ 2013-01-01 01:00:00 │  13.3182 │      3.3105 │     27.2424 │
├─────────────────────┼──────────┼─────────────┼─────────────┤
│ 2013-01-01 02:00:00 │   9.2086 │      2.3801 │     19.1203 │
├─────────────────────┼──────────┼─────────────┼─────────────┤
│ 2013-01-01 03:00:00 │   5.3295 │      0.5701 │     10.0137 │
├─────────────────────┼──────────┼─────────────┼─────────────┤
│ 2013-01-01 04:00:00 │   5.9792 │      1.3053 │     10.2660 │
├─────────────────────┼──────────┼─────────────┼─────────────┤
│ ...                 │      ... │         ... │         ... │
├─────────────────────┼──────────┼─────────────┼─────────────┤
│ 2013-01-02 07:00:00 │ 125.3912 │     77.8800 │    189.4169 │
├─────────────────────┼──────────┼─────────────┼─────────────┤
│ 2013-01-02 08:00:00 │ 253.6618 │    159.2711 │    365.0444 │
├─────────────────────┼──────────┼─────────────┼─────────────┤
│ 2013-01-02 09:00:00 │ 190.4495 │    133.4486 │    237.9817 │
├─────────────────────┼──────────┼─────────────┼─────────────┤
│ 2013-01-02 10:00:00 │ 132.0387 │    100.9730 │    173.2996 │
├─────────────────────┼──────────┼─────────────┼─────────────┤
│ 2013-01-02 11:00:00 │ 145.3529 │    110.8851 │    201.4766 │
└─────────────────────┴──────────┴─────────────┴─────────────┘

Generated codeCopy

import pandas as pd                                                                       
from lightgbm import LGBMRegressor                                                        
from skforecast.preprocessing import RollingFeatures, CalendarFeatures                    
from skforecast.recursive import ForecasterRecursive                                      
                                                                                          
# Load data                                                                               
data = pd.read_csv('data.csv')                                                            
                                                                                          
# Load future exogenous variables covering the forecast horizon                           
exog_future = pd.read_csv('exog_future.csv')                                              
                                                                                          
data['date_time'] = pd.to_datetime(data['date_time'])                                     
data = data.set_index('date_time')                                                        
data = data.asfreq('h')                                                                   
data = data.sort_index()                                                                  
                                                                                          
exog_future['date_time'] = pd.to_datetime(exog_future['date_time'])                       
exog_future = exog_future.set_index('date_time')                                          
exog_future = exog_future.asfreq('h')                                                     
exog_future = exog_future.sort_index()                                                    
                                                                                          
exog_features = ['holiday', 'weather', 'temp']                                            
                                                                                          
window_features = RollingFeatures(                                                        
    stats        = ['mean', 'std', 'mean', 'mean'],                                       
    window_sizes = [3, 3, 24, 168],                                                       
)                                                                                         
                                                                                          
calendar_features = CalendarFeatures(                                                     
    features = ['hour', 'day_of_week', 'weekend', 'month'],                               
    encoding = None,                                                                      
)                                                                                         
                                                                                          
# Create forecaster                                                                       
forecaster = ForecasterRecursive(                                                         
    estimator            = LGBMRegressor(random_state=123, verbose=-1),                   
    lags                 = [1, 2, 3, 5, 8, 10, 15, 17, 19, 20, 21, 22, 23, 24, 25, 26, 32,
33, 119, 121, 135, 136, 142, 143, 145, 160, 166, 167, 169, 313, 335, 337],                
    window_features      = window_features,                                               
    calendar_features    = calendar_features,                                             
    categorical_features = 'auto',                                                        
    dropna_from_series   = False,                                                         
)                                                                                         
                                                                                          
# Fit                                                                                     
forecaster.fit(                                                                           
    y                         = data['users'],                                            
    exog                      = data[exog_features],                                      
    store_in_sample_residuals = True,                                                     
)                                                                                         
                                                                                          
# Predict intervals                                                                       
steps = 36                                                                                
predictions = forecaster.predict_interval(                                                
    steps    = steps,                                                                     
    exog     = exog_future[exog_features],                                                
    method   = 'bootstrapping',                                                           
    interval = [0.1, 0.9],                                                                
)                                                                                         
print(predictions)                                                                        

Interpreting results¶

After executing the workflow, you can leverage the LLM reasoning layer to interpret the outputs. By passing the ForecastResult object to the ask() method, the assistant analyzes the generated predictions, performance metrics, and model configuration. This allows you to quickly extract actionable insights (such as contextualizing specific error metrics, identifying predicted trends, or diagnosing model behavior) directly in natural language.

In [10]:

# Asking the assistant about the forecast results
# ==============================================================================
explanation = assistant.ask(
    prompt = "Explain the results of the forecast, including the metrics and predictions.",
    result = results
)

# Asking the assistant about the forecast results # ============================================================================== explanation = assistant.ask( prompt = "Explain the results of the forecast, including the metrics and predictions.", result = results )

/var/folders/wt/8tvn563d5v55nspfbydgqb9r0000gp/T/ipykernel_48657/2167230761.py:3: DataSentToLLMWarning: `send_data_to_llm=False` does not apply to `result`: the predicted values it carries are sent to the LLM, because a question about a result cannot be answered from summary statistics alone. Your input data is not sent: a result holds only the model's output, never the data it was fitted on. To keep predictions local, ask without `result`.
  explanation = assistant.ask(

In [11]:

# Show the explanation generated by the LLM
# ==============================================================================
explanation.show_explanation()

# Show the explanation generated by the LLM # ============================================================================== explanation.show_explanation()

╭────────────────────────────────── Assistant Response ──────────────────────────────────╮
│                                                                                        │
│  The recursive machine learning forecaster successfully predicted future hourly        │
│  values of the users series over a 36-step horizon. Based on the evaluation metrics,   │
│  the model outperformed the naive baseline. The predictions include point forecasts    │
│  along with an eighty percent prediction interval generated via bootstrapping.         │
│                                                                                        │
│  Evaluation Metrics                                                                    │
│                                                                                        │
│  The model was evaluated using several key metrics to assess point forecast accuracy:  │
│                                                                                        │
│   • Mean Absolute Error (MAE): 32.013374                                               │
│   • Mean Squared Error (MSE): 2758.8615                                                │
│   • Mean Absolute Scaled Error (MASE): 0.497185                                        │
│   • Mean Absolute Percentage Error (MAPE): 0.415688                                    │
│                                                                                        │
│  The MASE value is 0.497185. Because this is below 1, it demonstrates that the         │
│  forecaster performed better than the naive baseline. The MAPE is 0.415688, though     │
│  percentage-based metrics like MAPE can become unreliable as the target values         │
│  approach zero.                                                                        │
│                                                                                        │
│  Predictions and Interval Summary                                                      │
│                                                                                        │
│  The forecast provides predictions (pred) along with lower and upper bounds for an     │
│  eighty percent prediction interval (0.1 to 0.9) calculated using bootstrapping.       │
│                                                                                        │
│  Across the entire 36-step forecast horizon, the summary statistics for the            │
│  predictions and bounds are:                                                           │
│                                                                                        │
│   • Predictions (pred): The minimum predicted value is 5.258729928204422, the maximum  │
│     is 260.7711523801768, and the mean is 103.19879814283314.                          │
│   • Lower Bound: The minimum lower bound is 0.44880061218456635, the maximum is        │
│     142.57459533864323, and the mean is 61.74293153975548.                             │
│   • Upper Bound: The minimum upper bound is 12.096545115409352, the maximum is         │
│     352.6076131902337, and the mean is 137.95308075514998.                             │
│                                                                                        │
│  For the initial period, the forecast starts at 2012-12-30 12:00:00 with a prediction  │
│  of 146.512223 (interval: 111.097550 to 181.185404). For the final period of the       │
│  horizon, the forecast ends at 2012-12-31 23:00:00 with a prediction of 39.970777      │
│  (interval: 17.809739 to 59.555964). Since 26 interior rows were omitted from the      │
│  provided context, overall trends or step-by-step progressions across the complete     │
│  horizon are not analyzed.                                                             │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯

Refining the modeling plan¶

Beyond explaining results, the assistant can also help you to improve the modeling strategy using the refine_plan() method. This method accepts an existing profile and plan and returns an updated ForecastPlan. It operates in two distinct modes:

• Deterministic mode (Explicit Overrides): When no prompt is provided, you can pass explicit configuration overrides (such as forecaster, estimator, estimator_kwargs, steps, interval, lags, or window_features). Only the explicitly specified fields are updated; the remaining configuration is deterministically re-derived from the original plan.

• LLM mode (Domain Knowledge Integration): When a prompt is provided, the assistant leverages the LLM to interpret your natural-language domain knowledge. The agent suggests domain-specific lags and window_features, which are then merged into the plan. To maintain strict interpretability, the LLM’s reasoning is appended to the returned plan's explanation attribute, allowing you to trace exactly why each feature was proposed.

The following example demonstrates the LLM mode. By supplying a prompt with context about the dataset, we instruct the assistant to engineer more appropriate lags and window features. We then execute forecast() using the refined plan and compare the new metrics against our original baseline run.

Keep in mind that a refined plan is a hypothesis, not a guaranteed improvement. The LLM proposes lags and window features from the domain knowledge you provide, but only a proper evaluation can confirm whether they actually help. Always compare the refined plan against the original before adopting it, and check that the suggested features are sensible for your data and use case rather than trusting them blindly.

In [12]:

# Ask the assistant to improve the forecast accuracy
# ==============================================================================
prompt  = (
    "I'm forecasting hourly bike rentals. Demand follows a clear daily rhythm with "
    "rush-hour peaks, and it changes between weekdays and weekends. It's also usually "
    "similar to what happened at the same time last week, and the last few hours give "
    "a good sense of the current trend. Please pick lags and rolling features that fit this."
)

refined_plan = assistant.refine_plan(
                   profile = results_pred.profile,
                   plan    = results_pred.plan,
                   prompt  = prompt
               )

# Ask the assistant to improve the forecast accuracy # ============================================================================== prompt = ( "I'm forecasting hourly bike rentals. Demand follows a clear daily rhythm with " "rush-hour peaks, and it changes between weekdays and weekends. It's also usually " "similar to what happened at the same time last week, and the last few hours give " "a good sense of the current trend. Please pick lags and rolling features that fit this." ) refined_plan = assistant.refine_plan( profile = results_pred.profile, plan = results_pred.plan, prompt = prompt )

In [13]:

# Refined plan proposed by the assistant
# ==============================================================================
refined_plan

# Refined plan proposed by the assistant # ============================================================================== refined_plan

Out[13]:

                                      Forecast Plan                                       
┏━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Property          ┃ Value                                                              ┃
┡━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Task type         │ single_series                                                      │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Forecaster        │ ForecasterRecursive                                                │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Estimator         │ LGBMRegressor                                                      │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Steps             │ 36                                                                 │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Frequency         │ h                                                                  │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Lags              │ [1, 2, 3, 24, 48, 168]  (LLM-suggested)                            │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Window features   │ [{'stats': ['mean', 'std'], 'window_size': 3}, {'stats': ['mean',  │
│                   │ 'std'], 'window_size': 24}, {'stats': ['mean'], 'window_size':     │
│                   │ 168}]  (LLM-suggested)                                             │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Calendar features │ ['hour', 'day_of_week', 'weekend', 'month'] (raw ordinal encoding) │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Use exog          │ True                                                               │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Interval          │ [0.1, 0.9]                                                         │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Interval method   │ bootstrapping                                                      │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Primary metric    │ mean_absolute_error                                                │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Preprocessing     │ 1 step                                                             │
└───────────────────┴────────────────────────────────────────────────────────────────────┘

                                   Preprocessing Steps                                    
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Step                    ┃ Reason                                                       ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ handle_categorical_exog │ Categorical exogenous variables detected: ['weather']. These │
│                         │ are handled automatically by skforecast                      │
│                         │ (categorical_features='auto').                               │
└─────────────────────────┴──────────────────────────────────────────────────────────────┘

╭─────────────────────────────────── Plan Explanation ───────────────────────────────────╮
│                                                                                        │
│  Plan: ForecasterRecursive + LGBMRegressor. Lags: [1, 2, 3, 24, 48, 168]. Window       │
│  features: ['mean(window=3)', 'std(window=3)', 'mean(window=24)', 'std(window=24)',    │
│  'mean(window=168)']. Calendar features: ['hour', 'day_of_week', 'weekend', 'month']   │
│  (raw ordinal encoding). Prediction intervals via bootstrapping. NaN rows kept         │
│  (NaN-tolerant estimator). Exogenous variables included. MAE is interpretable, robust  │
│  to outliers, and works at any scale.                                                  │
│                                                                                        │
│  LLM Refinement Reasoning: Based on the characteristics of hourly bike rentals:        │
│                                                                                        │
│   1 To capture the short-term trend and recent momentum ('the last few hours give a    │
│     good sense of the current trend'), we selected lags 1, 2, and 3, alongside a       │
│     short-term rolling window of size 3 (mean and standard deviation) to capture       │
│     immediate demand level and volatility.                                             │
│   2 To capture the daily rhythm and rush-hour peaks, we included lag 24 and lag 48,    │
│     paired with a rolling window of size 24 (mean and standard deviation) to track     │
│     the daily baseline demand and hourly volatility.                                   │
│   3 To address the weekly seasonality and weekday/weekend patterns, we included lag    │
│     168 (the exact same hour of the same day last week) and a rolling window of size   │
│     168 (mean) to represent the longer-term weekly trend.                              │
│                                                                                        │
│  Note: the LLM-suggested lags and window_features are hypotheses, not validated        │
│  improvements. Confirm any expected accuracy gain before relying on them.              │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯

In [14]:

# Run the forecast with the refined plan 
# ==============================================================================
results_refined = assistant.forecast(
                      data        = data,
                      target      = 'users',
                      date_column = "date_time",
                      steps       = 36,
                      test_size   = 36,           # Last 36 hours as test set (evaluation mode)
                      plan        = refined_plan  # Refined plan proposed by the assistant
                  )

# Run the forecast with the refined plan # ============================================================================== results_refined = assistant.forecast( data = data, target = 'users', date_column = "date_time", steps = 36, test_size = 36, # Last 36 hours as test set (evaluation mode) plan = refined_plan # Refined plan proposed by the assistant )

In [15]:

# Results of the forecast with the refined plan
# ==============================================================================
results_refined.metrics

# Results of the forecast with the refined plan # ============================================================================== results_refined.metrics

Out[15]:

	series	MAE	MSE	MASE	MAPE
0	users	46.574232	4934.245392	0.723323	0.525906

In [16]:

# Results of the forecast with the original plan
# ==============================================================================
results.metrics

# Results of the forecast with the original plan # ============================================================================== results.metrics

Out[16]:

	series	MAE	MSE	MASE	MAPE
0	users	32.013374	2758.8615	0.497185	0.415688

As noted above, a refined plan is a hypothesis, not a guaranteed improvement. The LLM may propose lags or window features that are not relevant to the series, or it may misread the domain knowledge you provided, so the refined plan can perform worse than the original.

More importantly, here we evaluate the change on a single 36-hour test window, which is far too small to draw a reliable conclusion. A single split can favor either plan by chance. To decide whether the refinement genuinely helps, evaluate both plans with a backtest over multiple folds (see the next section), which averages performance across many test windows and gives a far more trustworthy comparison.

Backtesting¶

In time series forecasting, backtesting is the process of evaluating a predictive model by retrospectively simulating its performance on historical data. It functions as a specialized form of temporal cross-validation, ensuring that data leakage is prevented by strictly respecting the chronological order of observations.

The reliability of backtesting metrics depends entirely on how closely the evaluation setup mirrors production conditions. To obtain trustworthy performance estimates, the backtesting strategy must accurately reflect the real-world prediction horizon, refit frequency, and data availability.

In skforecast-ai, this validation strategy is governed by the TimeSeriesFold object. It defines exactly how the historical data is partitioned into successive training and test windows. It controls critical parameters such as the forecast horizon (steps), the starting point for evaluation (initial_train_size), and whether the model is periodically retrained as the window rolls forward (refit). Because these choices dictate the interpretation of the resulting metrics, correctly configuring the TimeSeriesFold is the most critical decision in the evaluation phase. For a comprehensive overview of these mechanics, refer to the skforecast Backtesting user guide.

skforecast-ai provides three distinct ways to define this validation strategy, ranging from full manual control to automated generation:

• Explicit Instantiation (Recommended): Manually construct a TimeSeriesFold and pass it directly to the backtest() method. If you already know the exact operational constraints of your production environment, this is the most explicit and reproducible approach, keeping the validation setup entirely under your control.

• LLM Mode (create_cv() with a prompt): Describe your operational use case in natural language (e.g., required horizon, retraining frequency, trusted historical depth). The LLM reasoning layer interprets this context and translates it into a strictly defined TimeSeriesFold, materializing the setup into deterministic code.

• Deterministic Mode (create_cv() without a prompt): Allow the assistant to automatically derive a sensible TimeSeriesFold from the existing profile and plan using rule-based defaults. You can still explicitly override individual parameters such as initial_train_size or refit as needed.

The following example demonstrates the three approaches.

In [17]:

# Explicit Instantiation of the cv
# ==============================================================================
# Create your own TimeSeriesFold object
end_train = '2012-08-31 23:59:00'
cv = TimeSeriesFold(
         steps              = 36, 
         initial_train_size = end_train, 
         refit              = False
     )

results_backtest = assistant.backtest(
                       data        = data,
                       target      = 'users',
                       date_column = "date_time",
                       cv          = cv,          # TimeSeriesFold object
                       interval    = [0.1, 0.9],  # 80% prediction interval
                   )

# Explicit Instantiation of the cv # ============================================================================== # Create your own TimeSeriesFold object end_train = '2012-08-31 23:59:00' cv = TimeSeriesFold( steps = 36, initial_train_size = end_train, refit = False ) results_backtest = assistant.backtest( data = data, target = 'users', date_column = "date_time", cv = cv, # TimeSeriesFold object interval = [0.1, 0.9], # 80% prediction interval )

  0%|          | 0/82 [00:00<?, ?it/s]

In [18]:

# Show backtest results
# ==============================================================================
results_backtest.show_explanation()
display(results_backtest.metrics)
display(results_backtest.predictions.head())

# Show backtest results # ============================================================================== results_backtest.show_explanation() display(results_backtest.metrics) display(results_backtest.predictions.head())

╭───────────────────────────────── Backtest Explanation ─────────────────────────────────╮
│                                                                                        │
│  Initial training up to 2012-08-31 23:59:00, fixed window, no refit, 36-step horizon,  │
│  82 folds. Results — mean_absolute_error: 46.3122, mean_squared_error: 5495.8816,      │
│  mean_absolute_scaled_error: 0.7496, mean_absolute_percentage_error: 0.4727.           │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯

	mean_absolute_error	mean_squared_error	mean_absolute_scaled_error	mean_absolute_percentage_error
0	46.312218	5495.881638	0.749598	0.472735

	fold	pred	lower_bound	upper_bound
2012-09-01 00:00:00	0	124.163668	99.669766	149.384335
2012-09-01 01:00:00	0	96.989454	68.523139	127.651105
2012-09-01 02:00:00	0	65.894182	32.969786	97.883129
2012-09-01 03:00:00	0	29.961028	10.347714	60.808462
2012-09-01 04:00:00	0	9.103018	2.980393	28.085362

After running the workflow, you can ask the assistant to explain the backtesting results. The ask() method can take a BacktestResult object and provide insights about the strategy, the metrics, or any other relevant information.

In [19]:

# Asking the assistant about the backtesting
# ==============================================================================
explanation = assistant.ask(
    prompt = "Explain the results of the backtesting, including the strategy, metrics, and predictions.",
    result = results_backtest
)

# Asking the assistant about the backtesting # ============================================================================== explanation = assistant.ask( prompt = "Explain the results of the backtesting, including the strategy, metrics, and predictions.", result = results_backtest )

/var/folders/wt/8tvn563d5v55nspfbydgqb9r0000gp/T/ipykernel_48657/3622928719.py:3: DataSentToLLMWarning: `send_data_to_llm=False` does not apply to `result`: the predicted values it carries are sent to the LLM, because a question about a result cannot be answered from summary statistics alone. Your input data is not sent: a result holds only the model's output, never the data it was fitted on. To keep predictions local, ask without `result`.
  explanation = assistant.ask(

In [20]:

# Show the explanation generated by the LLM
# ==============================================================================
explanation.show_explanation()

# Show the explanation generated by the LLM # ============================================================================== explanation.show_explanation()

╭────────────────────────────────── Assistant Response ──────────────────────────────────╮
│                                                                                        │
│  The backtesting was conducted using a recursive machine learning forecaster with a    │
│  LightGBM regressor to predict the hourly target variable over 82 evaluation folds.    │
│  The validation evaluated a 36-step-ahead horizon with a fixed training window and no  │
│  model retraining, achieving a Mean Absolute Error of 46.3122. This performance        │
│  represents a solid predictive accuracy, outperforming the naive baseline.             │
│                                                                                        │
│  Backtesting Strategy                                                                  │
│                                                                                        │
│  The evaluation used a chronological validation setup with the following parameters:   │
│                                                                                        │
│   • Forecaster type: ForecasterRecursive with an LGBMRegressor estimator.              │
│   • Initial training size: Data up to 2012-08-31 23:59:00.                             │
│   • Refit policy: False, meaning the model was trained once on the initial training    │
│     data and was not retrained during backtesting.                                     │
│   • Window style: Fixed training size.                                                 │
│   • Horizon (steps): 36 steps per fold.                                                │
│   • Gap: 0, indicating no delay between the training data and forecast start.          │
│   • Fold stride: 36, meaning non-overlapping evaluation windows.                       │
│   • Number of folds: 82 folds.                                                         │
│                                                                                        │
│  The forecaster incorporates several input features, including specific historical     │
│  lags, window statistics (rolling mean and standard deviation over various windows),   │
│  calendar features (hour, day of week, weekend, and month), and three exogenous        │
│  variables (holiday, weather, and temp). Categorical features, such as the weather     │
│  variable, were managed automatically.                                                 │
│                                                                                        │
│  Evaluation Metrics                                                                    │
│                                                                                        │
│  The backtesting yielded the following metrics:                                        │
│                                                                                        │
│   • Mean Absolute Error (MAE): 46.312218, showing the average absolute deviation of    │
│     predictions from actual values.                                                    │
│   • Mean Squared Error (MSE): 5495.881638, which penalizes larger errors.              │
│   • Mean Absolute Scaled Error (MASE): 0.749598. Because this metric is below 1, it    │
│     demonstrates that the model outperformed the naive baseline.                       │
│   • Mean Absolute Percentage Error (MAPE): 0.472735.                                   │
│                                                                                        │
│  Predictions and Uncertainty                                                           │
│                                                                                        │
│  A total of 2928 predictions were generated across the backtesting folds. The point    │
│  forecasts and their corresponding 80% prediction intervals, calculated using          │
│  bootstrapping, show the following overall characteristics:                            │
│                                                                                        │
│   • Point forecasts (pred): Had a mean value of 235.999687947411, with a minimum       │
│     prediction of 1.9345238001281464 and a maximum prediction of 881.3578535350078.    │
│   • Lower bounds: Had a mean of 193.18431772887837, with a minimum value of            │
│     -2.1351855197660417 and a maximum value of 827.8249722414072.                      │
│   • Upper bounds: Had a mean of 283.5445453993996, with a minimum value of             │
│     4.739552115733364 and a maximum value of 914.3772544676444.                        │
│                                                                                        │
│  These intervals represent the estimated range where the actual value is expected to   │
│  fall with an 80% probability.                                                         │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯

Rather than manually configuring TimeSeriesFold parameters, you can describe your target backtesting strategy in natural language and allow the assistant's LLM layer to translate it into a rigorous cross-validation schema. Once you have generated a data profile and a modeling plan, simply pass a prompt detailing your operational constraints, such as the required forecast horizon, historical training depth, refitting frequency, or expected data gaps. The assistant processes this context and returns a fully configured TimeSeriesFold object, accompanied by an explicit explanation of its design choices. This transparency ensures you can thoroughly audit and verify the validation strategy before executing the backtest.

In [21]:

# Let the assistant create the TimeSeriesFold for you
# ==============================================================================
# Profile and create a plan for your data
profile = assistant.profile(data, target="users", date_column="date_time")
plan = assistant.plan(profile, steps=36)

# Call create_cv with a prompt that describes your prediction strategy
prompt = (
    "I forecast bike demand 36 hours ahead. "
    "The model should be trained once on all data up to the end of August 2012, 23:59. "
    "Do not refit the model as the window rolls forward."
)
cv, cv_explanation = assistant.create_cv(
                         profile = profile,
                         plan    = plan,
                         prompt  = prompt
                     )

# Let the assistant create the TimeSeriesFold for you # ============================================================================== # Profile and create a plan for your data profile = assistant.profile(data, target="users", date_column="date_time") plan = assistant.plan(profile, steps=36) # Call create_cv with a prompt that describes your prediction strategy prompt = ( "I forecast bike demand 36 hours ahead. " "The model should be trained once on all data up to the end of August 2012, 23:59. " "Do not refit the model as the window rolls forward." ) cv, cv_explanation = assistant.create_cv( profile = profile, plan = plan, prompt = prompt )

In [22]:

# TimeSeriesFold object
# ==============================================================================
cv

# TimeSeriesFold object # ============================================================================== cv

Out[22]:

TimeSeriesFold

General Information

• Initial train size: 2012-08-31 23:59:00
• Initial train size as int: 14616
• Steps: 36
• Fold stride: 36
• Overlapping folds: False
• Window size: None
• Differentiation: None
• Refit: False
• Fixed train size: False
• Gap: 0
• Skip folds: None
• Allow incomplete fold: True
• Return all indexes: False

📖 API Reference    📝 User Guide

In [23]:

# LLM Reasoning
# ==============================================================================
import textwrap
print(textwrap.fill(cv_explanation, width=88))

# LLM Reasoning # ============================================================================== import textwrap print(textwrap.fill(cv_explanation, width=88))

The model needs to be trained exactly once up to the end of August 2012 (2012-08-31
23:59:00) and evaluated as the window rolls forward without any retraining. This maps to
refit=False and setting the initial_train_size as a date string corresponding to the
user's cutoff. This setup leaves enough observations in the remaining months (September
to December 2012) to easily satisfy the minimum of 2 folds for a 36-hour horizon.
Initial training up to 2012-08-31 23:59:00, expanding window, no refit, 36-step horizon,
82 folds.

Since the prompt correctly describes the intended use case, the cv object returned by create_cv() is the same as the one we built manually above. However it was derived from a natural-language description rather than explicit parameters. The assistant also returns a cv_explanation string that details the choices it made. This allows you to verify that the resulting TimeSeriesFold matches your intended strategy.

The backtesting workflow returns a BacktestResult, a lightweight container that bundles everything the assistant used and produced, so you can inspect the outputs, audit the decisions, or lift the code straight into production.

Attribute	Type	Description
predictions	DataFrame	Full out-of-sample backtest predictions across all folds. When intervals (or quantiles) are requested, the bound columns are included alongside the point predictions.
metrics	DataFrame	Backtesting metrics (MAE, MSE, MASE) returned by skforecast, one row per series, computed over the reserved test window.
cv_config	dict	The resolved TimeSeriesFold parameters (steps, initial train size, refit, gap, etc.), kept for full traceability of the validation strategy.
code	str	The exact standalone skforecast script that reproduces the backtesting workflow, deterministic and ready to run on its own.
explanation	str	A human-readable summary of the backtesting configuration and what the results mean.
profile	ForecastingProfile	The data profile behind the backtest: metadata, summary statistics, detected frequency and seasonality, and the high-level modeling decisions.
plan	ForecastPlan	The detailed configuration that was executed: forecaster, estimator, lags, window features, preprocessing, and interval settings.

Displaying the object renders a rich summary of all of the above; the raw script is also available through results_backtest.show_code().

In [24]:

# Full results object
# ==============================================================================
results_backtest

# Full results object # ============================================================================== results_backtest

Out[24]:

╭───────────────────────────────── Backtest Explanation ─────────────────────────────────╮
│                                                                                        │
│  Initial training up to 2012-08-31 23:59:00, fixed window, no refit, 36-step horizon,  │
│  82 folds. Results — mean_absolute_error: 46.3122, mean_squared_error: 5495.8816,      │
│  mean_absolute_scaled_error: 0.7496, mean_absolute_percentage_error: 0.4727.           │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯
       Cross-Validation Configuration       
┏━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃ Parameter          ┃               Value ┃
┡━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ steps              │                  36 │
├────────────────────┼─────────────────────┤
│ initial_train_size │ 2012-08-31 23:59:00 │
├────────────────────┼─────────────────────┤
│ refit              │               False │
├────────────────────┼─────────────────────┤
│ fixed_train_size   │                True │
├────────────────────┼─────────────────────┤
│ gap                │                   0 │
├────────────────────┼─────────────────────┤
│ fold_stride        │                  36 │
├────────────────────┼─────────────────────┤
│ differentiation    │                None │
├────────────────────┼─────────────────────┤
│ n_folds            │                  82 │
└────────────────────┴─────────────────────┘
                                     Backtest Metrics                                     
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┓
┃ mean_absolute_error ┃ mean_squared_error ┃ mean_absolute_scale… ┃ mean_absolute_perce… ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━┩
│             46.3122 │          5495.8816 │               0.7496 │               0.4727 │
└─────────────────────┴────────────────────┴──────────────────────┴──────────────────────┘
                    Backtest Predictions (2928 rows)                    
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃ Index               ┃    fold ┃     pred ┃ lower_bound ┃ upper_bound ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│ 2012-09-01 00:00:00 │  0.0000 │ 124.1637 │     99.6698 │    149.3843 │
├─────────────────────┼─────────┼──────────┼─────────────┼─────────────┤
│ 2012-09-01 01:00:00 │  0.0000 │  96.9895 │     68.5231 │    127.6511 │
├─────────────────────┼─────────┼──────────┼─────────────┼─────────────┤
│ 2012-09-01 02:00:00 │  0.0000 │  65.8942 │     32.9698 │     97.8831 │
├─────────────────────┼─────────┼──────────┼─────────────┼─────────────┤
│ 2012-09-01 03:00:00 │  0.0000 │  29.9610 │     10.3477 │     60.8085 │
├─────────────────────┼─────────┼──────────┼─────────────┼─────────────┤
│ 2012-09-01 04:00:00 │  0.0000 │   9.1030 │      2.9804 │     28.0854 │
├─────────────────────┼─────────┼──────────┼─────────────┼─────────────┤
│ ...                 │     ... │      ... │         ... │         ... │
├─────────────────────┼─────────┼──────────┼─────────────┼─────────────┤
│ 2012-12-31 19:00:00 │ 81.0000 │ 145.8848 │     97.1293 │    180.5958 │
├─────────────────────┼─────────┼──────────┼─────────────┼─────────────┤
│ 2012-12-31 20:00:00 │ 81.0000 │ 103.3326 │     67.3546 │    140.2631 │
├─────────────────────┼─────────┼──────────┼─────────────┼─────────────┤
│ 2012-12-31 21:00:00 │ 81.0000 │  64.1944 │     33.2954 │     86.5947 │
├─────────────────────┼─────────┼──────────┼─────────────┼─────────────┤
│ 2012-12-31 22:00:00 │ 81.0000 │  46.9473 │     23.0095 │     64.8209 │
├─────────────────────┼─────────┼──────────┼─────────────┼─────────────┤
│ 2012-12-31 23:00:00 │ 81.0000 │  30.5361 │     12.3134 │     45.8523 │
└─────────────────────┴─────────┴──────────┴─────────────┴─────────────┘
                          Dataset Profile                          
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Property       ┃ Value                                          ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Format         │ single                                         │
├────────────────┼────────────────────────────────────────────────┤
│ Series         │ 1                                              │
├────────────────┼────────────────────────────────────────────────┤
│ Observations   │ 17544                                          │
├────────────────┼────────────────────────────────────────────────┤
│ Frequency      │ h                                              │
├────────────────┼────────────────────────────────────────────────┤
│ Target         │ users                                          │
├────────────────┼────────────────────────────────────────────────┤
│ Exog columns   │ holiday, weather, temp  (categorical: weather) │
├────────────────┼────────────────────────────────────────────────┤
│ Missing values │ None                                           │
└────────────────┴────────────────────────────────────────────────┘

                                    Recommendation                                     
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Property              ┃ Value                                                       ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Task type             │ single_series                                               │
├───────────────────────┼─────────────────────────────────────────────────────────────┤
│ Forecaster            │ ForecasterRecursive                                         │
├───────────────────────┼─────────────────────────────────────────────────────────────┤
│ Forecaster candidates │ ForecasterRecursive, ForecasterDirect, ForecasterFoundation │
├───────────────────────┼─────────────────────────────────────────────────────────────┤
│ Estimator             │ LGBMRegressor                                               │
├───────────────────────┼─────────────────────────────────────────────────────────────┤
│ Estimator candidates  │ LGBMRegressor, XGBRegressor, Ridge                          │
└───────────────────────┴─────────────────────────────────────────────────────────────┘

╭───────────────────────────────── Profile Explanation ──────────────────────────────────╮
│                                                                                        │
│  A single-series ML forecaster (ForecasterRecursive) is recommended. Data: 17544       │
│  observations, 'h' frequency. Alternative forecasters: ['ForecasterDirect',            │
│  'ForecasterFoundation']. Estimator: LGBMRegressor. A gradient boosting model is       │
│  preferred for a dataset of this size (17544 observations). Alternative estimators:    │
│  ['XGBRegressor', 'Ridge']. 3 exogenous variables (1 categorical) available as         │
│  predictors.                                                                           │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯
                                      Forecast Plan                                       
┏━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Property          ┃ Value                                                              ┃
┡━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Task type         │ single_series                                                      │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Forecaster        │ ForecasterRecursive                                                │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Estimator         │ LGBMRegressor                                                      │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Steps             │ 36                                                                 │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Frequency         │ h                                                                  │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Lags              │ [1, 2, 3, 5, 8, 10, 15, 17, 19, 20, 21, 22, 23, 24, 25, 26, 32,    │
│                   │ 33, 119, 121, 135, 136, 142, 143, 145, 160, 166, 167, 169, 313,    │
│                   │ 335, 337]                                                          │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Window features   │ [{'stats': ['mean', 'std'], 'window_size': 3}, {'stats': ['mean'], │
│                   │ 'window_size': 24}, {'stats': ['mean'], 'window_size': 168}]       │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Calendar features │ ['hour', 'day_of_week', 'weekend', 'month'] (raw ordinal encoding) │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Use exog          │ True                                                               │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Interval          │ [0.1, 0.9]                                                         │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Interval method   │ bootstrapping                                                      │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Primary metric    │ mean_absolute_error                                                │
├───────────────────┼────────────────────────────────────────────────────────────────────┤
│ Preprocessing     │ 1 step                                                             │
└───────────────────┴────────────────────────────────────────────────────────────────────┘

                                   Preprocessing Steps                                    
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Step                    ┃ Reason                                                       ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ handle_categorical_exog │ Categorical exogenous variables detected: ['weather']. These │
│                         │ are handled automatically by skforecast                      │
│                         │ (categorical_features='auto').                               │
└─────────────────────────┴──────────────────────────────────────────────────────────────┘

╭─────────────────────────────────── Plan Explanation ───────────────────────────────────╮
│                                                                                        │
│  Plan: ForecasterRecursive + LGBMRegressor. Lags: [1, 2, 3, 5, 8, 10, 15, 17, 19, 20,  │
│  21, 22, 23, 24, 25, 26, 32, 33, 119, 121, 135, 136, 142, 143, 145, 160, 166, 167,     │
│  169, 313, 335, 337]. Window features: ['mean(window=3)', 'std(window=3)',             │
│  'mean(window=24)', 'mean(window=168)']. Calendar features: ['hour', 'day_of_week',    │
│  'weekend', 'month'] (raw ordinal encoding). Prediction intervals via bootstrapping.   │
│  NaN rows kept (NaN-tolerant estimator). Exogenous variables included. MAE is          │
│  interpretable, robust to outliers, and works at any scale.                            │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯

Generated codeCopy

import pandas as pd                                                                       
from lightgbm import LGBMRegressor                                                        
from skforecast.preprocessing import RollingFeatures, CalendarFeatures                    
from skforecast.recursive import ForecasterRecursive                                      
from skforecast.model_selection import TimeSeriesFold, backtesting_forecaster             
                                                                                          
# Load data                                                                               
data = pd.read_csv('data.csv')                                                            
                                                                                          
data['date_time'] = pd.to_datetime(data['date_time'])                                     
data = data.set_index('date_time')                                                        
data = data.asfreq('h')                                                                   
data = data.sort_index()                                                                  
                                                                                          
window_features = RollingFeatures(                                                        
    stats        = ['mean', 'std', 'mean', 'mean'],                                       
    window_sizes = [3, 3, 24, 168],                                                       
)                                                                                         
                                                                                          
calendar_features = CalendarFeatures(                                                     
    features = ['hour', 'day_of_week', 'weekend', 'month'],                               
    encoding = None,                                                                      
)                                                                                         
                                                                                          
# Create forecaster                                                                       
forecaster = ForecasterRecursive(                                                         
    estimator            = LGBMRegressor(random_state=123, verbose=-1),                   
    lags                 = [1, 2, 3, 5, 8, 10, 15, 17, 19, 20, 21, 22, 23, 24, 25, 26, 32,
33, 119, 121, 135, 136, 142, 143, 145, 160, 166, 167, 169, 313, 335, 337],                
    window_features      = window_features,                                               
    calendar_features    = calendar_features,                                             
    categorical_features = 'auto',                                                        
    dropna_from_series   = False,                                                         
)                                                                                         
                                                                                          
# Time series cross-validation configuration                                              
cv = TimeSeriesFold(                                                                      
    steps              = 36,                                                              
    initial_train_size = '2012-08-31 23:59:00',                                           
    refit              = False,                                                           
)                                                                                         
                                                                                          
# Run backtesting                                                                         
exog_features = ['holiday', 'weather', 'temp']                                            
                                                                                          
metrics, predictions = backtesting_forecaster(                                            
    forecaster        = forecaster,                                                       
    y                 = data['users'],                                                    
    exog              = data[exog_features],                                              
    cv                = cv,                                                               
    metric            = ['mean_absolute_error', 'mean_squared_error',                     
'mean_absolute_scaled_error', 'mean_absolute_percentage_error'],                          
    interval          = [0.1, 0.9],                                                       
    n_jobs            = 'auto',                                                           
    verbose           = False,                                                            
    show_progress     = True,                                                             
    suppress_warnings = True,                                                             
)                                                                                         
                                                                                          
print(metrics)                                                                            
print(predictions.head())                                                                 

Comparing forecaster configurations¶

Choosing a forecasting model should not rely on intuition alone. Two configurations that look equally reasonable can perform very differently once evaluated on real temporal data. The most reliable approach is to test every candidate under identical conditions and compare their metrics.

The compare() method does exactly that. It receives a list of candidate configurations, backtests each one using the same TimeSeriesFold strategy, and returns a leaderboard ranked by the selected metric.

Candidates can be provided in two ways:

• Automatic candidates (candidates=None): the assistant builds the comparison set from profile.forecaster_candidates, using the forecaster types identified as suitable during profiling. This is useful when exploring a new dataset without a predefined shortlist.

• Explicit candidates (recommended): pass a list of (name, config) tuples, where name labels the row in the leaderboard and config holds the same override keys understood by plan(): 'forecaster', 'estimator', 'estimator_kwargs', 'lags' and 'window_features'. This provides full control and makes the resulting table easier to interpret.

A failed candidate does not stop the comparison. Instead, a CandidateFailedWarning is issued, the row records the error and is placed last.

The following examples demonstrate both approaches.

💡 Tip

All candidates use the same cross-validation strategy, ensuring a fair comparison. However, the results are only meaningful if the cv setup reflects the real use case where the model will be deployed. For example, if the production system retrains weekly, the backtest should also refit weekly. If the model is expected to forecast 24 hours ahead, the backtest should use a 24-hour horizon.

The evaluation window must also be representative. A period that is too short or dominated by unusual events (holidays, outages, or exceptional peaks) may favor a candidate that performs poorly over time. Define the validation setup carefully before comparing models so the final ranking is reliable.

In [ ]:

# Comparison of the forecaster candidates suggested by the profile
# ==============================================================================
end_train = '2012-08-31 23:59:00'
cv = TimeSeriesFold(
         steps              = 36,
         initial_train_size = end_train,
         refit              = False,
     )

results_compare = assistant.compare(
                      data        = data,
                      target      = 'users',
                      date_column = "date_time",
                      cv          = cv,   # Same TimeSeriesFold for every candidate
                      candidates  = None  # Candidates suggested by the assistant 
                  )

# Comparison of the forecaster candidates suggested by the profile # ============================================================================== end_train = '2012-08-31 23:59:00' cv = TimeSeriesFold( steps = 36, initial_train_size = end_train, refit = False, ) results_compare = assistant.compare( data = data, target = 'users', date_column = "date_time", cv = cv, # Same TimeSeriesFold for every candidate candidates = None # Candidates suggested by the assistant )

Comparing forecasters:   0%|          | 0/3 [00:00<?, ?it/s]

In [26]:

# Ranked leaderboard
# ==============================================================================
results_compare.results

# Ranked leaderboard # ============================================================================== results_compare.results

Out[26]:

	rank	name	forecaster	estimator	mean_absolute_error	mean_squared_error	mean_absolute_scaled_error	mean_absolute_percentage_error
0	1	ForecasterFoundation	ForecasterFoundation	Chronos-2	38.436160	4231.326474	0.597467	0.611349
1	2	ForecasterRecursive	ForecasterRecursive	LGBMRegressor	46.312218	5495.881638	0.749598	0.472735
2	3	ForecasterDirect	ForecasterDirect	LGBMRegressor	49.298312	5936.622526	0.797930	0.499413

In [27]:

# Deterministic summary of the comparison
# ==============================================================================
results_compare.show_explanation()

# Deterministic summary of the comparison # ============================================================================== results_compare.show_explanation()

╭──────────────────────────────── Comparison Explanation ────────────────────────────────╮
│                                                                                        │
│  Compared 3 configurations, ranked ascending by mean_absolute_error. Shared            │
│  cross-validation strategy: Initial training up to 2012-08-31 23:59:00, fixed window,  │
│  no refit, 36-step horizon, 82 folds. Best: 'ForecasterFoundation'                     │
│  (ForecasterFoundation / Chronos-2) = 38.4362, 17.0% ahead of 'ForecasterRecursive'    │
│  (46.3122).                                                                            │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯

In practice, you will often already have a shortlist in mind: a fast baseline, a gradient boosting model, or a variant with a richer feature set. Passing explicit (name, config) tuples keeps the comparison focused and makes the resulting leaderboard easy to understand at a glance.

The config dictionary accepts the same overrides as plan(). Any omitted option falls back to the deterministic recommendation derived from the dataset profile, so candidates can remain concise. For example, {'forecaster': 'ForecasterDirect'} changes only the forecaster while keeping the recommended estimator, lags, and features.

⚠️ Computational cost

Each candidate is backtested independently across all folds, so runtime increases with both the number and complexity of the configurations. Comparing four candidates will take roughly four times as long as running one backtest.

Start with a small set of clearly different options, review the results, and refine from there. Testing many near-identical variants is costly and rarely useful.

In [ ]:

# Comparison of an explicit shortlist of configurations
# ==============================================================================
candidates = [
    (
        "ridge_baseline",
        {
            "forecaster": "ForecasterRecursive",
            "estimator" : "Ridge",
            "lags"      : 24,
        }
    ),
    (
        "lgbm_daily_lags",
        {
            "forecaster": "ForecasterRecursive",
            "estimator" : "LGBMRegressor",
            "lags"      : 24,
        }
    ),
    (
        "lgbm_weekly_lags_rolling",
        {
            "forecaster"      : "ForecasterRecursive",
            "estimator"       : "LGBMRegressor",
            "lags"            : [1, 2, 3, 23, 24, 25, 167, 168, 169],
            "window_features" : [{"stats": ["mean", "std"], "window_size": 24}],
        }
    ),
    (
        "lgbm_direct",
        {
            "forecaster": "ForecasterDirect",
            "estimator" : "LGBMRegressor",
            "lags"      : 24,
        }
    ),
    (
        "foundation_model",
        {
            "forecaster": "ForecasterFoundation"
        }
    ),
]

results_compare = assistant.compare(
                      data        = data,
                      target      = 'users',
                      date_column = "date_time",
                      cv          = cv,
                      candidates  = candidates,  # Specific candidates to compare
                      metric      = ['mean_absolute_error', 'mean_absolute_scaled_error'],
                  )

# Comparison of an explicit shortlist of configurations # ============================================================================== candidates = [ ( "ridge_baseline", { "forecaster": "ForecasterRecursive", "estimator" : "Ridge", "lags" : 24, } ), ( "lgbm_daily_lags", { "forecaster": "ForecasterRecursive", "estimator" : "LGBMRegressor", "lags" : 24, } ), ( "lgbm_weekly_lags_rolling", { "forecaster" : "ForecasterRecursive", "estimator" : "LGBMRegressor", "lags" : [1, 2, 3, 23, 24, 25, 167, 168, 169], "window_features" : [{"stats": ["mean", "std"], "window_size": 24}], } ), ( "lgbm_direct", { "forecaster": "ForecasterDirect", "estimator" : "LGBMRegressor", "lags" : 24, } ), ( "foundation_model", { "forecaster": "ForecasterFoundation" } ), ] results_compare = assistant.compare( data = data, target = 'users', date_column = "date_time", cv = cv, candidates = candidates, # Specific candidates to compare metric = ['mean_absolute_error', 'mean_absolute_scaled_error'], )

Comparing forecasters:   0%|          | 0/5 [00:00<?, ?it/s]

When several metrics are requested, all of them are shown as columns but only the first one drives the ranking.

In [29]:

# Ranked leaderboard, sorted by the first metric requested
# ==============================================================================
results_compare.results

# Ranked leaderboard, sorted by the first metric requested # ============================================================================== results_compare.results

Out[29]:

	rank	name	forecaster	estimator	mean_absolute_error	mean_absolute_scaled_error
0	1	foundation_model	ForecasterFoundation	Chronos-2	38.436160	0.597467
1	2	lgbm_direct	ForecasterDirect	LGBMRegressor	50.224908	0.819189
2	3	lgbm_weekly_lags_rolling	ForecasterRecursive	LGBMRegressor	50.557647	0.824568
3	4	lgbm_daily_lags	ForecasterRecursive	LGBMRegressor	55.517689	0.905517
4	5	ridge_baseline	ForecasterRecursive	Ridge	93.145620	1.519244

The comparison returns a ComparisonResult, which groups the shared setup, ranked leaderboard, and individual backtests in a single object. Use it to inspect the results, audit the comparison, or reuse the winning candidate’s code in production.

Attribute	Type	Description
results	DataFrame	Ranked leaderboard, one row per candidate, sorted best to worst. Columns: rank, name, forecaster, estimator, the metric columns, and error when at least one candidate failed.
candidates	dict	Mapping of candidate name to the full BacktestResult object.
failures	dict	Mapping of candidate name to a CandidateFailure describing why it failed. Empty when every candidate succeeds.
ranking_metric	str	Name of the metric used to sort results.
cv_config	dict	Resolved TimeSeriesFold parameters plus the resulting n_folds, applied identically to every candidate.
profile	ForecastingProfile	The shared data profile behind every candidate.
explanation	str	Deterministic, human-readable summary of the comparison.
best_name	str	Name of the top-ranked candidate.
best_candidate	BacktestResult	Top-ranked candidate as a complete BacktestResult.

Displaying the object renders a rich summary of the explanation and the leaderboard.

In [30]:

# Rich summary of the comparison
# ==============================================================================
results_compare

# Rich summary of the comparison # ============================================================================== results_compare

Out[30]:

╭──────────────────────────────── Comparison Explanation ────────────────────────────────╮
│                                                                                        │
│  Compared 5 configurations, ranked ascending by mean_absolute_error. Shared            │
│  cross-validation strategy: Initial training up to 2012-08-31 23:59:00, fixed window,  │
│  no refit, 36-step horizon, 82 folds. Best: 'foundation_model' (ForecasterFoundation   │
│  / Chronos-2) = 38.4362, 23.5% ahead of 'lgbm_direct' (50.2249).                       │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯
                                    Comparison Results                                    
┏━━━━━━━┳━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Index ┃ rank ┃         name ┃   forecaster ┃    estimator ┃ mean_absol… ┃ mean_absolu… ┃
┡━━━━━━━╇━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ 0     │    1 │ foundation_… │ ForecasterF… │    Chronos-2 │     38.4362 │       0.5975 │
├───────┼──────┼──────────────┼──────────────┼──────────────┼─────────────┼──────────────┤
│ 1     │    2 │  lgbm_direct │ ForecasterD… │ LGBMRegress… │     50.2249 │       0.8192 │
├───────┼──────┼──────────────┼──────────────┼──────────────┼─────────────┼──────────────┤
│ 2     │    3 │ lgbm_weekly… │ ForecasterR… │ LGBMRegress… │     50.5576 │       0.8246 │
├───────┼──────┼──────────────┼──────────────┼──────────────┼─────────────┼──────────────┤
│ 3     │    4 │ lgbm_daily_… │ ForecasterR… │ LGBMRegress… │     55.5177 │       0.9055 │
├───────┼──────┼──────────────┼──────────────┼──────────────┼─────────────┼──────────────┤
│ 4     │    5 │ ridge_basel… │ ForecasterR… │        Ridge │     93.1456 │       1.5192 │
└───────┴──────┴──────────────┴──────────────┴──────────────┴─────────────┴──────────────┘

Because every candidate is a full BacktestResult, the details of any individual configuration remain available, including its metrics, its predictions and the standalone script that generated them.

In [31]:

# Inspect a specific candidate
# ==============================================================================
candidate = results_compare.candidates['foundation_model']

display(candidate.metrics)
display(candidate.predictions.head())
candidate.show_code()

# Inspect a specific candidate # ============================================================================== candidate = results_compare.candidates['foundation_model'] display(candidate.metrics) display(candidate.predictions.head()) candidate.show_code()

	mean_absolute_error	mean_absolute_scaled_error
0	38.43616	0.597467

	level	fold	pred
2012-09-01 00:00:00	users	0	148.059464
2012-09-01 01:00:00	users	0	102.715004
2012-09-01 02:00:00	users	0	66.084427
2012-09-01 03:00:00	users	0	40.878662
2012-09-01 04:00:00	users	0	29.577866

Generated codeCopy

import pandas as pd                                                                       
from skforecast.foundation import FoundationModel, ForecasterFoundation                   
from skforecast.model_selection import TimeSeriesFold, backtesting_foundation             
                                                                                          
# Load data                                                                               
data = pd.read_csv('data.csv')                                                            
                                                                                          
data['date_time'] = pd.to_datetime(data['date_time'])                                     
data = data.set_index('date_time')                                                        
data = data.asfreq('h')                                                                   
data = data.sort_index()                                                                  
                                                                                          
exog_features = ['holiday', 'weather', 'temp']                                            
                                                                                          
# Create foundation model (chronos-2-small)                                               
estimator = FoundationModel(                                                              
    model_id       = 'autogluon/chronos-2-small',                                         
    context_length = 8192,                                                                
)                                                                                         
                                                                                          
# Create forecaster                                                                       
forecaster = ForecasterFoundation(estimator=estimator)                                    
                                                                                          
# Time series cross-validation configuration                                              
cv = TimeSeriesFold(                                                                      
    steps              = 36,                                                              
    initial_train_size = '2012-08-31 23:59:00',                                           
    refit              = False,                                                           
)                                                                                         
                                                                                          
# Run backtesting                                                                         
metrics, predictions = backtesting_foundation(                                            
    forecaster        = forecaster,                                                       
    series            = data['users'],                                                    
    cv                = cv,                                                               
    metric            = ['mean_absolute_error', 'mean_absolute_scaled_error'],            
    exog              = data[exog_features],                                              
    verbose           = False,                                                            
    show_progress     = True,                                                             
    suppress_warnings = True,                                                             
)                                                                                         
                                                                                          
print(metrics)                                                                            
print(predictions.head())                                                                 

The most useful result of a comparison is often not the leaderboard, but best_candidate. It is a complete BacktestResult containing both the winning profile and plan, so it can be passed directly into the rest of the workflow without manually rebuilding the configuration.

In [32]:

# Winning configuration
# ==============================================================================
print(f"Best candidate: {results_compare.best_name}")
results_compare.best_candidate.plan

# Winning configuration # ============================================================================== print(f"Best candidate: {results_compare.best_name}") results_compare.best_candidate.plan

Best candidate: foundation_model

Out[32]:

              Forecast Plan              
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┓
┃ Property       ┃ Value                ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━┩
│ Task type      │ foundation           │
├────────────────┼──────────────────────┤
│ Forecaster     │ ForecasterFoundation │
├────────────────┼──────────────────────┤
│ Estimator      │ Chronos-2            │
├────────────────┼──────────────────────┤
│ Steps          │ 36                   │
├────────────────┼──────────────────────┤
│ Frequency      │ h                    │
├────────────────┼──────────────────────┤
│ Use exog       │ True                 │
├────────────────┼──────────────────────┤
│ Interval       │ None                 │
├────────────────┼──────────────────────┤
│ Primary metric │ mean_absolute_error  │
├────────────────┼──────────────────────┤
│ Preprocessing  │ 1 step               │
└────────────────┴──────────────────────┘

                                   Preprocessing Steps                                    
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Step                    ┃ Reason                                                       ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ handle_categorical_exog │ Categorical exogenous variables detected: ['weather'].       │
│                         │ Chronos-2 consumes categorical covariates natively, so no    │
│                         │ encoding is needed.                                          │
└─────────────────────────┴──────────────────────────────────────────────────────────────┘

╭─────────────────────────────────── Plan Explanation ───────────────────────────────────╮
│                                                                                        │
│  Plan: ForecasterFoundation + Chronos-2. No lag or window features: the foundation     │
│  model forecasts directly from the raw context window. Exogenous variables included.   │
│  MAE is interpretable, robust to outliers, and works at any scale.                     │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯

In [33]:

# Produce the final forecast with the winning configuration
# ==============================================================================

# Simulate future values of exogenous variables for the next 36 hours
exog = data[['holiday', 'weather', 'temp']].tail(36).copy()
exog.index = pd.date_range(
    start=pd.to_datetime(data['date_time'].max()) + pd.Timedelta(hours=1), periods=36, freq='h'
)

results_pred = assistant.forecast(
                   data        = data,
                   target      = 'users',
                   date_column = "date_time",
                   steps       = 36,
                   profile     = results_compare.profile,
                   plan        = results_compare.best_candidate.plan,
                   interval    = [0.1, 0.9],  # 80% prediction interval
                   test_size   = None,        # Use the entire dataset for training (prediction mode)
                   exog        = exog         # Future values of exogenous variables for the next 36 hours
               )

# Reproducible skforecast script for the winning configuration
# ==============================================================================
results_pred.show_code()

# Produce the final forecast with the winning configuration # ============================================================================== # Simulate future values of exogenous variables for the next 36 hours exog = data[['holiday', 'weather', 'temp']].tail(36).copy() exog.index = pd.date_range( start=pd.to_datetime(data['date_time'].max()) + pd.Timedelta(hours=1), periods=36, freq='h' ) results_pred = assistant.forecast( data = data, target = 'users', date_column = "date_time", steps = 36, profile = results_compare.profile, plan = results_compare.best_candidate.plan, interval = [0.1, 0.9], # 80% prediction interval test_size = None, # Use the entire dataset for training (prediction mode) exog = exog # Future values of exogenous variables for the next 36 hours ) # Reproducible skforecast script for the winning configuration # ============================================================================== results_pred.show_code()

╭─────────────────────────────── IgnoredArgumentWarning ───────────────────────────────╮
│ A pre-built `plan` was provided, so the following argument(s) are ignored:           │
│ ['interval']. To change these, refine the plan with `refine_plan()` before calling.  │
│                                                                                      │
│ Category : skforecast.exceptions.IgnoredArgumentWarning                              │
│ Location :                                                                           │
│ /opt/homebrew/Caskroom/miniconda/base/envs/skforecast_ai_py13/lib/python3.13/site-pa │
│ ckages/skforecast_ai/_utils.py:402                                                   │
│ Suppress : warnings.simplefilter('ignore', category=IgnoredArgumentWarning)          │
╰──────────────────────────────────────────────────────────────────────────────────────╯

Generated codeCopy

import pandas as pd                                                                       
from skforecast.foundation import FoundationModel, ForecasterFoundation                   
                                                                                          
# Load data                                                                               
data = pd.read_csv('data.csv')                                                            
                                                                                          
# Load future exogenous variables covering the forecast horizon                           
exog_future = pd.read_csv('exog_future.csv')                                              
                                                                                          
data['date_time'] = pd.to_datetime(data['date_time'])                                     
data = data.set_index('date_time')                                                        
data = data.asfreq('h')                                                                   
data = data.sort_index()                                                                  
                                                                                          
exog_future['date_time'] = pd.to_datetime(exog_future['date_time'])                       
exog_future = exog_future.set_index('date_time')                                          
exog_future = exog_future.asfreq('h')                                                     
exog_future = exog_future.sort_index()                                                    
                                                                                          
series = data['users']                                                                    
exog = data[['holiday', 'weather', 'temp']]                                               
                                                                                          
# Create foundation model (chronos-2-small)                                               
estimator = FoundationModel(                                                              
    model_id       = 'autogluon/chronos-2-small',                                         
    context_length = 8192,                                                                
)                                                                                         
                                                                                          
# Create forecaster                                                                       
forecaster = ForecasterFoundation(estimator=estimator)                                    
                                                                                          
# Fit (stores context only — no training)                                                 
forecaster.fit(series=series, exog=exog)                                                  
                                                                                          
# Predict                                                                                 
steps = 36                                                                                
predictions = forecaster.predict(steps=steps, exog=exog_future)                           
print(predictions)                                                                        

Like any other result, a ComparisonResult can be passed to ask() to explain why the ranking looks the way it does. However, the LLM cannot change the outcome: all metrics and rankings are computed deterministically before it sees the result.

In [34]:

# Asking the assistant about the comparison
# ==============================================================================
explanation = assistant.ask(
    prompt = (
        "Explain the comparison results. Is the margin between the top candidates "
        "meaningful, or are they practically equivalent?"
    ),
    result = results_compare
)

# Asking the assistant about the comparison # ============================================================================== explanation = assistant.ask( prompt = ( "Explain the comparison results. Is the margin between the top candidates " "meaningful, or are they practically equivalent?" ), result = results_compare )

/var/folders/wt/8tvn563d5v55nspfbydgqb9r0000gp/T/ipykernel_48657/3169532999.py:3: DataSentToLLMWarning: `send_data_to_llm=False` does not apply to `result`: the predicted values it carries are sent to the LLM, because a question about a result cannot be answered from summary statistics alone. Your input data is not sent: a result holds only the model's output, never the data it was fitted on. To keep predictions local, ask without `result`.
  explanation = assistant.ask(

In [35]:

# Show the explanation generated by the LLM
# ==============================================================================
explanation.show_explanation()

# Show the explanation generated by the LLM # ============================================================================== explanation.show_explanation()

╭────────────────────────────────── Assistant Response ──────────────────────────────────╮
│                                                                                        │
│  The comparison results show that the foundation model configuration (using            │
│  Chronos-2) is the top performer with a mean absolute error of 38.436160, while the    │
│  second-best model (lgbm_direct) achieved a mean absolute error of 50.224908.          │
│  According to the deterministic summary, the foundation model is 23.5% ahead of        │
│  lgbm_direct, indicating a meaningful performance margin between the two.              │
│  Furthermore, the top four configurations all successfully beat the naive baseline by  │
│  achieving mean absolute scaled error values below 1, whereas the baseline ridge       │
│  configuration did not.                                                                │
│                                                                                        │
│  Comparison Summary                                                                    │
│                                                                                        │
│  The 5 candidates evaluated were ranked ascendingly by their mean absolute error       │
│  (MAE). The complete standings are:                                                    │
│                                                                                        │
│   • foundation_model (ForecasterFoundation using Chronos-2): Rank 1, MAE of            │
│     38.436160, MASE of 0.597467                                                        │
│   • lgbm_direct (ForecasterDirect using LGBMRegressor): Rank 2, MAE of 50.224908,      │
│     MASE of 0.819189                                                                   │
│   • lgbm_weekly_lags_rolling (ForecasterRecursive using LGBMRegressor): Rank 3, MAE    │
│     of 50.557647, MASE of 0.824568                                                     │
│   • lgbm_daily_lags (ForecasterRecursive using LGBMRegressor): Rank 4, MAE of          │
│     55.517689, MASE of 0.905517                                                        │
│   • ridge_baseline (ForecasterRecursive using Ridge): Rank 5, MAE of 93.145620, MASE   │
│     of 1.519244                                                                        │
│                                                                                        │
│  The margin between the winner and the nearest alternative is notable, with the        │
│  foundation model positioned 23.5% ahead of the second-ranked lgbm_direct. While the   │
│  difference between the second and third place is relatively small (50.224908 versus   │
│  50.557647), the foundation model stands out with a substantial lead.                  │
│                                                                                        │
│  Baseline Performance                                                                  │
│                                                                                        │
│  The mean absolute scaled error (MASE) values provide context against a naive          │
│  baseline:                                                                             │
│                                                                                        │
│   • The foundation_model, lgbm_direct, lgbm_weekly_lags_rolling, and lgbm_daily_lags   │
│     all achieved MASE values below 1. This indicates that these four configurations    │
│     successfully outperformed the naive baseline.                                      │
│   • The ridge_baseline model achieved a MASE of 1.519244. Because this value is above  │
│     1, it did not beat the naive baseline.                                             │
│                                                                                        │
│  Evaluation Robustness                                                                 │
│                                                                                        │
│  The differences in performance are evaluated under a shared, multi-fold               │
│  cross-validation setup:                                                               │
│                                                                                        │
│   • Number of folds: 82                                                                │
│   • Forecast horizon: 36 steps                                                         │
│   • Initial training size: Up to 2012-08-31 23:59:00                                   │
│   • Strategy: Fixed training window size with no refit (refit is False)                │
│                                                                                        │
│  This rigorous evaluation across 82 folds ensures that the observed performance gap    │
│  is representative.                                                                    │
│                                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────╯

This tutorial covered the fast path of skforecast-ai: the most efficient route from raw time series data to a validated forecast with minimal manual configuration. This approach is highly effective for rapidly generating robust baseline models when you trust the assistant to implement sensible default strategies.

However, if your use case requires granular control to inspect, audit, or modify the intermediate objects (such as the data profile, the modeling plan, or the validation schema) before executing the next phase, we recommend exploring the step-by-step workflow for a comprehensive, stage-by-stage walkthrough of the underlying architecture.