Results¶
What the methods that generate code, run it or call the LLM return:
| Returned by | Result |
|---|---|
forecast_code(), backtest_code() |
CodeGenerationResult |
forecast() |
ForecastResult |
create_cv() |
CVResult |
backtest() |
BacktestResult |
compare() |
ComparisonResult |
ask() |
AskResult |
check_llm() |
LLMCheckResult |
Every result renders itself in a notebook and serializes to JSON, and all of them except AskResult and LLMCheckResult can be the context of ask(). Every result that ask() accepts, plus ForecastingProfile, also describes itself in plain text with describe(), without an LLM: the text is the context ask() sends about it, with the predictions summarized instead of listed row by row, without the instructions addressed to the LLM and with its lists cut when there are many series.
compare() also sends a CompareProgress to its progress_callback when each candidate starts and ends.
skforecast_ai.schemas.results ¶
Classes:
| Name | Description |
|---|---|
RenderedScript |
Structured representation of a rendered forecasting script. |
LLMContext |
Everything |
CodeGenerationResult |
Result of the |
SingleRunResult |
Shared base for the result of a single forecasting or backtesting run. |
ForecastResult |
Result of the |
BacktestResult |
Result of the |
CVResult |
Result of the |
AskResult |
Result of the |
LLMCheckResult |
Outcome of |
CandidateFailure |
Reason why a single |
CompareProgress |
Progress event that |
ComparisonResult |
Result of the |
Attributes:
| Name | Type | Description |
|---|---|---|
CVOverrideName |
|
|
CV_OVERRIDE_NAMES |
tuple[str, ...]
|
|
Attributes¶
CVOverrideName
module-attribute
¶
CVOverrideName = Literal[
"initial_train_size",
"fold_stride",
"refit",
"fixed_train_size",
"gap",
"skip_folds",
"allow_incomplete_fold",
]
Classes¶
RenderedScript ¶
Bases: BaseModel
Structured representation of a rendered forecasting script.
Splits the rendered script into logical sections so that
forecast() can exec the core logic while forecast_code()
returns the full standalone script.
Attributes:
| Name | Type | Description |
|---|---|---|
imports |
str
|
Import statements required by the script. |
data_loading |
str
|
Code that loads data from CSV and sets up the index. |
core |
str
|
Core execution logic (preprocessing, split, fit, predict,
metrics). Operates on a pre-existing |
LLMContext ¶
Bases: BaseModel
Everything ask() needs in order to explain a result object.
Produced by ExplainableResult._build_llm_context. Keeping the four
fields in a single object means ask() never reads a result's own
attributes, so a new kind of result can be explained without touching
ask().
Attributes:
| Name | Type | Description |
|---|---|---|
text |
str
|
Rendered plain-text context block inserted into the user message. |
profile |
ForecastingProfile, default None
|
Profile echoed back on |
plan |
ForecastPlan, default None
|
Plan echoed back on |
code |
str, default None
|
Generated script echoed back on |
sends_result_values |
bool, default True
|
Whether |
CodeGenerationResult ¶
Bases: PickleDefaultsMixin, DisplayMixin, ExplainableResult, BaseModel
Result of the forecast_code workflow.
Attributes:
| Name | Type | Description |
|---|---|---|
profile |
ForecastingProfile
|
Profile of the input dataset and high-level modeling decisions. |
plan |
ForecastPlan
|
Detailed forecasting plan. |
code |
str
|
Generated Python script. |
SingleRunResult ¶
Bases: PickleDefaultsMixin, DisplayMixin, ExplainableResult, BaseModel
Shared base for the result of a single forecasting or backtesting run.
Declares the fields that every single run produces, and renders them
into an LLM context block through _build_llm_context. Concrete
results (for example ForecastResult and BacktestResult) inherit
from this class and add the fields specific to them.
Aggregate results that wrap several runs (for example
ComparisonResult) do not inherit from this class; they implement
ExplainableResult directly so they can send a compact summary rather
than a concatenation of everything they wrap.
Every result serializes to JSON with model_dump(mode="json") or
model_dump_json(): DataFrames become lists of row records with the
index as a leading column. model_dump() keeps the live DataFrames.
Attributes:
| Name | Type | Description |
|---|---|---|
profile |
ForecastingProfile
|
Profile of the input dataset and high-level modeling decisions. |
plan |
ForecastPlan
|
Detailed forecasting plan that was executed. |
code |
str
|
Generated Python script equivalent to the execution. |
predictions |
pandas DataFrame
|
Forecasted values produced by the run. |
metrics |
pandas DataFrame
|
Evaluation metrics produced by the run. |
ForecastResult ¶
Bases: SingleRunResult
Result of the forecast workflow (executes the pipeline end-to-end).
Attributes:
| Name | Type | Description |
|---|---|---|
profile |
ForecastingProfile
|
Profile of the input dataset and high-level modeling decisions. |
plan |
ForecastPlan
|
Detailed forecasting plan that was executed. |
code |
str
|
Generated Python script equivalent to the execution. |
metrics |
pandas DataFrame, None
|
Evaluation metrics. DataFrame with a |
predictions |
pandas DataFrame
|
Forecasted values for the requested steps. When prediction intervals (or quantiles) are requested, the corresponding bound columns are included alongside the point predictions. |
BacktestResult ¶
Bases: SingleRunResult
Result of the backtest workflow.
Attributes:
| Name | Type | Description |
|---|---|---|
profile |
ForecastingProfile
|
Profile of the input dataset and high-level modeling decisions. |
plan |
ForecastPlan
|
Detailed forecasting plan that was executed. |
cv_config |
dict
|
Resolved |
metrics |
pandas DataFrame
|
Backtesting metric values returned by skforecast. |
predictions |
pandas DataFrame
|
Full backtest predictions across all folds. |
code |
str
|
Generated Python script reproducing the backtesting workflow. |
explanation |
str
|
Human-readable explanation of the backtesting configuration and results summary. |
cv_overridden_fields |
(list, None)
|
Names of the strategy parameters the user passed to |
cv_fields_without_effect |
list
|
Names in |
cv_llm_configured |
bool
|
Whether the LLM of |
cv_defaults_explanation |
str
|
Why the strategy parameters that the user did not pass have their
value, and which ones the user passed. Shown after |
CVResult ¶
Bases: PickleDefaultsMixin, DisplayMixin, ExplainableResult, BaseModel
Result of the create_cv workflow (a cross-validation strategy).
Wraps the TimeSeriesFold splitter together with the resolved
parameters, the fold count, the snippet that builds the splitter, and
the explanation of the choices. Pass it to backtest(),
backtest_code() or compare() as cv, or to ask() as context.
Attributes:
| Name | Type | Description |
|---|---|---|
profile |
ForecastingProfile
|
Profile of the input dataset and high-level modeling decisions the strategy was derived from. |
plan |
ForecastPlan
|
Forecasting plan the strategy was derived from (its |
cv |
TimeSeriesFold
|
Configured cross-validation fold splitter. |
cv_config |
dict
|
Resolved |
code |
str
|
Python snippet that builds the |
explanation |
str
|
Human-readable explanation of the chosen configuration. When the strategy was derived from a prompt, the LLM reasoning comes first. |
overridden_fields |
list
|
Names of the parameters the user passed to |
fields_without_effect |
list
|
Names in |
llm_configured |
bool
|
Whether the LLM of |
defaults_explanation |
str
|
Why the parameters that the user did not pass have their value
(the rule that fixed |
Notes
create_cv() used to return a (TimeSeriesFold, str) tuple. A
CVResult is not iterable, so unpacking it raises a TypeError that
points to the cv and explanation attributes.
Attributes¶
model_config
class-attribute
instance-attribute
¶
model_config = ConfigDict(arbitrary_types_allowed=True)
overridden_fields
class-attribute
instance-attribute
¶
overridden_fields = Field(default_factory=list)
fields_without_effect
class-attribute
instance-attribute
¶
fields_without_effect = Field(default_factory=list)
AskResult ¶
Bases: DisplayMixin, BaseModel
Result of the ask workflow (requires LLM).
Attributes:
| Name | Type | Description |
|---|---|---|
profile |
ForecastingProfile, default None
|
Profile of the input dataset and high-level modeling decisions, if data was provided. |
plan |
ForecastPlan, default None
|
Detailed forecasting plan, if the agent produced one. |
code |
str, default None
|
Generated Python script, if the agent produced one. |
explanation |
str
|
LLM-generated explanation or response. |
skills |
list of str, default []
|
Names of the skill documents sent to the model, after any trimming to fit the context budget. Recorded because the budget depends on the profile and context size at call time, so the selection cannot be re-derived from the question alone. |
Attributes¶
LLMCheckResult ¶
Bases: DisplayMixin, BaseModel
Outcome of ForecastingAssistant.check_llm().
Reports how the LLM configuration resolves before any workflow runs:
which provider and model the string names, where the credentials
come from, what base_url means for that provider, whether the
optional dependencies are installed and, for Ollama, whether the
server answers. Credential values are never stored, so the result
can be printed or serialized safely.
Attributes:
| Name | Type | Description |
|---|---|---|
llm |
str
|
Provider string as given, in the format |
provider |
(str, None)
|
Provider prefix, None when the string could not be parsed. |
model_name |
(str, None)
|
Model name, None when the string could not be parsed. |
credential_source |
str
|
Where credentials come from: |
env_var |
(str, None)
|
Name of the environment variable that would be read, when the provider has one. |
env_var_set |
(bool, None)
|
Whether that variable is set. None when it is not consulted
(explicit |
credential_note |
(str, None)
|
Plain-language description of the credential resolution. |
base_url |
(str, None)
|
Endpoint that will be used, after applying the provider default (Ollama) or dropping a value the provider ignores. |
base_url_note |
(str, None)
|
What |
dependencies_ok |
bool
|
Whether every module the provider needs is importable. |
missing_dependencies |
list of str
|
Modules that are not importable, with the extra that installs them. |
reachable |
(bool, None)
|
For Ollama, whether the server answered. None for other providers. |
call_ok |
(bool, None)
|
Whether the round-trip call of |
error |
(str, None)
|
First failure found, None when every check passed. |
ok |
bool
|
True when no check failed (computed). |
CandidateFailure ¶
Bases: BaseModel
Reason why a single compare() candidate failed to run.
Holds a plain-data snapshot of the failure instead of the live
exception. An exception object keeps its traceback frames alive, and
those frames reference the execution namespace (which contains a copy
of the dataset, the fitted forecaster and the predictions), so
retaining one per failed candidate would pin an unbounded amount of
memory. The formatted traceback carries the same debugging
information at a fixed, small cost, and keeps ComparisonResult
serializable.
Attributes:
| Name | Type | Description |
|---|---|---|
error_type |
str
|
Class name of the root-cause exception, for example
|
message |
str
|
Message of the root-cause exception. |
traceback |
str
|
Full formatted traceback of the failure. |
generated_code |
str, default None
|
Generated script that failed, when the failure happened while
executing rendered code. |
Methods:
| Name | Description |
|---|---|
from_exception |
Build a |
summary |
Build a concise one-line |
Attributes¶
Methods:¶
from_exception
classmethod
¶
from_exception(exc)
Build a CandidateFailure from the exception a candidate raised.
A ForecastExecutionError wraps the generated code and the
formatted execution traceback; it is unwrapped to its
original_error root cause so the failure reports the underlying
reason rather than the verbose execution-context message.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
exc
|
Exception
|
Exception raised while evaluating a candidate. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
failure |
CandidateFailure
|
Plain-data snapshot of the failure. |
Source code in skforecast_ai/schemas/results.py
814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 | |
summary ¶
summary(max_length=200)
Build a concise one-line "ErrorType: message" summary.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
max_length
|
int
|
Maximum length of the returned summary. Longer summaries are truncated with a trailing ellipsis. |
200
|
Returns:
| Name | Type | Description |
|---|---|---|
summary |
str
|
Single-line summary of the failure. |
Source code in skforecast_ai/schemas/results.py
855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 | |
CompareProgress ¶
Bases: BaseModel
Progress event that compare() passes to its progress_callback.
compare() sends one event when a candidate starts (status
'started') and one when it ends ('succeeded' or 'failed'), so
every comparison of total candidates sends 2 * total events.
Attributes:
| Name | Type | Description |
|---|---|---|
candidate |
str
|
Name of the candidate the event refers to, as in the |
status |
{'started', 'succeeded', 'failed'}
|
Whether the candidate is about to run, ran, or failed. |
completed |
int
|
Number of candidates that have ended so far, this one included
when |
total |
int
|
Number of candidates the comparison runs, the baseline included. |
error |
str, default None
|
One-line summary of the failure, the same text as the |
ComparisonResult ¶
Bases: PickleDefaultsMixin, DisplayMixin, ExplainableResult, BaseModel
Result of the compare workflow (ranks several forecasters).
Backtests several forecaster/estimator configurations with the same
cross-validation strategy and returns a metric-ranked leaderboard
plus the winning configuration as a reusable BacktestResult.
Attributes:
| Name | Type | Description |
|---|---|---|
profile |
ForecastingProfile
|
Shared profile used for every candidate. |
cv_config |
dict
|
Resolved |
results |
pandas DataFrame
|
Ranked comparison table, one row per candidate sorted best to
worst by |
candidates |
dict
|
Mapping of candidate name to the full |
failures |
dict
|
Mapping of candidate name to a |
ranking_metric |
str
|
Name of the metric used to sort |
explanation |
str
|
Human-readable summary of the comparison. |
baseline_name |
str, default None
|
Name of the |
cv_overridden_fields |
(list, None)
|
Names of the strategy parameters the user passed to |
cv_fields_without_effect |
list
|
Names in |
cv_llm_configured |
bool
|
Whether the LLM of |
cv_defaults_explanation |
str
|
Why the strategy parameters that the user did not pass have their
value, and which ones the user passed. Shown after |
strategy_plan_name |
str, default None
|
Name of the row that runs the plan the strategy was created for
( |
best_name |
str
|
Name of the top-ranked candidate. |
best_candidate |
BacktestResult
|
Top-ranked candidate. Always present: a comparison in which every
candidate fails raises |
Notes
Every candidate name appears in exactly one of candidates and
failures, never in both and never in neither. The two mappings
therefore partition the candidates that were evaluated, and their
union matches the 'name' column of results.
best_name is a computed field, so it is included by model_dump();
best_candidate is a plain property, so the winning BacktestResult
is not serialized a second time.
Every result serializes to JSON with model_dump(mode="json") or
model_dump_json(): DataFrames become lists of row records with the
index as a leading column. model_dump() keeps the live DataFrames.
Functions:¶
skforecast_ai.schemas.explainable.ExplainableResult.describe ¶
describe()
Describe the result in plain text.
The text is the context ask() sends to the LLM about this result,
with the predictions summarized instead of listed row by row,
without the sentences that only tell the LLM how to answer, and
with its lists cut as described in the Notes when there are many
series. It is deterministic and needs no LLM, so it can be shown to
a user or passed to an agent as it is. A plan is
described through the script rendered from it:
assistant.forecast_code(profile=profile, plan=plan).describe().
Returns:
| Name | Type | Description |
|---|---|---|
description |
str
|
Sections wrapped in XML-style tags ( |
Notes
It never includes values row by row: predictions are summarized by their shape, columns, minimum, maximum, mean and standard deviation, and metrics are included as computed.
Its length does not grow with the number of series: it keeps the
first 15 items of each list (target and exogenous columns, series
or columns with missing values, data warnings, lags, window
features, failed candidates) and the statistics, significant lags
and metrics of the first 5 series, plus the aggregated metric rows
(average, weighted_average, pooling), and says how many
there are. With 500 series it is about 4,000 to 5,500 characters. The
explanation texts (of the profile, the plan, the cross-validation
or the comparison) are kept whole, so the lags that the plan
explanation names are all listed there. The context of ask()
keeps every list whole.
The summary of the predictions is computed over all their rows,
and with several series the point forecast is also summarized for
each of the first 5 series, while ask() sends their rows. The
fold column of a backtest is an
identifier: only the number of folds is given. The script of
backtest_code() is described as a backtest, with its
cross-validation strategy, number of folds and trainings.
Source code in skforecast_ai/schemas/explainable.py
36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 | |