|
| MomentTable | load_moment_table (Any parts, *int S, Any roster=None) |
| |
| Triple | merge_moments (Triple a, Triple b) |
| |
| Triple | pooled_moments (Iterable[Triple] blocks) |
| |
| float|None | ranking_statistic (int n, float s, float m2) |
| |
| dict[str, int] | estimate_work (int S, int N) |
| |
| str | roster_label (Any sampler, *Any early_stop=_UNSET) |
| |
| Iterator[tuple[tuple[int,...], tuple[int,...]]] | iter_ordered_splits (int S) |
| |
| dict[str, Any] | pbo (Any parts, *int S, Any roster=None, int|float|None max_work=None, str|None sampler=None, Any early_stop=_UNSET, bool with_lambdas=False) |
| |
Probability of Backtest Overfitting (CSCV) over per-block moments: the exact stdlib reference.
Post hoc and pure. The input is an N x S table of block moments, one record per trial
(``{trial_id, status, block_returns: {status, count[S], sum[S], m2[S]}}``), never the T x N
return matrix. ``count`` is the number of intervals in a block, ``sum`` the sum of the
per-interval returns and ``m2`` the sum of squared deviations from the block mean. A row with
status ``ok`` or ``constraint_violation`` is computed (default D5); a failed, pruned or partial
row, or a payload that reports a non-positive equity, refuses with a code of its own. The rows
come in parts, each part with the result-level ``return_blocks`` object its rows were reduced
under plus a ``dataset_identity`` that the caller states (a call with one part passes a list of
one pair), and the caller declares the roster (the declared trial set). The module refuses
unless the parts declare one and the same object, every row's counts equal the ``n_b`` of its
own part's object, and the rows of all the parts together are exactly the roster. It checks what
it is handed: the grouping of the rows into parts and the object declared for each part are the
caller's. The module enumerates all C(S, S/2) ordered train/test splits of the S blocks, ranks
every trial on the training blocks, and reports how the training winner ranks on the
complementary blocks.
Every number is plain binary64 arithmetic in a fixed operation order, with no fused
operations and no tolerance. Counts and the sign of the logit are exact integer arithmetic; only
the logarithm of a logit value depends on the platform math library. Contract text and defaults:
``docs/pbo.md``.
Status: the native-proof PBO run uses synthetic data only; PBO on real data is not yet proven.
| dict[str, int] pineforge_hpo.pbo.estimate_work |
( |
int |
S, |
|
|
int |
N |
|
) |
| |
The admission figure of contract clause 3.7, exact integers, before any split is run.
``n_splits = C(S, S/2)``. Each complementary pair is evaluated once, so the merges are
``(n_splits/2) * N * (S - 2)``; the comparisons are ``2 * n_splits * N`` (a count of the
selected trial's rank, not a sort). ``work`` is their sum.
Definition at line 996 of file pbo.py.
Referenced by pineforge_hpo.pbo.pbo().
| MomentTable pineforge_hpo.pbo.load_moment_table |
( |
Any |
parts, |
|
|
*int |
S, |
|
|
Any |
roster = None |
|
) |
| |
Validate the parts (contract clause 3.5) and return the merged table sorted by trial id.
``parts`` is a list of ``(declared object, rows)`` pairs, one for each part of the lineage (a
call with one part passes a list of one pair): the declared object is the result-level
``return_blocks`` object the rows of that part were reduced under, plus the
``dataset_identity`` that the caller states. ``roster`` is the declared trial set of the
lineage. Both are required. The declared objects of all the parts must be
one and the same, every row's counts must equal the ``n_b`` of its own part's object, and the
rows of all the parts together must be exactly the roster. Refuses with ``PboError`` and
never subsets: one bad row refuses the whole roster.
Definition at line 813 of file pbo.py.
| Triple pineforge_hpo.pbo.merge_moments |
( |
Triple |
a, |
|
|
Triple |
b |
|
) |
| |
Merge two ``(count, sum, m2)`` triples, ``a`` before ``b``, by the Chan/Pebay rule.
``M2 = M2a + M2b + d*d*na*nb/n`` with ``d = sum_b/nb - sum_a/na``. The operation order is
pinned: ``(((d*d)*na)*nb)/n`` is formed first and added to ``(M2a + M2b)``; each operation
is a separate binary64 step, so nothing is fused. The term added is never negative, hence
neither is the result.
Definition at line 941 of file pbo.py.
Referenced by pineforge_hpo.pbo.pooled_moments().
| dict[str, Any] pineforge_hpo.pbo.pbo |
( |
Any |
parts, |
|
|
*int |
S, |
|
|
Any |
roster = None, |
|
|
int | float | None |
max_work = None, |
|
|
str | None |
sampler = None, |
|
|
Any |
early_stop = _UNSET, |
|
|
bool |
with_lambdas = False |
|
) |
| |
Run CSCV over every ordered split and return the result object (contract clause 3.6).
``parts`` is a list of ``(declared object, rows)`` pairs, one for each part of the lineage (a
call with one part passes a list of one pair): the declared object is the result-level
``return_blocks`` object the rows of that part were reduced under, plus the
``dataset_identity`` that the caller states, and the rows are those that
``load_moment_table`` takes. A table is not accepted: it has lost its parts.
``roster`` (the declared trial set of the lineage) is required: the call refuses unless the
declared objects of all the parts are one and the same, every row's counts equal the ``n_b``
of its own part's object, and the rows of all the parts together are exactly the roster
(contract clause 3.5). Block sizes may differ by at most one interval (default D2); the
result then says ``equal_blocks: false`` and names ``block_counts``, and a larger spread
refuses.
``sampler`` and ``early_stop`` (the result's ``early_stop`` object, or None when it has none)
are both required and decide the roster label of default D6.
Per split, the training winner is the trial with the largest ranking statistic; k tied
winners (exact binary64 equality) each carry weight 1/k. Its out-of-sample rank is the
mid-rank among the N trials, ``omega = rank / (N + 1)`` and
``lambda = ln(omega / (1 - omega))``. The primary rate ``pbo`` is the share of splits with
lambda <= 0 (default D3); the share with lambda < 0 is reported next to it. A refusal raises
``PboError``; nothing partial is returned.
Definition at line 1112 of file pbo.py.
References pineforge_hpo.pbo.estimate_work(), and pineforge_hpo.pbo.roster_label().
| str pineforge_hpo.pbo.roster_label |
( |
Any |
sampler, |
|
|
*Any |
early_stop = _UNSET |
|
) |
| |
The roster label of default D6; an unknown or missing sampler refuses.
``diagnostic_adaptive_roster`` for an adaptive sampler and for any sampler whose study had a
no-improvement early stop; ``paper_conformant`` for ``grid``, ``random`` and ``sobol`` without
one. The early stop is not a sampler: it is a separate object of the result, so it is its own
argument and ``"early_stop"`` is not a sampler name. As for ``pbo``, the caller must say
whether the study had one (the result's ``early_stop`` object or ``True``; ``None`` or
``False`` when it had none): leaving it out refuses with ``pbo_parameter_missing``, so a study
with an early stop cannot get ``paper_conformant`` by a default.
Definition at line 1017 of file pbo.py.
Referenced by pineforge_hpo.pbo.pbo().
1.9.8