# Results

> **Status: SCAFFOLD ONLY — no results yet.** This section is structured so that
> figures, tables, and numbers can be dropped in once the experimental runs
> complete. Every `[[PLACEHOLDER]]` marks a value, table, or figure to be generated
> by the analysis pipeline. Do not report any number here that is not produced by
> `experiments/` and written under `results/`.

## 0. Reporting conventions

All point estimates are reported with 95% cluster-level bootstrap confidence
intervals in brackets. Cell-agreement and F1 metrics are fractions in [0, 1].
"Pass" denotes output equivalence at or above the configured threshold
(`output_equivalence_min`, currently 0.95). Unless stated otherwise, per-mode
numbers pool stochastic repeats and we also report run-to-run variance.

## 1. Gold corpus characterization

> Descriptive, generated by `notebooks/01_explore_gold.ipynb`.

- Number of gold datasets, and the include/exclude split: `[[PLACEHOLDER]]`.
- Distribution of structural types (wide vs. long; time-series vs. discrete;
  coordinate system; unit conventions; presence of separate location/sensor files):
  **Table 1** `[[PLACEHOLDER]]`.
- Cluster membership and sizes (`config/cv_folds.yaml`): `[[PLACEHOLDER]]`.
- Pairwise dataset-similarity distribution (motivates the grouped-CV design):
  **Figure 1** `[[PLACEHOLDER]]`.

## 2. Phase A — Retrospective cross-validation

### 2.1 Skill 1 (curator) in isolation

> Generated by `experiments/phase_a_crossval.py` (mode `skill1_isolated`),
> scored by `src/metrics/skill1_metrics.py`.

**Table 2 — Curator performance.** `[[PLACEHOLDER]]`

| Metric | Score [95% CI] |
|---|---|
| Decision accuracy (INCLUDE/EXCLUDE/FLAG) | `[[ ]]` |
| Decision macro-F1 | `[[ ]]` |
| File-selection F1 — payload / location / sensor | `[[ ]]` |
| Time-series binary accuracy | `[[ ]]` |
| Sampling-interval error (min) | `[[ ]]` |
| Location-source accuracy | `[[ ]]` |
| `qc_flag` accuracy | `[[ ]]` |
| Manipulation-detection accuracy | `[[ ]]` |
| Exemplar-selection match | `[[ ]]` |
| Deferral (FLAG) calibration | `[[ ]]` |
| **Composite curator score** | `[[ ]]` |

- Confusion matrix over the three decisions: **Figure 2** `[[PLACEHOLDER]]`.
- Where does the curator most often err (file selection vs. time-series vs.
  location)? `[[PLACEHOLDER narrative]]`.

### 2.2 Skill 2 (harmonizer) with oracle input

> Mode `skill2_oracle`; primary scoring by `src/metrics/skill2_output_equiv.py`.

**Table 3 — Harmonizer performance given correct curator input.** `[[PLACEHOLDER]]`

| Metric | Score [95% CI] |
|---|---|
| **Output-data equivalence (overall cell agreement)** | `[[ ]]` |
| Row-alignment F1 | `[[ ]]` |
| Datetime accuracy | `[[ ]]` |
| Coordinate accuracy | `[[ ]]` |
| Depth accuracy | `[[ ]]` |
| Unit-conversion accuracy | `[[ ]]` |
| Schema conformance (pass rate) | `[[ ]]` |
| Semantic mapping F1 | `[[ ]]` |
| Code executability (pass rate) | `[[ ]]` |
| Documentation completeness | `[[ ]]` |
| **Composite harmonizer score** | `[[ ]]` |
| **Output-equivalence pass rate (≥ threshold)** | `[[ ]]` |

- Per-column cell-agreement breakdown: **Figure 3** `[[PLACEHOLDER]]`.
- Distribution of per-dataset output equivalence (how many are near-perfect vs.
  partial): **Figure 4** `[[PLACEHOLDER]]`.

### 2.3 End-to-end pipeline

> Mode `end_to_end`.

**Table 4 — End-to-end vs. oracle.** `[[PLACEHOLDER]]`

| Metric | Oracle (Skill 2) | End-to-end | Gap [95% CI] |
|---|---|---|---|
| Output-data equivalence | `[[ ]]` | `[[ ]]` | `[[ ]]` |
| Output-equivalence pass rate | `[[ ]]` | `[[ ]]` | `[[ ]]` |
| Composite score | `[[ ]]` | `[[ ]]` | `[[ ]]` |

The **error-propagation gap** (oracle − end-to-end) is `[[PLACEHOLDER]]`
(`src/analysis/stats.py::error_propagation_gap`), quantifying the fraction of
end-to-end degradation attributable to curator error.

### 2.4 Error-propagation taxonomy

> `src/analysis/error_taxonomy.py`.

**Figure 5 / Table 5 — Attribution of end-to-end failures.** `[[PLACEHOLDER]]`

| Failure category | Share of failures |
|---|---|
| Skill-1 error propagated | `[[ ]]` |
| Skill-2 error (correct input) | `[[ ]]` |
| Inter-skill interface inconsistency | `[[ ]]` |
| Genuinely ambiguous (expert also uncertain) | `[[ ]]` |

### 2.5 Baselines

> `config/experiment.yaml::baselines`.

**Table 6 — Agent pair vs. baselines** (output equivalence, composite).
`[[PLACEHOLDER]]`

| System | Output equivalence [95% CI] | Pass rate |
|---|---|---|
| Agent pair (end-to-end) | `[[ ]]` | `[[ ]]` |
| Single-call LLM (non-agentic) | `[[ ]]` | `[[ ]]` |
| Agent without exemplars | `[[ ]]` | `[[ ]]` |
| Naïve string-matching heuristic | `[[ ]]` | `[[ ]]` |

Key contrasts to report: agent-pair vs. single-call (value of agentic structure);
agent vs. no-exemplars (value of the 19 references).

### 2.6 Similarity–performance relationship

> `src/analysis/similarity.py`; mixed-effects model in `src/analysis/stats.py`.

- Output equivalence vs. nearest-exemplar similarity, with regression fit:
  **Figure 6** `[[PLACEHOLDER]]`.
- Mixed-effects coefficients (mode, similarity) with CIs: **Table 7**
  `[[PLACEHOLDER]]`.
- Interpretation: the agent generalizes up to similarity `[[X]]`, beyond which
  output equivalence degrades to `[[Y]]`. `[[PLACEHOLDER narrative]]`.

### 2.7 Stochasticity

- Run-to-run variance per dataset/mode and pass@k: **Figure 7** `[[PLACEHOLDER]]`.

## 3. Metric validation

> `experiments/metric_validation.py`; `src/analysis/irr.py`.

Agreement between automated output-equivalence scoring and an expert rubric on the
gold data: correlation `[[PLACEHOLDER]]`, inter-rater reliability
`[[PLACEHOLDER]]`. This determines whether automated scoring can stand in for
expert judgment in Phase B. **Figure 8** `[[PLACEHOLDER]]`.

## 4. Phase B — Prospective blind evaluation

> `experiments/phase_b_prospective.py`. To be run once novel datasets and the
> expert's blind harmonizations are available.

- Number and provenance of prospective datasets: `[[PLACEHOLDER]]`.
- Agent vs. expert output equivalence on novel data: **Table 8** `[[PLACEHOLDER]]`.
- Comparison of Phase B performance to the Phase A end-to-end estimate (does
  retrospective CV predict prospective performance?): `[[PLACEHOLDER narrative]]`.
- Curator decision agreement with the expert on novel inclusion/exclusion calls:
  `[[PLACEHOLDER]]`.

## 5. Summary of findings

> One-paragraph synthesis to be written once the tables above are populated.
> Anticipated structure: headline output-equivalence number end-to-end; the size of
> the curator-induced error-propagation gap; the dominant failure category; the
> value added by agentic structure and by exemplars (vs. baselines); and the
> similarity threshold beyond which the system should defer to a human.