Research Code, Ready to Share

The script is now reproducible and all tests pass, but the number in Table 3.2 has to change, and the corrected result shows the model has no predictive skill.

The corrected number

• Seed 0: test accuracy 0.444 (32 of 72 seeds in 6 held-out trays), 95% CI 0.264–0.653.

Across 20 seeds: 0.481 ± 0.075. Always predicting the most common class scores 0.475 on the same splits, so the model does no better than that.

• The old script gave about 0.57–0.79 depending on the run, with a mean of 0.661 across 20 seeds.

Why it dropped: the old script split individual seeds at random, so seeds from the same tray landed in both training and test. pH is the same for every seed in a tray, so it tells the model which tray a seed came from. Germination is mostly decided by the tray (11 of 24 trays germinated entirely or not at all). The forest was recognising trays it had already seen. When whole trays are held out, the accuracy falls to the baseline.

Two other bugs barely moved the number (about 0.01): the scaler was fitted on all the data before the split, and missing dry-mass values were filled with the mean of the whole table (6.09) instead of each column's mean.

Fixes

scripts/train.py: whole trays are held out for testing; imputation and scaling are fitted on the training trays only; one --seed controls all the randomness, so two runs give identical metrics.json files; the confidence interval resamples whole trays; the script reports the spread across 20 seeds and the majority-class baseline; the data path is no longer hard-coded.

• The failing test: it checked 0.1 + 0.2 == 0.3 exactly. It now compares with a small tolerance.

germ/features.py: the bare except quietly turned any unreadable value into a blank. Now only real missing-value markers like n/a count as missing, and anything else raises an error.

• Setup: added packaging metadata, an editable install and requirements-lock.txt with the exact versions.

Tests and final run

17 tests pass. The new ones check that the data file hasn't changed, that the same seed gives the same result, the exact headline values, that no tray appears in both training and test, and a full run of the script.

• The tests catch the old mistake: when I put back the random per-seed split, 8 of them failed.

The pinned values hold only for these versions of Python, numpy and scikit-learn. I haven't tried a fresh install.

Still open

Data problem: dry mass is larger than fresh mass in 105 of 283 rows, which shouldn't be physically possible, so the dry-matter ratio feature is suspect. I didn't edit the data.

Reporting choice: with only 24 trays, report the mean ± sd across seeds next to the baseline, or use cross-validation by tray.

• pH conclusions: the effective sample size for any pH effect is 24 trays, not 288 seeds.

Full review: research/code-review.md.