papers / evaluation / paper

software paper · JOSS draft · author list pre-submission

The population-view harness, as a standalone referee

popdgp formalizes the population-view harness — energy distance, calibration-blind coverage, classifier two-sample AUC, and an uncapped tail block, each anchored by a sampling-noise floor — as an eval-only package any candidate synthetic population can be scored against. This is the package's JOSS software paper, extracted behavior-preserving from imputation-paper, where the harness is proven and shipping. The author list carries a pre-submission TODO(authorship) note, kept honestly on the draft below.

where the harness actually lives today
The working implementation, all four scoring blocks, and every published number are in imputation-paper (src/imputation_paper/experiments/). popdgp's own main branch is README-stage; a harness-extraction branch is in progress and unmerged.
what's proven
Floors behave correctly (energy 0.009–0.012, AUC ~0.50 on a survey's own holdout split); the block decomposition localizes failures a single metric misses (a weight-blind fit ties on energy distance and coverage while its tail runs 1.9–2.3× the holdout's).
what this paper is, and isn't
This is the JOSS software paper: a citable reference for the package and its scoring blocks. The broader methods study — a metric-blind-spot power analysis and multi-view generalization — remains future work, tracked as an open issue in the popdgp repository.
paper

Web paper

Open full page