This is what it takes to make valid instruments.
Measurement begins before data collection. The construct, item population, range, calibration, fit, uncertainty, and record all belong to the instrument.
Define the quantity before choosing the observations.
Instrument construction begins with the construct: the latent quantity the observations are intended to reveal. Items are then selected or developed to span a useful range of that quantity rather than merely to accumulate points.
Artificial respondents
Models are treated as respondent populations under explicitly recorded conditions. Model family, prompt, temperature, seed, and other experimental factors can define respondent configurations when the research question requires them.
The model must earn the right to become a ruler.
Calibration alone is insufficient. Item behavior, respondent behavior, residual structure, dimensionality, fit, and unexpected interactions must be examined before the resulting scale can support substantive claims.
Failure is informative: an item that does not conform to the instrument may be ambiguous, multidimensional, contaminated, or simply measuring something else.
The questions can change while the scale remains comparable.
Linked forms allow an instrument to evolve without discarding its measurement history. Common calibrated items or other defensible linking designs can place new forms onto an existing scale, supporting repeated measurement while individual items are replaced or retired.
An instrument has a lineage.
Instrument identifiers, calibration samples, item versions, software, procedures, linking decisions, and uncertainty estimates should be preserved as part of the scientific record. Provenance is not paperwork after measurement; it is part of the measurement system.