Transparency and Severity, Not Exemption: The Provenance-Independence Thesis in Bayesian Defenses of HARKing

Qualitative researchers routinely move back and forth between data and hypotheses, and a widely read Japanese methods textbook (Kishi, Ishioka and Maruyama 2016) goes further, teaching authors to write as if the research question had been fixed from the outset. Igashira (2024), drawing on Fairfield and Charman's (2022) logical/objective Bayesianism, defends such iterative practice against the charge of HARKing (Hypothesizing After the Results are Known): because the order in which evidence is incorporated does not affect the posterior, the accommodation/prediction distinction is dispensable, and preregistration and timestamps are "neither necessary nor appropriate" for iterative case research. I argue that the dissolution of that distinction is not a neutral consequence of Bayesian rationality but rests on a further, contestable premise, which I call the provenance-independence thesis: that the fact of a hypothesis having been selected to fit the data may be kept out of the description of the evidence. Three lines of criticism follow. First, even granting the likelihood principle, HARKing systematically inflates apparent over actual evidence — the prior of a selected hypothesis is diluted across the searched space, and a report generated by selection raises the likelihood under the alternatives; the Bayesian Occam factor is the Bayesian counterpart of the very flexibility penalty the defense denies. Second, model misspecification cannot be detected from inside the model, so confirmation requires exposure to open-world data, i.e. severity in Mayo's sense — a line that does reject the likelihood principle as a norm of scientific inference. Third, and independently of both, misreporting a finding's provenance is a failure of honesty that no epistemology excuses. The relevant distinction is therefore not qualitative versus quantitative but test-under-risk versus mere accommodation, a line that cuts across both. What iterative research needs is not exemption but transparency about the selection process and severity where it can be had. (The paper is written in Japanese.)