mardi 5 novembre 2019

Hypothesis testing of binomial: regular, and near-zero case different interpretation?

Suppose we want to do hypothesis testing, checking how unlikely it is for the observed data to come from hypothesized distribution. Compare two cases:

  1. Binomial(n=500, prob=0.1) draw, observed data: 73, Two-sided p-value: 0.0009
  2. Binomial(n=500, prob=0.00006) draw, observed data: 1, Two-sided p-value: 0.0009

Despite similar p-values, intuition suggests to raise flags only in the first case. The lack of "resolution" in the second case makes it look like the unlucky event happened, and there still ample evidence to suggest that the hypothesized distribution is still likely.

Is the intuition just wrong? Or is there a basis for it? I can imagine if I explore the likely parameter values that could explain the observed data, they would probably follow beta distribution, which would be bell shaped for the first case, and heavily skewed in the second case, suggesting larger range of parameters that would still explain the data.

Aucun commentaire:

Enregistrer un commentaire