EmpiricalDistribution

class pyhf.infer.calculators.EmpiricalDistribution(samples)

Bases: object

The empirical distribution of the test statistic.

Unlike AsymptoticTestStatDistribution where the distribution for the test statistic is normally distributed, the \(p\)-values etc are computed from the sampled distribution.

__init__(samples)

Empirical distribution.

Parameters:

samples (tensor) – The test statistics sampled from the distribution.

Returns:

The empirical distribution of the test statistic.

Return type:

EmpiricalDistribution

Methods

expected_value(nsigma)

Return the expected value of the test statistic.

Examples

>>> import pyhf
>>> import numpy.random as random
>>> random.seed(0)
>>> pyhf.set_backend("numpy")
>>> mean = pyhf.tensorlib.astensor([5])
>>> std = pyhf.tensorlib.astensor([1])
>>> normal = pyhf.probability.Normal(mean, std)
>>> samples = normal.sample((100,))
>>> dist = pyhf.infer.calculators.EmpiricalDistribution(samples)
>>> dist.expected_value(nsigma=1)
6.15094381...

>>> import pyhf
>>> import numpy.random as random
>>> random.seed(0)
>>> pyhf.set_backend("numpy")
>>> model = pyhf.simplemodels.uncorrelated_background(
...     signal=[12.0, 11.0], bkg=[50.0, 52.0], bkg_uncertainty=[3.0, 7.0]
... )
>>> init_pars = model.config.suggested_init()
>>> par_bounds = model.config.suggested_bounds()
>>> fixed_params = model.config.suggested_fixed()
>>> mu_test = 1.0
>>> pdf = model.make_pdf(pyhf.tensorlib.astensor(init_pars))
>>> samples = pdf.sample((100,))
>>> dist = pyhf.infer.calculators.EmpiricalDistribution(
...     pyhf.tensorlib.astensor(
...         [
...             pyhf.infer.test_statistics.qmu_tilde(
...                 mu_test, sample, model, init_pars, par_bounds, fixed_params
...             )
...             for sample in samples
...         ]
...     )
... )
>>> n_sigma = pyhf.tensorlib.astensor([-2, -1, 0, 1, 2])
>>> dist.expected_value(n_sigma)
array([0.00000000e+00, 0.00000000e+00, 5.53671231e-04, 8.29987137e-01,
       2.99592664e+00])

Parameters:

nsigma (int or tensor) – The number of standard deviations.

Returns:

The expected value of the test statistic.

Return type:

Float

pvalue(value)

Compute the \(p\)-value for a given value of the test statistic.

Examples

>>> import pyhf
>>> import numpy.random as random
>>> random.seed(0)
>>> pyhf.set_backend("numpy")
>>> mean = pyhf.tensorlib.astensor([5])
>>> std = pyhf.tensorlib.astensor([1])
>>> normal = pyhf.probability.Normal(mean, std)
>>> samples = normal.sample((100,))
>>> dist = pyhf.infer.calculators.EmpiricalDistribution(samples)
>>> dist.pvalue(7)
array(0.02)

>>> import pyhf
>>> import numpy.random as random
>>> random.seed(0)
>>> pyhf.set_backend("numpy")
>>> model = pyhf.simplemodels.uncorrelated_background(
...     signal=[12.0, 11.0], bkg=[50.0, 52.0], bkg_uncertainty=[3.0, 7.0]
... )
>>> init_pars = model.config.suggested_init()
>>> par_bounds = model.config.suggested_bounds()
>>> fixed_params = model.config.suggested_fixed()
>>> mu_test = 1.0
>>> pdf = model.make_pdf(pyhf.tensorlib.astensor(init_pars))
>>> samples = pdf.sample((100,))
>>> test_stat_dist = pyhf.infer.calculators.EmpiricalDistribution(
...     pyhf.tensorlib.astensor(
...         [pyhf.infer.test_statistics.qmu_tilde(mu_test, sample, model, init_pars, par_bounds, fixed_params) for sample in samples]
...     )
... )
>>> test_stat_dist.pvalue(test_stat_dist.samples[9])
array(0.3)

Parameters:

value (float) – The test statistic value.

Returns:

The integrated probability to observe a value at least as large as the observed one.

Return type:

Tensor