ToyCalculator

class pyhf.infer.calculators.ToyCalculator(data, pdf, init_pars=None, par_bounds=None, fixed_params=None, test_stat='qtilde', ntoys=2000, track_progress=True)[source]

Bases: object

The Toy-based Calculator.

__init__(data, pdf, init_pars=None, par_bounds=None, fixed_params=None, test_stat='qtilde', ntoys=2000, track_progress=True)[source]

Toy-based Calculator.

Parameters:

  • data (tensor) – The observed data.

  • pdf (Model) – The statistical model adhering to the schema model.json.

  • init_pars (tensor of float) – The starting values of the model parameters for minimization.

  • par_bounds (tensor) – The extrema of values the model parameters are allowed to reach in the fit. The shape should be (n, 2) for n model parameters.

  • fixed_params (tuple or list of bool) – The flag to set a parameter constant to its starting value during minimization.

  • test_stat (str) –

    The test statistic to use as a numerical summary of the data: 'qtilde', 'q', or 'q0'.

    • 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), as defined under the Wald approximation in Equation (62) of [1007.1727] (qmu_tilde()).

    • 'q': performs the calculation using the test statistic \(q_{\mu}\) (qmu()).

    • 'q0': performs the calculation using the discovery test statistic \(q_{0}\) (q0()).

  • ntoys (int) – Number of toys to use (how many times to sample the underlying distributions).

  • track_progress (bool) – Whether to display the tqdm progress bar or not (outputs to stderr).

Returns:

The calculator for toy-based quantities.

Return type:

ToyCalculator

Methods

distributions(poi_test, track_progress=None)[source]

Probability distributions of the test statistic value under the signal + background and background-only hypotheses.

These distributions are produced by generating pseudo-data (“toys”) with the nuisance parameters set to their conditional maximum likelihood estimators at the corresponding value of the parameter of interest for each hypothesis, following the joint recommendations of the ATLAS and CMS experiments in Procedure for the LHC Higgs boson search combination in Summer 2011.

Example

>>> import pyhf
>>> import numpy.random as random
>>> random.seed(0)
>>> pyhf.set_backend("numpy")
>>> model = pyhf.simplemodels.uncorrelated_background(
...     signal=[12.0, 11.0], bkg=[50.0, 52.0], bkg_uncertainty=[3.0, 7.0]
... )
>>> observations = [51, 48]
>>> data = observations + model.config.auxdata
>>> mu_test = 1.0
>>> toy_calculator = pyhf.infer.calculators.ToyCalculator(
...     data, model, ntoys=100, track_progress=False
... )
>>> sig_plus_bkg_dist, bkg_dist = toy_calculator.distributions(mu_test)
>>> sig_plus_bkg_dist.pvalue(mu_test), bkg_dist.pvalue(mu_test)
(array(0.14), array(0.79))
Parameters:

  • poi_test (float or tensor) – The value for the parameter of interest.

  • track_progress (bool) – Whether to display the tqdm progress bar or not (outputs to stderr)

Returns:

The distributions under the hypotheses.

Return type:

Tuple (EmpiricalDistribution)

expected_pvalues(sig_plus_bkg_distribution, bkg_only_distribution)[source]

Calculate the \(\mathrm{CL}_{s}\) values corresponding to the median significance of variations of the signal strength from the background only hypothesis \(\left(\mu=0\right)\) at \((-2,-1,0,1,2)\sigma\).

Example

>>> import pyhf
>>> import numpy.random as random
>>> random.seed(0)
>>> pyhf.set_backend("numpy")
>>> model = pyhf.simplemodels.uncorrelated_background(
...     signal=[12.0, 11.0], bkg=[50.0, 52.0], bkg_uncertainty=[3.0, 7.0]
... )
>>> observations = [51, 48]
>>> data = observations + model.config.auxdata
>>> mu_test = 1.0
>>> toy_calculator = pyhf.infer.calculators.ToyCalculator(
...     data, model, ntoys=100, track_progress=False
... )
>>> sig_plus_bkg_dist, bkg_dist = toy_calculator.distributions(mu_test)
>>> CLsb_exp_band, CLb_exp_band, CLs_exp_band = toy_calculator.expected_pvalues(sig_plus_bkg_dist, bkg_dist)
>>> CLs_exp_band
[array(0.), array(0.), array(0.08403955), array(0.21892596), array(0.86072977)]
Parameters:

  • sig_plus_bkg_distribution (EmpiricalDistribution) – The distribution for the signal + background hypothesis.

  • bkg_only_distribution (EmpiricalDistribution) – The distribution for the background-only hypothesis.

Returns:

The \(p\)-values for the test statistic corresponding to the \(\mathrm{CL}_{s+b}\), \(\mathrm{CL}_{b}\), and \(\mathrm{CL}_{s}\).

Return type:

Tuple (tensor)

pvalues(teststat, sig_plus_bkg_distribution, bkg_only_distribution)[source]

Calculate the \(p\)-values for the observed test statistic under the signal + background and background-only model hypotheses.

Example

>>> import pyhf
>>> import numpy.random as random
>>> random.seed(0)
>>> pyhf.set_backend("numpy")
>>> model = pyhf.simplemodels.uncorrelated_background(
...     signal=[12.0, 11.0], bkg=[50.0, 52.0], bkg_uncertainty=[3.0, 7.0]
... )
>>> observations = [51, 48]
>>> data = observations + model.config.auxdata
>>> mu_test = 1.0
>>> toy_calculator = pyhf.infer.calculators.ToyCalculator(
...     data, model, ntoys=100, track_progress=False
... )
>>> q_tilde = toy_calculator.teststatistic(mu_test)
>>> sig_plus_bkg_dist, bkg_dist = toy_calculator.distributions(mu_test)
>>> CLsb, CLb, CLs = toy_calculator.pvalues(q_tilde, sig_plus_bkg_dist, bkg_dist)
>>> CLsb, CLb, CLs
(array(0.03), array(0.37), array(0.08108108))
Parameters:

  • teststat (tensor) – The test statistic.

  • sig_plus_bkg_distribution (EmpiricalDistribution) – The distribution for the signal + background hypothesis.

  • bkg_only_distribution (EmpiricalDistribution) – The distribution for the background-only hypothesis.

Returns:

The \(p\)-values for the test statistic corresponding to the \(\mathrm{CL}_{s+b}\), \(\mathrm{CL}_{b}\), and \(\mathrm{CL}_{s}\).

Return type:

Tuple (tensor)

teststatistic(poi_test)[source]

Compute the test statistic for the observed data under the studied model.

Example

>>> import pyhf
>>> import numpy.random as random
>>> random.seed(0)
>>> pyhf.set_backend("numpy")
>>> model = pyhf.simplemodels.uncorrelated_background(
...     signal=[12.0, 11.0], bkg=[50.0, 52.0], bkg_uncertainty=[3.0, 7.0]
... )
>>> observations = [51, 48]
>>> data = observations + model.config.auxdata
>>> mu_test = 1.0
>>> toy_calculator = pyhf.infer.calculators.ToyCalculator(
...     data, model, ntoys=100, track_progress=False
... )
>>> toy_calculator.teststatistic(mu_test)
array(3.93824492)
Parameters:

poi_test (float or tensor) – The value for the parameter of interest.

Returns:

The value of the test statistic.

Return type:

Tensor