AsymptoticCalculator

class pyhf.infer.calculators.AsymptoticCalculator(data, pdf, init_pars=None, par_bounds=None, fixed_params=None, test_stat='qtilde', calc_base_dist='normal')[source]

Bases: object

The Asymptotic Calculator.

__init__(data, pdf, init_pars=None, par_bounds=None, fixed_params=None, test_stat='qtilde', calc_base_dist='normal')[source]

Asymptotic Calculator.

Parameters:

  • data (tensor) – The observed data.

  • pdf (Model) – The statistical model adhering to the schema model.json.

  • init_pars (tensor of float) – The starting values of the model parameters for minimization.

  • par_bounds (tensor) – The extrema of values the model parameters are allowed to reach in the fit. The shape should be (n, 2) for n model parameters.

  • fixed_params (tuple or list of bool) – The flag to set a parameter constant to its starting value during minimization.

  • test_stat (str) –

    The test statistic to use as a numerical summary of the data: 'qtilde', 'q', or 'q0'.

    • 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), as defined under the Wald approximation in Equation (62) of [1007.1727] (qmu_tilde()).

    • 'q': performs the calculation using the test statistic \(q_{\mu}\) (qmu()).

    • 'q0': performs the calculation using the discovery test statistic \(q_{0}\) (q0()).

  • calc_base_dist (str) –

    The statistical distribution, 'normal' or 'clipped_normal', to use for calculating the \(p\)-values.

    • 'normal': (default) use the full Normal distribution in \(\hat{\mu}/\sigma\) space. Note that expected limits may correspond to unphysical test statistics from scenarios with the expected \(\hat{\mu} > \mu\).

    • 'clipped_normal': use a clipped Normal distribution in \(\hat{\mu}/\sigma\) space to avoid expected limits that correspond to scenarios with the expected \(\hat{\mu} > \mu\). This will properly cap the test statistic at 0, as noted in Equation (14) and Equation (16) in [1007.1727].

    The choice of calc_base_dist only affects the \(p\)-values for expected limits, and the default value will be changed in a future release.

Returns:

The calculator for asymptotic quantities.

Return type:

AsymptoticCalculator

Methods

distributions(poi_test)[source]

Probability distributions of the test statistic, as defined in \(\S\) 3 of [1007.1727] under the Wald approximation, under the signal + background and background-only hypotheses.

Example

>>> import pyhf
>>> pyhf.set_backend("numpy")
>>> model = pyhf.simplemodels.uncorrelated_background(
...     signal=[12.0, 11.0], bkg=[50.0, 52.0], bkg_uncertainty=[3.0, 7.0]
... )
>>> observations = [51, 48]
>>> data = observations + model.config.auxdata
>>> mu_test = 1.0
>>> asymptotic_calculator = pyhf.infer.calculators.AsymptoticCalculator(data, model, test_stat="qtilde")
>>> _ = asymptotic_calculator.teststatistic(mu_test)
>>> sig_plus_bkg_dist, bkg_dist = asymptotic_calculator.distributions(mu_test)
>>> sig_plus_bkg_dist.pvalue(mu_test), bkg_dist.pvalue(mu_test)
(array(0.00219262), array(0.15865525))
Parameters:

poi_test (float or tensor) – The value for the parameter of interest.

Returns:

The distributions under the hypotheses.

Return type:

Tuple (AsymptoticTestStatDistribution)

expected_pvalues(sig_plus_bkg_distribution, bkg_only_distribution)[source]

Calculate the \(\mathrm{CL}_{s}\) values corresponding to the median significance of variations of the signal strength from the background only hypothesis \(\left(\mu=0\right)\) at \((-2,-1,0,1,2)\sigma\).

Example

>>> import pyhf
>>> pyhf.set_backend("numpy")
>>> model = pyhf.simplemodels.uncorrelated_background(
...     signal=[12.0, 11.0], bkg=[50.0, 52.0], bkg_uncertainty=[3.0, 7.0]
... )
>>> observations = [51, 48]
>>> data = observations + model.config.auxdata
>>> mu_test = 1.0
>>> asymptotic_calculator = pyhf.infer.calculators.AsymptoticCalculator(
...     data, model, test_stat="qtilde"
... )
>>> _ = asymptotic_calculator.teststatistic(mu_test)
>>> sig_plus_bkg_dist, bkg_dist = asymptotic_calculator.distributions(mu_test)
>>> CLsb_exp_band, CLb_exp_band, CLs_exp_band = asymptotic_calculator.expected_pvalues(sig_plus_bkg_dist, bkg_dist)
>>> CLs_exp_band
[array(0.00260626), array(0.01382005), array(0.06445321), array(0.23525644), array(0.57303621)]
Parameters:
Returns:

The \(p\)-values for the test statistic corresponding to the \(\mathrm{CL}_{s+b}\), \(\mathrm{CL}_{b}\), and \(\mathrm{CL}_{s}\).

Return type:

Tuple (tensor)

pvalues(teststat, sig_plus_bkg_distribution, bkg_only_distribution)[source]

Calculate the \(p\)-values for the observed test statistic under the signal + background and background-only model hypotheses.

Example

>>> import pyhf
>>> pyhf.set_backend("numpy")
>>> model = pyhf.simplemodels.uncorrelated_background(
...     signal=[12.0, 11.0], bkg=[50.0, 52.0], bkg_uncertainty=[3.0, 7.0]
... )
>>> observations = [51, 48]
>>> data = observations + model.config.auxdata
>>> mu_test = 1.0
>>> asymptotic_calculator = pyhf.infer.calculators.AsymptoticCalculator(
...     data, model, test_stat="qtilde"
... )
>>> q_tilde = asymptotic_calculator.teststatistic(mu_test)
>>> sig_plus_bkg_dist, bkg_dist = asymptotic_calculator.distributions(mu_test)
>>> CLsb, CLb, CLs = asymptotic_calculator.pvalues(q_tilde, sig_plus_bkg_dist, bkg_dist)
>>> CLsb, CLb, CLs
(array(0.02332502), array(0.4441594), array(0.05251497))
Parameters:
Returns:

The \(p\)-values for the test statistic corresponding to the \(\mathrm{CL}_{s+b}\), \(\mathrm{CL}_{b}\), and \(\mathrm{CL}_{s}\).

Return type:

Tuple (tensor)

teststatistic(poi_test)[source]

Compute the test statistic for the observed data under the studied model.

The fitted parameters of the five fits that are implicitly run for each call of this method are afterwards accessible through the fitted_pars attribute, which is a HypoTestFitResults instance.

Example

>>> import pyhf
>>> pyhf.set_backend("numpy")
>>> model = pyhf.simplemodels.uncorrelated_background(
...     signal=[12.0, 11.0], bkg=[50.0, 52.0], bkg_uncertainty=[3.0, 7.0]
... )
>>> observations = [51, 48]
>>> data = observations + model.config.auxdata
>>> mu_test = 1.0
>>> asymptotic_calculator = pyhf.infer.calculators.AsymptoticCalculator(data, model, test_stat="qtilde")
>>> asymptotic_calculator.teststatistic(mu_test)
array(0.14043184)
>>> asymptotic_calculator.fitted_pars
HypoTestFitResults(asimov_pars=array([0.        , 1.0030482 , 0.96264534]), free_fit_to_data=array([0.        , 1.0030512 , 0.96266961]), free_fit_to_asimov=array([0.        , 1.00304893, 0.96263365]), fixed_poi_fit_to_data=array([1.        , 0.97224597, 0.87553894]), fixed_poi_fit_to_asimov=array([1.        , 0.97276864, 0.87142047]))
>>> asymptotic_calculator.fitted_pars.free_fit_to_asimov  # best-fit parameters to Asimov dataset
array([0.        , 1.00304893, 0.96263365])
Parameters:

poi_test (float or tensor) – The value for the parameter of interest.

Returns:

The value of the test statistic.

Return type:

Tensor