pyhf.infer.hypotest#

pyhf.infer.hypotest(poi_test, data, pdf, init_pars=None, par_bounds=None, fixed_params=None, calctype='asymptotics', return_tail_probs=False, return_expected=False, return_expected_set=False, return_calculator=False, **kwargs)[source]#

Compute $p$-values and test statistics for a single value of the parameter of interest.

See AsymptoticCalculator and ToyCalculator on additional keyword arguments to be specified.

Example

>>> import pyhf
>>> pyhf.set_backend("numpy")
>>> model = pyhf.simplemodels.uncorrelated_background(
...     signal=[12.0, 11.0], bkg=[50.0, 52.0], bkg_uncertainty=[3.0, 7.0]
... )
>>> observations = [51, 48]
>>> data = pyhf.tensorlib.astensor(observations + model.config.auxdata)
>>> mu_test = 1.0
>>> CLs_obs, CLs_exp_band = pyhf.infer.hypotest(
...     mu_test, data, model, return_expected_set=True, test_stat="qtilde"
... )
>>> CLs_obs
array(0.05251497)
>>> CLs_exp_band
[array(0.00260626), array(0.01382005), array(0.06445321), array(0.23525644), array(0.57303621)]

Parameters:

• poi_test (Number or Tensor) – The value of the parameter of interest (POI)

• data (Number or Tensor) – The data considered

• pdf (Model) – The statistical model adhering to the schema model.json

• init_pars (tensor of float) – The starting values of the model parameters for minimization.

• par_bounds (tensor) – The extrema of values the model parameters are allowed to reach in the fit. The shape should be (n, 2) for n model parameters.

• fixed_params (tuple or list of bool) – The flag to set a parameter constant to its starting value during minimization.

• calctype (str) – The calculator to create. Choose either ‘asymptotics’ (default) or ‘toybased’.

• return_tail_probs (bool) – Bool for returning ${\mathrm{CL}}_{s+b}$ and ${\mathrm{CL}}_b$

• return_expected (bool) – Bool for returning ${\mathrm{CL}}_{\exp }$

• return_expected_set (bool) – Bool for returning the $(-2,-1,0,1,2)\sigma$ ${\mathrm{CL}}_{\exp }$ — the “Brazil band”

• return_calculator (bool) – Bool for returning calculator.

Returns:

Tuple of Floats and lists of Floats and a AsymptoticCalculator or ToyCalculator instance:

• ${\mathrm{CL}}_s$: The modified $p$-value compared to the given threshold $\alpha$, typically taken to be $0.05$, defined in [1007.1727] as

$${\mathrm{CL}}_s=\frac{{\mathrm{CL}}_{s+b}}{{\mathrm{CL}}_b}=\frac{p_{s+b}}{1-p_b}$$

to protect against excluding signal models in which there is little sensitivity. In the case that ${\mathrm{CL}}_s\le \alpha$ the given signal model is excluded.

• $[{\mathrm{CL}}_{s+b},{\mathrm{CL}}_b]$: The signal + background model hypothesis $p$-value

$${\mathrm{CL}}_{s+b}=p_{s+b}=p(q\ge q_{\mathrm{obs}}|s+b)=\underset{q_{\mathrm{obs}}}{\overset{\mathrm{\infty }}{\int }}f\left(q\right|s+b)dq=1-F(q_{\mathrm{obs}}(\mu )|{\mu }^{\prime })$$

and 1 minus the background only model hypothesis $p$-value

$${\mathrm{CL}}_b=1-p_b=p(q\ge q_{\mathrm{obs}}|b)=1-\underset{-\mathrm{\infty }}{\overset{q_{\mathrm{obs}}}{\int }}f\left(q\right|b)dq=1-F(q_{\mathrm{obs}}(\mu )|0)$$

for signal strength $\mu$ and model hypothesis signal strength ${\mu }^{\prime }$, where the cumulative density functions $F(q(\mu )|{\mu }^{\prime })$ are given by Equations (57) and (65) of [1007.1727] for upper-limit-like test statistic $q\in \{q_{\mu },{\tilde{q}}_{\mu }\}$. Only returned when return_tail_probs is True.

Note

The definitions of the ${\mathrm{CL}}_{s+b}$ and ${\mathrm{CL}}_b$ used are based on profile likelihood ratio test statistics. This procedure is common in the LHC-era, but differs from procedures used in the LEP and Tevatron eras, as briefly discussed in $\mathrm{\S }$ 3.8 of [1007.1727].

• ${\mathrm{CL}}_{s,\exp }$: The expected ${\mathrm{CL}}_s$ value corresponding to the test statistic under the background only hypothesis $(\mu =0)$. Only returned when return_expected is True.

• ${\mathrm{CL}}_{s,\exp }$ band: The set of expected ${\mathrm{CL}}_s$ values corresponding to the median significance of variations of the signal strength from the background only hypothesis $(\mu =0)$ at $(-2,-1,0,1,2)\sigma$. That is, the $p$-values that satisfy Equation (89) of [1007.1727]

$${\mathrm{band}}_{N\sigma }={\mu }^{\prime }+\sigma {\mathrm{\Phi }}^{-1}(1-\alpha )\pm N\sigma$$

for ${\mu }^{\prime }=0$ and $N\in \{-2,-1,0,1,2\}$. These values define the boundaries of an uncertainty band sometimes referred to as the “Brazil band”. Only returned when return_expected_set is True.

• a calculator: The calculator instance used in the computation of the $p$-values. Either an instance of AsymptoticCalculator or ToyCalculator, depending on the value of calctype. Only returned when return_calculator is True.