# hepstats.hypotests.calculators.basecalculator module#

class hepstats.hypotests.calculators.basecalculator.BaseCalculator(input, minimizer)[source]#

Base class for calculator.

Parameters:
• input – loss or fit result

• minimizer – minimizer to use to find the minimum of the loss function

Example with zfit:
>>> import zfit
>>> from zfit.core.loss import UnbinnedNLL
>>> from zfit.minimize import Minuit
>>>
>>> obs = zfit.Space('x', limits=(0.1, 2.0))
>>> data = zfit.data.Data.from_numpy(obs=obs, array=np.random.normal(1.2, 0.1, 10000))
>>> mean = zfit.Parameter("mu", 1.2)
>>> sigma = zfit.Parameter("sigma", 0.1)
>>> model = zfit.pdf.Gauss(obs=obs, mu=mean, sigma=sigma)
>>> loss = UnbinnedNLL(model=model, data=data)
>>>
>>> calc = BaseCalculator(input=loss, minimizer=Minuit())

obs_nll(pois)[source]#

Compute observed negative log-likelihood values for given parameters of interest.

Parameters:

pois (POIarray) – parameters of interest.

Return type:

ndarray

Returns:

Observed nll values.

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poi = POI(mean, [1.1, 1.2, 1.0])
>>> nll = calc.obs_nll(poi)

qobs(poinull, onesided=True, onesideddiscovery=True, qtilde=False)[source]#

Computes observed values of the $$\Delta$$ log-likelihood test statistic.

Parameters:
• poinull (POI) – parameters of interest for the null hypothesis.

• qtilde (bool) – if True use the $$\tilde{q}$$ test statistics else (default) use the $$q$$ test statistic.

• onesided (bool) – if True (default) computes onesided pvalues.

• onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery test.

Returns:

Observed values of q.

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poi = POI(mean, [1.1, 1.2, 1.0])
>>> q = calc.qobs(poi)

pvalue(poinull, poialt=None, qtilde=False, onesided=True, onesideddiscovery=False)[source]#

Computes pvalues for the null and alternative hypothesis.

Parameters:
• poinull () – parameters of interest for the null hypothesis.

• poialt (POI | None) – parameters of interest for the alternative hypothesis.

• qtilde (bool) – if True use the $$\widetilde{q}$$ test statistics else (default) use the $$q$$ test statistic.

• onesided (bool) – if True (default) computes onesided pvalues.

• onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery test.

Return type:

tuple[ndarray, ndarray]

Returns:

Tuple of arrays for pnull, palt

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poinull = POI(mean, [1.1, 1.2, 1.0])
>>> poialt = POI(mean, 1.2)
>>> pvalues = calc.pavalue(poinull, poialt)

expected_pvalue(poinull, poialt, nsigma, CLs=False, qtilde=False, onesided=True, onesideddiscovery=False)[source]#

Computes the expected pvalues and error bands for different values of $$\sigma$$ (0=expected/median)

Parameters:
• poinull () – parameters of interest for the null hypothesis.

• poialt () – parameters of interest for the alternative hypothesis.

• nsigma (list[int]) – list of values of $$\sigma$$ to compute the expected pvalue.

• CLs (bool) – if True computes pvalues as $$p_{cls}=p_{null}/p_{alt}=p_{clsb}/p_{clb}$$ else as $$p_{clsb} = p_{null}$$.

• qtilde (bool) – if True use the $$\widetilde{q}$$ test statistics else (default) use the $$q$$ test statistic.

• onesided (bool) – if True (default) computes onesided pvalues.

• onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery.

Return type:

list[array]

Returns:

Array of expected pvalues for each $$\sigma$$ value

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poinull = POI(mean, [1.1, 1.2, 1.0])
>>> poialt = POI(mean, 1.2)
>>> nll = calc.expected_pvalue(poinull, poialt)

static check_pois(pois)[source]#

Checks if the parameter of interest is a hepstats.parameters.POIarray instance.

Parameters:

pois () – the parameter of interest to check.

Raises:

TypeError – if pois is not an instance of hepstats.parameters.POIarray.

static check_pois_compatibility(poi1, poi2)[source]#

Checks compatibility between two lists of hepstats.parameters.POIarray() instances.

Parameters:
• poi1 () – the first parameter of interest.

• poi2 () – the second parameter of interest.

Raises:

ValueError – if the two parameters of interests don’t have the same parameters, check by their names.

q(nll1, nll2, poi1, poi2, onesided=True, onesideddiscovery=False)[source]#

Compute values of the test statistic q defined as the difference between negative log-likelihood values $$q = nll1 - nll2$$.

Parameters:
• nll1 (array) – array of nll values #1, evaluated with poi1.

• nll2 (array) – array of nll values #2, evaluated with poi2.

• poi1 (POIarray) – POI’s #1.

• poi2 (POIarray) – POI’s #2.

• onesided (bool) – if True (default) computes onesided pvalues.

• onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery.

Return type:

ndarray

Returns:

Array of $$q$$ values.

property bestfit#

Returns the best fit values of the model parameters.

property constraints#

Returns the constraints on the loss / likehood function.

property data#

Returns the data.

get_parameter(name)#

Returns the parameter in loss function with given input name.

Parameters:

name (str) – name of the parameter to return

property loss#

Returns the loss / likelihood function.

lossbuilder(model, data, weights=None, oldloss=None)#

Method to build a new loss function.

Parameters:
• model (*) – The model or models to evaluate the data on

• data (*) – Data to use

• weights (*) – the data weights

• oldloss (*) – Previous loss that has data, models, type

Example with zfit:
>>> data = zfit.data.Data.from_numpy(obs=obs, array=np.random.normal(1.2, 0.1, 10000))
>>> mean = zfit.Parameter("mu", 1.2)
>>> sigma = zfit.Parameter("sigma", 0.1)
>>> model = zfit.pdf.Gauss(obs=obs, mu=mean, sigma=sigma)
>>> loss = calc.lossbuilder(model, data)

Returns:

Loss function

property minimizer#

Returns the minimizer.

property model#

Returns the model.

property parameters#

Returns the list of free parameters in loss / likelihood function.

set_params_to_bestfit()#

Set the values of the parameters in the models to the best fit values

class hepstats.hypotests.calculators.basecalculator.BaseToysCalculator(input, minimizer, sampler, sample)[source]#

Bases: BaseCalculator

Basis for toys calculator class.

Parameters:
• input – loss or fit result.

• minimizer – minimizer to use to find the minimum of the loss function.

• sampler (Callable) – function used to create sampler with models, number of events and floating parameters in the sample.

• sample (Callable) – function used to get samples from the sampler.

property bestfit#

Returns the best fit values of the model parameters.

static check_pois(pois)#

Checks if the parameter of interest is a hepstats.parameters.POIarray instance.

Parameters:

pois () – the parameter of interest to check.

Raises:

TypeError – if pois is not an instance of hepstats.parameters.POIarray.

static check_pois_compatibility(poi1, poi2)#

Checks compatibility between two lists of hepstats.parameters.POIarray() instances.

Parameters:
• poi1 () – the first parameter of interest.

• poi2 () – the second parameter of interest.

Raises:

ValueError – if the two parameters of interests don’t have the same parameters, check by their names.

property constraints#

Returns the constraints on the loss / likehood function.

property data#

Returns the data.

expected_pvalue(poinull, poialt, nsigma, CLs=False, qtilde=False, onesided=True, onesideddiscovery=False)#

Computes the expected pvalues and error bands for different values of $$\sigma$$ (0=expected/median)

Parameters:
• poinull () – parameters of interest for the null hypothesis.

• poialt () – parameters of interest for the alternative hypothesis.

• nsigma (list[int]) – list of values of $$\sigma$$ to compute the expected pvalue.

• CLs (bool) – if True computes pvalues as $$p_{cls}=p_{null}/p_{alt}=p_{clsb}/p_{clb}$$ else as $$p_{clsb} = p_{null}$$.

• qtilde (bool) – if True use the $$\widetilde{q}$$ test statistics else (default) use the $$q$$ test statistic.

• onesided (bool) – if True (default) computes onesided pvalues.

• onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery.

Return type:

list[array]

Returns:

Array of expected pvalues for each $$\sigma$$ value

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poinull = POI(mean, [1.1, 1.2, 1.0])
>>> poialt = POI(mean, 1.2)
>>> nll = calc.expected_pvalue(poinull, poialt)

get_parameter(name)#

Returns the parameter in loss function with given input name.

Parameters:

name (str) – name of the parameter to return

property loss#

Returns the loss / likelihood function.

lossbuilder(model, data, weights=None, oldloss=None)#

Method to build a new loss function.

Parameters:
• model (*) – The model or models to evaluate the data on

• data (*) – Data to use

• weights (*) – the data weights

• oldloss (*) – Previous loss that has data, models, type

Example with zfit:
>>> data = zfit.data.Data.from_numpy(obs=obs, array=np.random.normal(1.2, 0.1, 10000))
>>> mean = zfit.Parameter("mu", 1.2)
>>> sigma = zfit.Parameter("sigma", 0.1)
>>> model = zfit.pdf.Gauss(obs=obs, mu=mean, sigma=sigma)
>>> loss = calc.lossbuilder(model, data)

Returns:

Loss function

property minimizer#

Returns the minimizer.

property model#

Returns the model.

obs_nll(pois)#

Compute observed negative log-likelihood values for given parameters of interest.

Parameters:

pois (POIarray) – parameters of interest.

Return type:

ndarray

Returns:

Observed nll values.

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poi = POI(mean, [1.1, 1.2, 1.0])
>>> nll = calc.obs_nll(poi)

property parameters#

Returns the list of free parameters in loss / likelihood function.

pvalue(poinull, poialt=None, qtilde=False, onesided=True, onesideddiscovery=False)#

Computes pvalues for the null and alternative hypothesis.

Parameters:
• poinull () – parameters of interest for the null hypothesis.

• poialt (POI | None) – parameters of interest for the alternative hypothesis.

• qtilde (bool) – if True use the $$\widetilde{q}$$ test statistics else (default) use the $$q$$ test statistic.

• onesided (bool) – if True (default) computes onesided pvalues.

• onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery test.

Return type:

tuple[ndarray, ndarray]

Returns:

Tuple of arrays for pnull, palt

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poinull = POI(mean, [1.1, 1.2, 1.0])
>>> poialt = POI(mean, 1.2)
>>> pvalues = calc.pavalue(poinull, poialt)

q(nll1, nll2, poi1, poi2, onesided=True, onesideddiscovery=False)#

Compute values of the test statistic q defined as the difference between negative log-likelihood values $$q = nll1 - nll2$$.

Parameters:
• nll1 (array) – array of nll values #1, evaluated with poi1.

• nll2 (array) – array of nll values #2, evaluated with poi2.

• poi1 (POIarray) – POI’s #1.

• poi2 (POIarray) – POI’s #2.

• onesided (bool) – if True (default) computes onesided pvalues.

• onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery.

Return type:

ndarray

Returns:

Array of $$q$$ values.

qobs(poinull, onesided=True, onesideddiscovery=True, qtilde=False)#

Computes observed values of the $$\Delta$$ log-likelihood test statistic.

Parameters:
• poinull (POI) – parameters of interest for the null hypothesis.

• qtilde (bool) – if True use the $$\tilde{q}$$ test statistics else (default) use the $$q$$ test statistic.

• onesided (bool) – if True (default) computes onesided pvalues.

• onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery test.

Returns:

Observed values of q.

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poi = POI(mean, [1.1, 1.2, 1.0])
>>> q = calc.qobs(poi)

set_params_to_bestfit()#

Set the values of the parameters in the models to the best fit values

class hepstats.hypotests.calculators.basecalculator.ToysCalculator(input, minimizer, ntoysnull=100, ntoysalt=100, sampler=<function base_sampler>, sample=<function base_sample>)[source]#

Class for calculators using toys.

Toys calculator class.

Parameters:
• input – loss or fit result.

• minimizer – minimizer to use to find the minimum of the loss function.

• ntoysnull (int) – minimum number of toys to generate for the null hypothesis.

• ntoysalt (int) – minimum number of toys to generate for the alternative hypothesis.

• sampler (Callable) – function used to create sampler with models, number of events and floating** parameters. in the sample Default is hepstats.utils.fit.sampling.base_sampler().

• sample (Callable) – function used to get samples from the sampler. Default is hepstats.utils.fit..sampling.base_sample().

classmethod from_yaml(filename, input, minimizer, sampler=<function base_sampler>, sample=<function base_sample>, **kwargs)[source]#

ToysCalculator constructor with the toys loaded from a yaml file.

Parameters:
• filename (str) – the yaml file name.

• input – loss or fit result.

• minimizer – minimizer to use to find the minimum of the loss function.

• sampler (Callable) – function used to create sampler with models, number of events and floating parameters in the sample Default is hepstats.utils.fit.sampling.base_sampler().

• sample (Callable) – function used to get samples from the sampler. Default is hepstats.fitutils.sampling.base_sample().

property ntoysnull: int#

Returns the number of toys generated for the null hypothesis.

property ntoysalt: int#

Returns the number of toys generated for the alternative hypothesis.

Parameters:

toy (ToyResult) – the toy result to add

property bestfit#

Returns the best fit values of the model parameters.

static check_pois(pois)#

Checks if the parameter of interest is a hepstats.parameters.POIarray instance.

Parameters:

pois () – the parameter of interest to check.

Raises:

TypeError – if pois is not an instance of hepstats.parameters.POIarray.

static check_pois_compatibility(poi1, poi2)#

Checks compatibility between two lists of hepstats.parameters.POIarray() instances.

Parameters:
• poi1 () – the first parameter of interest.

• poi2 () – the second parameter of interest.

Raises:

ValueError – if the two parameters of interests don’t have the same parameters, check by their names.

property constraints#

Returns the constraints on the loss / likehood function.

property data#

Returns the data.

expected_pvalue(poinull, poialt, nsigma, CLs=False, qtilde=False, onesided=True, onesideddiscovery=False)#

Computes the expected pvalues and error bands for different values of $$\sigma$$ (0=expected/median)

Parameters:
• poinull () – parameters of interest for the null hypothesis.

• poialt () – parameters of interest for the alternative hypothesis.

• nsigma (list[int]) – list of values of $$\sigma$$ to compute the expected pvalue.

• CLs (bool) – if True computes pvalues as $$p_{cls}=p_{null}/p_{alt}=p_{clsb}/p_{clb}$$ else as $$p_{clsb} = p_{null}$$.

• qtilde (bool) – if True use the $$\widetilde{q}$$ test statistics else (default) use the $$q$$ test statistic.

• onesided (bool) – if True (default) computes onesided pvalues.

• onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery.

Return type:

list[array]

Returns:

Array of expected pvalues for each $$\sigma$$ value

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poinull = POI(mean, [1.1, 1.2, 1.0])
>>> poialt = POI(mean, 1.2)
>>> nll = calc.expected_pvalue(poinull, poialt)

generate_and_fit_toys(ntoys, poigen, poieval)#

Generate and fit toys for at a given POI (poigen). The toys are then fitted, and the likelihood is profiled at the values of poigen and poieval.

Parameters:
• ntoys (int) – number of toys to generate

• poigen (POI) – POI used to generate the toys

• poieval (POIarray) – POI values to evaluate the loss function

get_parameter(name)#

Returns the parameter in loss function with given input name.

Parameters:

name (str) – name of the parameter to return

get_toyresult(poigen, poieval)#

Getter function.

Parameters:
Return type:

ToyResult

get_toys_null(poigen, poieval=None, qtilde=False)[source]#

Return the generated toys for the null hypothesis.

Parameters:
• poigen () – POI used to generate the toys.

• poieval (POI | POIarray | None) – POI values to evaluate the loss function.

• qtilde (bool) – if True use the :math: ilde{q} test statistics else (default) use the $$q$$ test statistic.

Return type:

dict[POI, ToyResult]

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poinull = POIarray(mean, [1.1, 1.2, 1.0])
>>> poialt = POI(mean, 1.2)
>>> for p in poinull:
...     calc.get_toys_alt(p, poieval=poialt)

keys()#

Returns keys of the ToysManager instance defined as key = (toy.poigen, toy.poieval) for a given ToyResult instance toy.

property loss#

Returns the loss / likelihood function.

lossbuilder(model, data, weights=None, oldloss=None)#

Method to build a new loss function.

Parameters:
• model (*) – The model or models to evaluate the data on

• data (*) – Data to use

• weights (*) – the data weights

• oldloss (*) – Previous loss that has data, models, type

Example with zfit:
>>> data = zfit.data.Data.from_numpy(obs=obs, array=np.random.normal(1.2, 0.1, 10000))
>>> mean = zfit.Parameter("mu", 1.2)
>>> sigma = zfit.Parameter("sigma", 0.1)
>>> model = zfit.pdf.Gauss(obs=obs, mu=mean, sigma=sigma)
>>> loss = calc.lossbuilder(model, data)

Returns:

Loss function

property minimizer#

Returns the minimizer.

property model#

Returns the model.

ntoys(poigen, poieval)#

Return the number of toys generated from given value of a POI, and scanned/evaluated for given values_equal of the same POI.

Parameters:
Return type:

int

obs_nll(pois)#

Compute observed negative log-likelihood values for given parameters of interest.

Parameters:

pois (POIarray) – parameters of interest.

Return type:

ndarray

Returns:

Observed nll values.

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poi = POI(mean, [1.1, 1.2, 1.0])
>>> nll = calc.obs_nll(poi)

property parameters#

Returns the list of free parameters in loss / likelihood function.

pvalue(poinull, poialt=None, qtilde=False, onesided=True, onesideddiscovery=False)#

Computes pvalues for the null and alternative hypothesis.

Parameters:
• poinull () – parameters of interest for the null hypothesis.

• poialt (POI | None) – parameters of interest for the alternative hypothesis.

• qtilde (bool) – if True use the $$\widetilde{q}$$ test statistics else (default) use the $$q$$ test statistic.

• onesided (bool) – if True (default) computes onesided pvalues.

• onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery test.

Return type:

tuple[ndarray, ndarray]

Returns:

Tuple of arrays for pnull, palt

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poinull = POI(mean, [1.1, 1.2, 1.0])
>>> poialt = POI(mean, 1.2)
>>> pvalues = calc.pavalue(poinull, poialt)

q(nll1, nll2, poi1, poi2, onesided=True, onesideddiscovery=False)#

Compute values of the test statistic q defined as the difference between negative log-likelihood values $$q = nll1 - nll2$$.

Parameters:
• nll1 (array) – array of nll values #1, evaluated with poi1.

• nll2 (array) – array of nll values #2, evaluated with poi2.

• poi1 (POIarray) – POI’s #1.

• poi2 (POIarray) – POI’s #2.

• onesided (bool) – if True (default) computes onesided pvalues.

• onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery.

Return type:

ndarray

Returns:

Array of $$q$$ values.

qobs(poinull, onesided=True, onesideddiscovery=True, qtilde=False)#

Computes observed values of the $$\Delta$$ log-likelihood test statistic.

Parameters:
• poinull (POI) – parameters of interest for the null hypothesis.

• qtilde (bool) – if True use the $$\tilde{q}$$ test statistics else (default) use the $$q$$ test statistic.

• onesided (bool) – if True (default) computes onesided pvalues.

• onesideddiscovery (bool) – if True (default) computes onesided pvalues for a discovery test.

Returns:

Observed values of q.

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poi = POI(mean, [1.1, 1.2, 1.0])
>>> q = calc.qobs(poi)

sample(sampler, ntoys, poi, constraints=None)#

Generator function of samples from the sampler for a given value of a parameter of interest. Returns a dictionnary of samples constraints in any.

Parameters:
• sampler (list) – generator of samples

• ntoys (int) – number of samples to generate

• poi (POI) – in the sampler

• constraints (list, optional) – list of constraints to sample

Example with zfit:
>>> mean = zfit.Parameter("mean")
>>> sampler = calc.sampler(floating_params=[mean])
>>> sample = calc.sample(sampler, 1000, POI(mean, 1.2))

Returns:

dictionnary of sampled values of the constraints at each iteration

sampler(floating_params=None)#

Create sampler with models.

Parameters:

floating_params – floating parameters in the sampler

Example with zfit:
>>> sampler = calc.sampler(floating_params=[zfit.Parameter("mean")])

set_params_to_bestfit()#

Set the values of the parameters in the models to the best fit values

to_yaml(filename)#

Save the toys into a yaml file under the key toys.

Parameters:

filename (str) – the yaml file name.

toyresults_to_dict()#

Returns a list of all the toy results converted into dictionaries.

Return type:

list[dict]

toys_loss(parameter_name)#

Construct a loss function constructed with a sampler for a given floating parameter

Parameters:

parameter_name (str) – name floating parameter in the sampler

Returns:

Loss function

Example with zfit:
>>> loss = calc.toys_loss(zfit.Parameter("mean"))

toysresults_from_yaml(filename)#

Extract toy results from a yaml file.

Parameters:

filename (str) – the yaml file name.

Return type:

list[ToyResult]

values()#

Returns values of ToysManager instance that are ToyResult instances.

get_toys_alt(poigen, poieval=None, qtilde=False)[source]#

Return the generated toys for the alternative hypothesis.

Parameters:
• poigen () – POI used to generate the toys.

• poieval (POI | POIarray | None) – POI values to evaluate the loss function.

• qtilde (bool) – if True use the :math: ilde{q} test statistics else (default) use the $$q$$ test statistic.

Return type:

dict[POI, ToyResult]

Example with zfit:
>>> mean = zfit.Parameter("mu", 1.2)
>>> poinull = POIarray(mean, [1.1, 1.2, 1.0])
>>> poialt = POI(mean, 1.2)
>>> calc.get_toys_alt(poialt, poieval=poinull)