pyFAI.utils package

pyFAI.utils.bayes module

Bayesian evaluation of background for 1D powder diffraction pattern.

Code according to Sivia and David, J. Appl. Cryst. (2001). 34, 318-324

• Version: 0.1 2012/03/28

• Version: 0.2 2016/10/07: OOP implementation

class pyFAI.utils.bayes.BayesianBackground

Bases: object

This class estimates the background of a powder diffraction pattern

http://journals.iucr.org/j/issues/2001/03/00/he0278/he0278.pdf

The log likelihood is described in correspond to eq7 of the paper:

\[z = y / sigma^2\]

• if z<0 a quadratic behaviour is expected

• if z>>1 it is likely a bragg peak so the penalty should be small: log(z).

• The spline is used to have a quadratic behaviour near 0 and the log one near the junction

The threshold is taken at 8 as erf is 1 above 6: The points 6, 7 and 8 are used in the spline to ensure a continuous junction with the logarithmic continuation.

PREFACTOR = 1

__init__()

background_image(img, sigma=None, mask=None, npt=10, k=3)

classmethod bayes_llk(z)

Calculate actually the log-likelihood from a set of weighted error

Re implementation of the following code even slightly faster:

(y<=0)*5*y**2 + (y>0)*(y<8)*pyFAI.utils.bayes.background.spline(y) + (y>=8)*(s1+log(abs(y)+1*(y<8)))

Parameters:

z (float[:]) – weighted error

Returns:

log likelihood

Return type:

float[:]

classmethod bayes_llk_large(z)

used to calculate the log-likelihood of large positive values: logarithmic

classmethod bayes_llk_negative(z)

used to calculate the log-likelihood of negative values: quadratic

classmethod bayes_llk_small(z)

used to calculate the log-likelihood of small positive values: fitted with spline

classmethod classinit()

classmethod func2d_min(values, d0_sparse, d1_sparse, d0_pos, d1_pos, y_obs, w_obs, valid, k)

Function to optimize

Parameters:

• values – values of the background on spline knots

• d0_sparse – positions along slowest axis of the spline knots

• d1_pos – positions along fastest axis of the spline knots

• d0_pos – positions along slowest axis (all coordinates)

• d1_pos – positions along fastest axis (all coordinates)

• y_obs – intensities actually measured

• w_obs – weights of the experimental points

• valid – coordinated of valid pixels

• k – order of the spline, usually 3

Returns:

sum of the log-likelihood to be minimized

classmethod func_min(y0, x_obs, y_obs, w_obs, x0, k)

Function to optimize

Parameters:

• y0 – values of the background

• x_obs – experimental values

• y_obs – experimental values

• w_obs – weights of the experimental points

• x0 – position of evaluation of the spline

• k – order of the spline, usually 3

Returns:

sum of the log-likelihood to be minimized

s1 = np.float64(2.5435828321944816)

spline = <scipy.interpolate._fitpack2.InterpolatedUnivariateSpline object>

classmethod test_bayes_llk()

Test plot of log(likelihood) Similar to as figure 3 of Sivia and David, J. Appl. Cryst. (2001). 34, 318-324

pyFAI.utils.decorators module

Bunch of useful decorators

pyFAI.utils.decorators.deprecated(func=None, reason=None, replacement=None, since_version=None, only_once=False, skip_backtrace_count=1, deprecated_since=None)

Decorator that deprecates the use of a function

Parameters:

• reason (str) – Reason for deprecating this function (e.g. “feature no longer provided”,

• replacement (str) – Name of replacement function (if the reason for deprecating was to rename the function)

• since_version (str) – First pyFAI version for which the function was deprecated (e.g. “0.5.0”).

• only_once (bool) – If true, the deprecation warning will only be generated one time. Default is true.

• skip_backtrace_count (int) – Amount of last backtrace to ignore when logging the backtrace

• deprecated_since (Union[int,str]) – If provided, log it as warning since a version of the library, else log it as debug

pyFAI.utils.decorators.deprecated_warning(type_, name, reason=None, replacement=None, since_version=None, only_once=True, skip_backtrace_count=0, deprecated_since=None)

Function to log a deprecation warning

Parameters:

• type (str) – Nature of the object to be deprecated: “Module”, “Function”, “Class” …

• name – Object name.

• reason (str) – Reason for deprecating this function (e.g. “feature no longer provided”,

• replacement (str) – Name of replacement function (if the reason for deprecating was to rename the function)

• since_version (str) – First pyFAI version for which the function was deprecated (e.g. “0.5.0”).

• only_once (bool) – If true, the deprecation warning will only be generated one time for each different call locations. Default is true.

• skip_backtrace_count (int) – Amount of last backtrace to ignore when logging the backtrace

• deprecated_since (Union[int,str]) – If provided, log the deprecation as warning since a version of the library, else log it as debug.

pyFAI.utils.decorators.timeit(func)

pyFAI.utils.ellipse module

This modules contains a function to fit without refinement an ellipse on a set of points ….

class pyFAI.utils.ellipse.Ellipse(center_1, center_2, angle, half_long_axis, half_short_axis)

Bases: NamedTuple

angle: float

Angle in radian

center_1: float

Center position in the slow axis (pty)

center_2: float

Center position in the fast axis (ptx)

half_long_axis: float

Alias for field number 3

half_short_axis: float

Alias for field number 4

pyFAI.utils.ellipse.fit_ellipse(pty, ptx, _allow_delta=True)

Fit an ellipse

Math from https://mathworld.wolfram.com/Ellipse.html #15

inspired from http://nicky.vanforeest.com/misc/fitEllipse/fitEllipse.html

Parameters:

• pty – point coordinates in the slow dimension (y)

• ptx – point coordinates in the fast dimension (x)

Raises:

ValueError – If the ellipse can’t be fitted

pyFAI.utils.header_utils module

This modules contains helper function relative to image header.

exception pyFAI.utils.header_utils.MonitorNotFound

Bases: Exception

Raised when monitor information in not found or is not valid.

pyFAI.utils.header_utils.get_monitor_value(image, monitor_key)

Return the monitor value from an image using an header key.

Parameters:

• image (fabio.fabioimage.FabioImage) – Image containing the header

• monitor_key (str) – Key containing the monitor

Returns:

returns the monitor else raise an exception

Return type:

float

Raises:

MonitorNotFound – when the expected monitor is not found on the header

pyFAI.utils.logging_utils module

This modules contains helper function relative to logging system.

class pyFAI.utils.logging_utils.PrePostEmitStreamHandler(handler)

Bases: Handler

Handler to allow to hook a function before and after the emit function.

The main logging feature is delegated to a sub handler.

__init__(handler)

Initializes the instance - basically setting the formatter to None and the filter list to empty.

emit(record)

Call pre_emit function then delegate the emit to the sub handler.

post_emit()

pre_emit()

pyFAI.utils.logging_utils.prepost_emit_callback(logger, pre_callback, post_callback)

Context manager to add pre/post emit callback to a logger

pyFAI.utils.logging_utils.set_prepost_emit_callback(logger, pre_callback, post_callback)

Patch the logging system to have a working progress bar without glitch. pyFAI define a default handler then we have to rework it

Returns:

The new handler

pyFAI.utils.mathutil module

Utilities, mainly for image treatment

class pyFAI.utils.mathutil.LongestRunOfHeads

Bases: object

Implements the “longest run of heads” by Mark F. Schilling The College Mathematics Journal, Vol. 21, No. 3, (1990), pp. 196-207

See: http://www.maa.org/sites/default/files/pdf/upload_library/22/Polya/07468342.di020742.02p0021g.pdf

A(n, c)

Calculate A(number_of_toss, length_of_longest_run)

Parameters:

• n – number of coin toss in the experiment, an integer

• c – length of the longest run of

Returns:

The A parameter used in the formula

B(n, c)

Calculate B(number_of_toss, length_of_longest_run) to have either a run of Heads either a run of Tails

Parameters:

• n – number of coin toss in the experiment, an integer

• c – length of the longest run of

Returns:

The B parameter used in the formula

__init__()

We store already calculated values for (n,c)

probaHeadOrTail(n, c)

Calculate the probability of a longest run of head or tails to occur

Parameters:

• n – number of coin toss in the experiment, an integer

• c – length of the longest run of heads or tails, an integer

Returns:

The probablility of having c subsequent heads or tails in a n toss of fair coin

probaLongerRun(n, c)

Calculate the probability for the longest run of heads or tails to exceed the observed length

Parameters:

• n – number of coin toss in the experiment, an integer

• c – length of thee observed run of heads or tails, an integer

Returns:

The probablility of having more than c subsequent heads or tails in a n toss of fair coin

pyFAI.utils.mathutil.allclose_mod(a, b, modulo=6.283185307179586, **kwargs)

Returns True if the two arrays a & b are equal within the given tolerance modulo modulo; False otherwise.

Thanks to “Serguei Sokol” <sokol@insa-toulouse.fr>

pyFAI.utils.mathutil.binning(input_img, binsize, norm=True)

Parameters:

• input_img – input ndarray

• binsize – int or 2-tuple representing the size of the binning

• norm – if False, do average instead of sum

Returns:

binned input ndarray

pyFAI.utils.mathutil.center_of_mass(img)

Calculate the center of mass of of the array. Like scipy.ndimage.measurements.center_of_mass :param img: 2-D array :return: 2-tuple of float with the center of mass

pyFAI.utils.mathutil.chi_square(obt, ref)

Compute \(\sqrt{\sum \frac{4\cdot(obt-ref)^2}{(obt + ref)^2}}\).

This is done for symmetry reason between obt and ref

Parameters:

• obt (3-tuple of array of the same size containing position, intensity, variance) – obtained data

• obt – reference data

Returns:

Chi² value, lineary interpolated

pyFAI.utils.mathutil.cormap(ref, obt)

Calculate the probabily of two array to be the same based on the CorMap algorithm This is a simplifed implementation

pyFAI.utils.mathutil.deg2rad(dd, disc=1)

Convert degrees to radian in the range [-π->π[ or [0->2π[

Parameters:

dd – angle in degrees

Returns:

angle in radians in the selected range

pyFAI.utils.mathutil.dog(s1, s2, shape=None)

2D difference of gaussian typically 1 to 10 parameters

pyFAI.utils.mathutil.dog_filter(input_img, sigma1, sigma2, mode='reflect', cval=0.0)

2-dimensional Difference of Gaussian filter implemented with FFT

Parameters:

• input_img (array-like) – input_img array to filter

• sigma (scalar or sequence of scalars) – standard deviation for Gaussian kernel. The standard deviations of the Gaussian filter are given for each axis as a sequence, or as a single number, in which case it is equal for all axes.

• mode – {‘reflect’,’constant’,’nearest’,’mirror’, ‘wrap’}, optional The mode parameter determines how the array borders are handled, where cval is the value when mode is equal to ‘constant’. Default is ‘reflect’

• cval – scalar, optional Value to fill past edges of input if mode is ‘constant’. Default is 0.0

pyFAI.utils.mathutil.expand(input_img, sigma, mode='constant', cval=0.0)

Expand array a with its reflection on boundaries

Parameters:

• a – 2D array

• sigma – float or 2-tuple of floats.

• mode – “constant”, “nearest”, “reflect” or “mirror”

• cval – filling value used for constant, 0.0 by default

Nota: sigma is the half-width of the kernel. For gaussian convolution it is adviced that it is 4*sigma_of_gaussian

pyFAI.utils.mathutil.expand2d(vect, size2, vertical=True)

This expands a vector to a 2d-array.

The result is the same as:

if vertical:
    numpy.outer(numpy.ones(size2), vect)
else:
    numpy.outer(vect, numpy.ones(size2))

This is a ninja optimization: replace *1 with a memcopy, saves 50% of time at the ms level.

Parameters:

• vect – 1d vector

• size2 – size of the expanded dimension

• vertical – if False the vector is expanded to the first dimension. If True, it is expanded to the second dimension.

pyFAI.utils.mathutil.gaussian(M, std)

Return a Gaussian window of length M with standard-deviation std.

This differs from the scipy.signal.gaussian implementation as: - The default for sym=False (needed for gaussian filtering without shift) - This implementation is normalized

Parameters:

• M – length of the windows (int)

• std – standatd deviation sigma

The FWHM is 2*numpy.sqrt(2 * numpy.pi)*std

pyFAI.utils.mathutil.gaussian_filter(input_img, sigma, mode='reflect', cval=0.0, use_scipy=True)

2-dimensional Gaussian filter implemented with FFT

Parameters:

• input_img (array-like) – input array to filter

• sigma (scalar or sequence of scalars) – standard deviation for Gaussian kernel. The standard deviations of the Gaussian filter are given for each axis as a sequence, or as a single number, in which case it is equal for all axes.

• mode – {‘reflect’,’constant’,’nearest’,’mirror’, ‘wrap’}, optional The mode parameter determines how the array borders are handled, where cval is the value when mode is equal to ‘constant’. Default is ‘reflect’

• cval – scalar, optional Value to fill past edges of input if mode is ‘constant’. Default is 0.0

pyFAI.utils.mathutil.interp_filter(ary, out=None)

Interpolate missing values (nan or infinite) in a 1D array

Parameters:

• ary – 1D array

• out – destination array (use ary to avoid allocation)

Returns:

1D array

pyFAI.utils.mathutil.is_far_from_group_python(pt, lst_pts, d2)

Tells if a point is far from a group of points, distance greater than d2 (distance squared)

Parameters:

• pt – point of interest

• lst_pts – list of points

• d2 – minimum distance squarred

Returns:

True If the point is far from all others.

pyFAI.utils.mathutil.maximum_position(img)

Same as scipy.ndimage.measurements.maximum_position: Find the position of the maximum of the values of the array.

Parameters:

img – 2-D image

Returns:

2-tuple of int with the position of the maximum

pyFAI.utils.mathutil.measure_offset(img1, img2, method='numpy', withLog=False, withCorr=False)

Measure the actual offset between 2 images :param img1: ndarray, first image :param img2: ndarray, second image, same shape as img1 :param withLog: shall we return logs as well ? boolean :return: tuple of floats with the offsets

pyFAI.utils.mathutil.relabel(label, data, blured, max_size=None)

Relabel limits the number of region in the label array. They are ranked relatively to their max(I0)-max(blur(I0))

Parameters:

• label – a label array coming out of scipy.ndimage.measurement.label

• data – an array containing the raw data

• blured – an array containing the blurred data

• max_size – the max number of label wanted

Returns:

array like label

pyFAI.utils.mathutil.round_fft(N)

This function returns the integer >=N for which size the Fourier analysis is faster (fron the FFT point of view)

Credit: Alessandro Mirone, ESRF, 2012

Parameters:

N – interger on which one would like to do a Fourier transform

Returns:

integer with a better choice

pyFAI.utils.mathutil.roundfft(*args, **kwargs)

pyFAI.utils.mathutil.rwp(obt, ref, scale=1.0)

Compute \(\sqrt{\sum \frac{4\cdot(obt-ref)^2}{(obt + ref)^2}}\).

This is done for symmetry reason between obt and ref

Parameters:

• obt (2-list of array of the same size) – obtained data

• obt – reference data

• scale – scale obt intensity

Returns:

Rwp value, lineary interpolated

pyFAI.utils.mathutil.shift(input_img, shift_val)

Shift an array like scipy.ndimage.interpolation.shift(input_img, shift_val, mode=”wrap”, order=0) but faster :param input_img: 2d numpy array :param shift_val: 2-tuple of integers :return: shifted image

pyFAI.utils.mathutil.shiftFFT(*args, **kwargs)

pyFAI.utils.mathutil.shift_fft(input_img, shift_val, method='fft')

Do shift using FFTs

Shift an array like scipy.ndimage.interpolation.shift(input, shift, mode=”wrap”, order=”infinity”) but faster :param input_img: 2d numpy array :param shift_val: 2-tuple of float :return: shifted image

pyFAI.utils.mathutil.unBinning(*args, **kwargs)

pyFAI.utils.mathutil.unbinning(binnedArray, binsize, norm=True)

Parameters:

• binnedArray – input ndarray

• binsize – 2-tuple representing the size of the binning

• norm – if True (default) decrease the intensity by binning factor. If False, it is non-conservative

Returns:

unBinned input ndarray

pyFAI.utils.orderedset module

class pyFAI.utils.orderedset.OrderedSet(iterable=None)

Bases: MutableSet

__init__(iterable=None)

add(key)

Add an element.

discard(key)

Remove an element. Do not raise an exception if absent.

pop(last=True)

Return the popped value. Raise KeyError if empty.

pyFAI.utils.shell module

Module containing utilities around shell command line.

class pyFAI.utils.shell.ProgressBar(title, max_value, bar_width)

Bases: object

Progress bar in shell mode

__init__(title, max_value, bar_width)

Create a progress bar using a title, a maximum value and a graphical size.

The display is done with stdout using carriage return to to hide the previous progress. It is not possible to use stdout for something else whill a progress bar is in use.

The result looks like:

Title [■■■■■■      ]  50%  Message

Parameters:

• title (str) – Title displayed before the progress bar

• max_value (float) – The maximum value of the progress bar

• bar_width (int) – Size of the progressbar in the screen

clear()

Remove the progress bar from the display and move the cursor at the beginning of the line using carriage return.

display()

Display the progress bar to stdout

update(value, message='', max_value=None)

Update the progrss bar with the progress bar’s current value.

Set the progress bar’s current value, compute the percentage of progress and update the screen with. Carriage return is used first and then the content of the progress bar. The cursor is at the begining of the line.

Parameters:

• value (float) – progress bar’s current value

• message (str) – message displayed after the progress bar

• max_value (float) – If not none, update the maximum value of the progress bar

pyFAI.utils.stringutil module

Module containing enhanced string formatters.

class pyFAI.utils.stringutil.SafeFormatter

Bases: Formatter

Like default formater but unmatched keys are still present into the result string

get_field(field_name, args, kwargs)

pyFAI.utils.stringutil.latex_to_unicode(string)

Returns a unicode representation from latex strings used by pyFAI.

Note

The latex string could be removed from the pyFAI core.

Parameters:

string (str) – A latex string to convert

Return type:

str

pyFAI.utils.stringutil.safe_format(format_string, arguments)

Like default str.format but unmatching patterns will be still present into the result string.

Parameters:

• str (format_string) – Format string as defined in the default formatter.

• tuple (arguments dict or) – Arbitrary set of positional and keyword arguments.

Return type:

str

pyFAI.utils.stringutil.to_bool(string)

Returns a safe boolean from a string.

Raises:

ValueError – If the string do not contains a boolean information.

pyFAI.utils.stringutil.to_eng(value, fmt=None, space='')

Return an engineering notation for the numerical value

Parameters:

• value – the actual value

• fmt – the formating, for example “5.3f”

• space – can be used to insert a “_” im 1_k

Returns:

string

pyFAI.utils.stringutil.to_ordinal(number)

Returns a string from an ordinal value with it’s suffix.

Parameters:

number (int) – A number refering to a position

Return type:

str

pyFAI.utils.stringutil.to_scientific_unicode(value, digits=3)

Convert a float value into a string using scientific notation and superscript unicode character.

This avoid using HTML in some case, when Qt widget does not support it.

Parameters:

• value (float) – Value to convert to displayable string

• digits (int) – Number of digits expected (3 means 1.000).

Module contents

Module with miscelaneous tools

class pyFAI.utils.FixedParameters

Bases: set

Like a set, made for FixedParameters in geometry refinement

add_or_discard(key, value=True)

Add a value to a set if value, else discard it :param key: element to added or discared from set :type value: boolean. If None do nothing ! :return: None

pyFAI.utils.calc_checksum(ary, safe=True)

Calculate the checksum by default (or returns its buffer location if unsafe)

pyFAI.utils.convert_CamelCase(name)

convert a function name in CamelCase into camel_case

pyFAI.utils.expand_args(args)

Takes an argv and expand it (under Windows, cmd does not convert *.tif into a list of files. Keeps only valid files (thanks to glob)

Parameters:

args – list of files or wilcards

Returns:

list of actual args

pyFAI.utils.float_(val)

Convert anything to a float … or None if not applicable

pyFAI.utils.fully_qualified_name(obj)

Return the fully qualified name of an object

pyFAI.utils.get_calibration_dir()

get the full path of a calibration directory

Returns:

the full path of the calibrant file

pyFAI.utils.get_cl_file(resource)

get the full path of a openCL resource file

The resource name can be prefixed by the name of a resource directory. For example “silx:foo.png” identify the resource “foo.png” from the resource directory “silx”. See also silx.resources.register_resource_directory().

Parameters:

resource (str) – Resource name. File name contained if the opencl directory of the resources.

Returns:

the full path of the openCL source file

pyFAI.utils.get_ui_file(filename)

get the full path of a user-interface file

Returns:

the full path of the ui

pyFAI.utils.int_(val)

Convert anything to an int … or None if not applicable

class pyFAI.utils.lazy_property(fget)

Bases: object

meant to be used for lazy evaluation of an object attribute. property should represent non-mutable data, as it replaces itself.

__init__(fget)

pyFAI.utils.readFloatFromKeyboard(text, dictVar)

Read float from the keyboard ….

Parameters:

• text – string to be displayed

• dictVar – dict of this type: {1: [set_dist_min],3: [set_dist_min, set_dist_guess, set_dist_max]}

pyFAI.utils.str_(val)

Convert anything to a string … but None -> “”