Configuration parameters
Contents
Configuration parameters#
This file lists all the current configuration parameters available in the configuration file.
dataset#
Dataset(s) to integrate. This can be a comma-separated list, with wildcards. The HDF5-Nexus files have to contain entries like ‘1.1’.
location =
Entry in the HDF5 file. Default (empty) means that ALL the entries will be processed.
hdf5_entry =
Which folders layout to use for parsing datasets and creating output files. Available are: ID15A, ID11
layout = ID15A
azimuthal integration#
Number of azimuthal bins.
n_pts = 2500
Radial unit. Can be r_mm, q_A^-1, 2th_deg
unit = r_mm
Detector name (ex. eiger2_cdte_4m), or path to the a detector file specification (ex. id15_pilatus.h5)
detector =
Force detector name if not present in the ‘detector’ file description
detector_name =
Path to the mask file
mask_file =
Path to the flatfield file. If provided, flat-field normalization will be applied.
flatfield_file =
Path to the dark file. If provided, dark current will be subtracted from the raw data.
dark_file =
Path to the pyFAI calibration file
poni_file =
Error model for azimuthal integration. Can be poisson or None.
error_model = poisson
azimuthal ranges in the form (min, max, n_slices)
azimuthal_range = (-180., 180., 1)
Lower and upper range of the radial unit. If not provided, range is simply (data.min(), data.max()). Values outside the range are ignored.
radial_range =
Which azimuthal integration method to use.
ai_method = opencl
Polarization factor for PyFAI. Default is 1.0
polarization_factor = 1.0
Whether to correct solid angle. Put this parameter if you wish to correct solid angle.
correct_solid_angle = 0
Whether to additionally perform azimuthal integration on average of each image stack
average_xy = 0
Whether to perform pixel splitting.
pixel_splitting = 0
Which outliers removal method to use. Can be none (default), median, or sigma clip. NB: when using one of the latter two, neither azimuthal caking nor uncertainty estimation are possible, only one azimuthal slice is used.
trim_method =
Number of azimuthal bins for trimmed mean.
trim_n_pts =
Bounds for trim methods:
For median: percentiles in the form (cut_low, cut_high). Integration uses medfilt1d with only one azimuthal slice.
sigma clip: thresholds in the form (thresh_low, thresh_high).
trim_bounds =
computations distribution#
Number of workers to use
n_workers = 10
Number of CPU cores per workers to use
cores_per_worker = 10
Time limit for SLURM job duration
time = 02:00:00
Amount of memory per worker. Default is 10 GB.
memory_per_worker = 10GB
Name of the project to be displayed on SLURM
project_name = distributed-azimuthal-integration
Whether to automatically respawn SLURM resources once the walltime is reached.
auto_respawn = 1
If non-empty, connect to an existing dask-distributed cluster. It should be in the form ‘tcp://ip:port’
cluster_address =
Name of the SLURM partition (queue)
partition = gpu
Path to the python executable. Useful if the compute nodes have a different environment from the front-end. Defaults to current python executable (so accounts for virtual environments)
python_exe =
TCP Port for the dask-distributed scheduler
port = 36087
TCP Port for the dashboard
dashboard_port = 36088
Number of images processed in a single pass by each integrator. By default (empty), each integrator determines the optimal number of images to load.
n_images_per_integrator =
If auto_respawn is set to True, this defines the number of times the resources can be respawned.
max_respawns = 100
Period (in seconds)
healthcheck_period = 10
Defines a python executable to use for each SLURM partition. Mind the semicolon (;) as a separator.
python_executables = nice='/scisoft/tomotools_env/integrator/ubuntu20.04/x86_64/2022.1.3/bin/python' ; p9gpu='/scisoft/tomotools_env/integrator/ubuntu20.04/ppc64le/2022.1.3/bin/python' ; p9gpu-long='/scisoft/tomotools_env/integrator/ubuntu20.04/ppc64le/2022.1.3/bin/python' ; gpu='/scisoft/tomotools_env/integrator/ubuntu20.04/x86_64/2022.1.3/bin/python'
output#
Path to the output file. If not provided, it will be in the same directory as the input dataset. NOTA: a directory with the same name (less the extension) will be created at the same level, for storing the actual integrated data.
location =
What to do if output already exists. Possible values are:
skip: go to next image of the output already exists (and has the same processing configuration)
overwrite: re-do the processing, overwrite the file
raise: raise an error and exit
existing_output = skip
Whether to repack output data, meaning transform the virtual datasets to a contiguous dataset. If activated, the partial result files are deleted.
repack_output = 1
live integration#
pipeline#
Level of verbosity of the processing. 0 = terse, 3 = much information.
verbosity = 2