integrator.app.utils module

integrator.app.utils module#

integrator.app.utils.get_computations_config(conf)#

Converts a dictionary to a ClusterConfiguration object

integrator.app.utils.get_ai_config(conf)#

Converts a dictionary to a AIConfiguration object

integrator.app.utils.browse_datasets_wildcard(wildcards, entry=None, detector_name=None, errors='print', extra_metadata_fields=[])#

Browse folders and look for datasets. This assumes that the datasets are stored in this layout: /path/to/samplename_num/samplename_num.h5

Parameters:

wildcards (str or list of str) – Wildcard or list of wildcards,

integrator.app.utils.get_output_folders(datasets, output_prefix, name_template='azint_{dataset}_{scan_num}_{detector}.h5', create_dirs=False, mode=None)#

From a list of datasets, returns a list of output paths corresponding to each file name.

Parameters:

  • datasets (list of HDF5Dataset) – List HDF5Dataset objects. The file path is expected to be in the ID15A layout “proposal/sample/dataset/scan(s)” example: /data/id15/inhouse6/ihxxx/id15/sampletest/sampletest_0008/sampletest_0008.h5 (possibly several scans therein)

  • output_prefix (str) – Common path where output files will be saved

  • name_template (str) – Template name. For example azint_{dataset}_{scan}_{detector}.h5 for azint_ACL9011001b_0009_scan0001_pilatus.h5

  • create_dirs (bool, optional) – Whether to create output directories. Default is False.

Returns:

  • output_files (list of str) – List of output files corresponding to each dataset

  • datasets_info (dict) – Nested dictionary with the information on samples/datasets/scans

integrator.app.utils.get_distributed_integrator_params(conf_file, overwritten_params=None, conf_dict=None, print_already_processed=True, ignore_already_processed=True, parsing_error_handling='print')#

Parse and validate a configuration file, return several data structures detailed below.

Parameters:

  • conf_file (str) – Path to the configuration file

  • overwritten_params (dict, optional) – Dictionary of parameters overwriting some options of the configuration file

  • conf_dict (dict, optional) – Configuration dict to use instead of first parameter ‘conf_file’.

  • print_already_processed (bool, optional) – Print datasets already processed (if existing_output = skip)

  • ignore_already_processed (bool, optional) – Remove already-processed datasets from the results list. Default is True

  • parsing_error_handling (str, optional) – How to handle datasets parsing errors. Can be “print”, “raise” or “pass”. Default is “print”.

Returns:

  • conf (dict) – Nested dictionary extracted from the configuration file

  • ai_config (AIConfiguration) – Data structure with the azimuthal integration configuration

  • comp_resources (ComputationResources) – Data structure with the computation resources

  • datasets (list of Dataset objects) – List where each item is a data structure with information on the dataset to process

  • output_files (list of str) – List where each item is the path to the output file, either a .nx file or a directory

Contents