nabu.pipeline.fullfield.reconstruction module

nabu.pipeline.fullfield.reconstruction.variable_idxlen_sort(fname)[source]
class nabu.pipeline.fullfield.reconstruction.FullFieldReconstructor(process_config, logger=None, backend='cuda', extra_options=None, cuda_options=None)[source]

Bases: object

A reconstructor spawns and manages Pipeline objects, depending on the current chunk/group size.

Initialize a Reconstructor object. This class is used for managing pipelines.

Parameters:

  • process_config (ProcessConfig object) – Data structure with process configuration

  • logger (Logger, optional) – logging object

  • backend (str, optional) – Which backend to use. Available are: “cuda”, “numpy”.

  • extra_options (dict, optional) – Dictionary with advanced options. Please see ‘Other parameters’ below

  • cuda_options (dict, optional) – Dictionary with cuda options passed to nabu.cuda.processing.CudaProcessing

  • be (Advanced options can be passed in the 'extra_options' dictionary. These can) –

    • cpu_mem_fraction: 0.9,

    • gpu_mem_fraction: 0.9,

    • chunk_size: None,

    • margin: None,

    • force_grouped_mode: False

default_advanced_options = {'chunk_size': None, 'cpu_mem_fraction': 0.9, 'force_grouped_mode': False, 'gpu_mem_fraction': 0.9, 'margin': None}
reconstruct()[source]
get_relative_files(files=None)[source]
merge_hdf5_reconstructions(output_file=None, prefix=None, files=None, process_name=None, axis=0, merge_histograms=True, output_dir=None)[source]

Merge existing hdf5 files by creating a HDF5 virtual dataset.

Parameters:

output_file (str, optional) – Output file name. If not given, the file prefix in section “output” of nabu config will be taken.

merge_histograms(output_file=None, force_merge=False)[source]

Merge the partial histograms

merge_data_dumps(axis=1)[source]
write_metadata_file()[source]
finalize_files_saving()[source]
Last step to save data. This will do several things:

  • Merge data dumps (which are always HDF5 files): create a master file for all data-dump sub-volumes

  • Merge HDF5 reconstruction (if output format is HDF5)

  • Create a “metadata file” (if output format is not HDF5)

  • Merge histograms (if output format is not HDF5)