nabu.io.reader

source module nabu.io.reader

Classes

• Reader — Abstract class for various file readers.

• NPReader — Reader for NPY/NPZ files. Mostly used for internal development. Please refer to the documentation of nabu.io.reader.Reader

• EDFReader — A class for reading series of EDF Files. Multi-frames EDF are not supported.

• HDF5Reader — A class for reading a HDF5 File.

• HDF5Loader — An alternative class to HDF5Reader where information is first passed at class instantiation

• ChunkReader — A reader of chunk of images.

• VolReaderBase — Base class with common code for data readers (EDFStackReader, NXTomoReader, etc)

• NXTomoReader — Read a HDF5 file in NXTomo layout.

• NXDarksFlats

• EDFStackReader

Functions

• load_images_from_dataurl_dict — Load a dictionary of dataurl into numpy arrays.

• load_images_stack_from_hdf5 — Load a 3D dataset from a HDF5 file.

• get_hdf5_dataset_shape

• get_hdf5_dataset_dtype

• get_entry_from_h5_path

• list_hdf5_entries

• check_virtual_sources_exist

• get_hdf5_file_all_virtual_sources

• import_h5_to_dict — Wrapper on top of silx.io.dictdump.dicttoh5 replacing "None" with None

source class Reader(sub_region=None)

Abstract class for various file readers.

Parameters

sub_region: tuple, optional Coordinates in the form (start_x, end_x, start_y, end_y), to read a subset of each frame. It can be used for Regions of Interest (ROI). Indices start at zero !

Methods

• get_data — Get data from a silx.io.url.DataUrl

• release — Release the file if needed.

source method Reader.get_data(data_url)

Get data from a silx.io.url.DataUrl

Raises

• ValueError

source method Reader.release()

Release the file if needed.

source class NPReader(sub_region=None, mmap=True)

Bases : Reader

Reader for NPY/NPZ files. Mostly used for internal development. Please refer to the documentation of nabu.io.reader.Reader

Methods

• get_data

• release

source method NPReader.get_data(data_url)

source method NPReader.release()

source class EDFReader(sub_region=None)

Bases : Reader

A class for reading series of EDF Files. Multi-frames EDF are not supported.

Methods

• read

• get_data

source method EDFReader.read(fname)

source method EDFReader.get_data(data_url)

source class HDF5Reader(sub_region=None)

Bases : Reader

A class for reading a HDF5 File.

Methods

• get_data

• release

source method HDF5Reader.get_data(data_url)

source method HDF5Reader.release()

source class HDF5Loader(fname, data_path, sub_region=None, data_buffer=None, pre_allocate=True, dtype='f')

An alternative class to HDF5Reader where information is first passed at class instantiation

Methods

• load_data

source method HDF5Loader.load_data(force_load=False, output=None)

source class ChunkReader(files, sub_region=None, detector_corrector=None, pre_allocate=True, data_buffer=None, convert_float=False, shape=None, dtype=None, binning=None, dataset_subsampling=None, num_threads=None)

A reader of chunk of images.

Initialize a "ChunkReader". A chunk is a stack of images.

Parameters

• files : dict — Dictionary where the key is the file/data index, and the value is a silx.io.url.DataUrl pointing to the data. The dict must contain only the files which shall be used ! Note: the shape and data type is infered from the first data file.

• sub_region : tuple, optional — If provided, this must be a tuple in the form (start_x, end_x, start_y, end_y). Each image will be cropped to this region. This is used to specify a chunk of files. Each of the parameters can be None, in this case the default start and end are taken in each dimension.

• pre_allocate : bool — Whether to pre-allocate data before reading.

• data_buffer : array-like, optional — If pre_allocate is set to False, this parameter has to be provided. It is an array-like object which will hold the data.

• convert_float : bool — Whether to convert data to float32, regardless of the input data type.

• shape : tuple, optional — Shape of each image. If not provided, it is inferred from the first image in the collection.

• dtype : numpy.<span class="mkapi-tooltip" title="numpy._core.dtype">dtype</span>, optional — Data type of each image. If not provided, it is inferred from the first image in the collection.

• binning : int or tuple of int, optional — Whether to bin the data. If multi-dimensional binning is done, the parameter must be in the form (binning_x, binning_y). Each image will be binned by these factors.

• dataset_subsampling : int or tuple, optional — Subsampling factor when reading the images. If an integer n is provided, then one image out of n will be read. If a tuple of integers (step, begin) is given, the data is read as data[begin::step]

• num_threads : int, optional — Number of threads to use for binning the data. Default is to use all available threads. This parameter has no effect when binning is disabled.

Notes

The files are provided as a collection of silx.io.DataURL. The file type is inferred from the extension.

Binning is different from subsampling. Using binning will not speed up the data retrieval (quite the opposite), since the whole (subregion of) data is read and then binning is performed.

Methods

• get_data — Get the data associated to a file url.

• load_files — Load the files whose links was provided at class instantiation.

source method ChunkReader.get_data(file_url)

Get the data associated to a file url.

source method ChunkReader.load_files(overwrite: bool = False)

Load the files whose links was provided at class instantiation.

Parameters

• overwrite : bool, optional — Whether to force reloading the files if already loaded.

Raises

• ValueError

source property ChunkReader.data

source class VolReaderBase(*args, **kwargs)

Base class with common code for data readers (EDFStackReader, NXTomoReader, etc)

Methods

• get_frames_indices

source method VolReaderBase.get_frames_indices()

source class NXTomoReader(filename, data_path='{entry}/instrument/detector/data', sub_region=None, image_key=0, output_dtype=np.float32, processing_func=None, processing_func_args=None, processing_func_kwargs=None)

Bases : VolReaderBase

Read a HDF5 file in NXTomo layout.

Parameters

• filename : str — Path to the file to read.

• data_path : str — Path within the HDF5 file, eg. "entry/instrument/detector/data". Default is {entry}/data/data where {entry} is a magic keyword for the first entry.

• sub_region : slice or tuple, optional — Region to select within the data, once the "image key" selection has been done. If None, all the data (corresponding to image_key) is selected. If slice(start, stop) is provided, the selection is done along dimension 0. Otherwise, it must be a 3-tuple of slices in the form (slice(start_angle, end_angle, step_angle), slice(start_z, end_z, step_z), slice(start_x, end_x, step_x)) Each of the parameters can be None, in this case the default start and end are taken in each dimension.

• output_dtype : numpy.dtype, optional — Output data type if the data memory is allocated by this class. Default is float32.

• image_key : int, or None, optional — Image type to read (see NXTomo documentation). 0 for projections, 1 for flat-field, 2 for dark field. If set to None, all the data will be read.

• processing_func : callable, optional — Function to be called on each loaded stack of images. If provided, this function first argument must be the source buffer (3D array: stack of raw images), and the second argument must be the destination buffer (3D array, stack of output images). It can be None.

Other Parameters

The other parameters are passed to "processing_func" if this parameter is not None.

Methods

• load_data — Read data.

source method NXTomoReader.load_data(output=None, convert_after_reading=True)

Read data.

Parameters

• output : array-like, optional — Destination 3D array that will hold the data. If provided, use this memory buffer instead of allocating the memory. Its shape must be compatible with the selection of 'sub_region' and 'image_key'.

• conver_after_reading : bool, optional — Whether to do the dtype conversion (if any, eg. uint16 to float32) after data reading. With using h5py's read_direct(), reading from uint16 to float32 is extremely slow, so data type conversion should be done after reading. The drawback is that it requires more memory.

source class NXDarksFlats(filename, **nxtomoreader_kwargs)

Methods

• get_raw_darks

• get_raw_flats

• get_reduced_darks

• get_reduced_flats

• get_raw_current

• get_reduced_current

source method NXDarksFlats.get_raw_darks(force_reload=False, as_multiple_array=True)

source method NXDarksFlats.get_raw_flats(force_reload=False, as_multiple_array=True)

source method NXDarksFlats.get_reduced_darks(method='mean', force_reload=False, as_dict=False)

source method NXDarksFlats.get_reduced_flats(method='median', force_reload=False, as_dict=False)

source method NXDarksFlats.get_raw_current(h5_path='{entry}/control/data')

source method NXDarksFlats.get_reduced_current(h5_path='{entry}/control/data', method='median')

source class EDFStackReader(filenames, sub_region=None, output_dtype=np.float32, n_reading_threads=1, processing_func=None, processing_func_args=None, processing_func_kwargs=None)

Bases : VolReaderBase

Methods

• load_data

source method EDFStackReader.load_data(output=None)

source load_images_from_dataurl_dict(data_url_dict, sub_region=None, dtype='f', binning=None)

Load a dictionary of dataurl into numpy arrays.

Parameters

• data_url_dict : dict — A dictionary where the keys are integers (the index of each image in the dataset), and the values are numpy.ndarray (data_url_dict).

• sub_region : tuple, optional — Tuple in the form (y_subregion, x_subregion) where xy_subregion is a tuple in the form slice(start, stop, step)

Returns

• res : dict — A dictionary where the keys are the same as data_url_dict, and the values are numpy arrays.

Notes

This function is used to load flats/darks. Usually, these are reduced flats/darks, meaning that 'data_url_dict' is a collection of a handful of files (less than 10). To load more frames, it would be best to use NXTomoReader / EDFStackReader.

source load_images_stack_from_hdf5(fname, h5_data_path, sub_region=None)

Load a 3D dataset from a HDF5 file.

Parameters

• fname : str — File path

• h5_data_path : str — Data path within the HDF5 file

• sub_region : tuple, optional — Tuple indicating which sub-volume to load, in the form (xmin, xmax, ymin, ymax, zmin, zmax) where the 3D dataset has the python shape (N_Z, N_Y, N_X). This means that the data will be loaded as data[zmin:zmax, ymin:ymax, xmin:xmax].

source get_hdf5_dataset_shape(fname, h5_data_path, sub_region=None)

source get_hdf5_dataset_dtype(fname, h5_data_path)

source get_entry_from_h5_path(h5_path)

source list_hdf5_entries(fname)

source check_virtual_sources_exist(fname, data_path)

source get_hdf5_file_all_virtual_sources(file_path_or_obj, return_only_filenames=False)

source import_h5_to_dict(h5file, h5path, asarray=False)

Wrapper on top of silx.io.dictdump.dicttoh5 replacing "None" with None

Parameters

• h5file : str — File name

• h5path : str — Path in the HDF5 file

• asarray : bool, optional — Whether to convert each numeric value to an 0D array. Default is False.