h5py_utils: HDF5 I/O utilities

This module provides utility methods on top of h5py, mainly to handle parallel writing and reading.

retry(**kw)

Decorator for a method that needs to be executed until it not longer fails on HDF5 IO. Mainly used for reading an HDF5 file that is being written.

Parameters:

**kw – see silx.utils.retry

retry_contextmanager(**kw)

Decorator to make a context manager from a method that needs to be entered until it not longer fails on HDF5 IO. Mainly used for reading an HDF5 file that is being written.

Parameters:

**kw – see silx.utils.retry_contextmanager

retry_in_subprocess(**kw)

Same as retry but it also retries segmentation faults.

On Window you cannot use this decorator with the “@” syntax:

def _method(*args, **kw):
    ...

method = retry_in_subprocess()(_method)

Parameters:

**kw – see silx.utils.retry_in_subprocess

group_has_end_time(h5item)

Returns True when the HDF5 item is a Group with an “end_time” dataset. A reader can use this as an indication that the Group has been fully written (at least if the writer supports this).

Parameters:

h5item (Union[h5py.Group,h5py.Dataset]) –

Returns bool:

open_item(filename, name, retry_invalid=False, validate=None, **open_options)

Yield an HDF5 dataset or group (retry until it can be instantiated).

Parameters:

• filename (str) –

• retry_invalid (bool) – retry when item is missing or not valid

• validate (callable or None) –

• **open_options – see File.__init__

Yields Dataset, Group or None:

top_level_names(filename, include_only=<function group_has_end_time>, **open_options)

Return all valid top-level HDF5 names.

Parameters:

• filename (str) –

• include_only (callable or None) –

• **open_options – see File.__init__

Returns list(str):

safe_top_level_names(filename, include_only=<function group_has_end_time>, **open_options)

Return all valid top-level HDF5 names.

Parameters:

• filename (str) –

• include_only (callable or None) –

• **open_options – see File.__init__

Returns list(str):

class Hdf5FileLockingManager

Manage HDF5 file locking in the current process through the HDF5_USE_FILE_LOCKING environment variable.

class File(filename, mode=None, locking=None, enable_file_locking=None, swmr=None, libver=None, **kwargs)

Takes care of HDF5 file locking and SWMR mode without the need to handle those explicitely.

When file locking is managed through the HDF5_USE_FILE_LOCKING environment variable, you cannot open different files simultaneously with different modes.

close()

Close the file. All open objects become invalid