{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "skip" } }, "source": [ "# Abstract\n", "\n", "[**hdf5plugin**](https://github.com/silx-kit/hdf5plugin) is a *Python* package (1) providing a set of [**HDF5**](https://portal.hdfgroup.org/display/HDF5/) compression filters (namely: blosc, bitshuffle, lz4, FCIDECOMP, ZFP, Zstandard) and (2) enabling their use from the *Python* programming language with [**h5py**](https://docs.h5py.org/) a thin, pythonic wrapper around `libHDF5`.\n", "\n", "This presentation illustrates how to use **hdf5plugin** for reading and writing compressed datasets from *Python* and gives an overview of the different HDF5 compression filters it provides.\n", "\n", "It also illustrates how the provided compression filters can be enabled to read compressed datasets from other (non-*Python*) application.\n", "\n", "Finally, it discusses how **hdf5plugin** manages to distribute the HDF5 plugins for reuse with different `libHDF5`.\n", "\n", "License: [CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "nbsphinx": "hidden", "slideshow": { "slide_type": "skip" } }, "outputs": [], "source": [ "# Notebook requirements\n", "# A recent version of Pillow is required!\n", "%pip install numpy h5py hdf5plugin h5glance rise jupyterlab matplotlib ipympl Pillow" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "nbsphinx": "hidden", "slideshow": { "slide_type": "skip" } }, "outputs": [], "source": [ "%matplotlib inline\n", "\n", "# Creates data.h5 used for demos\n", "from matplotlib import pyplot as plt\n", "import h5py\n", "import hdf5plugin\n", "import numpy\n", "from PIL import Image\n", "import urllib.request\n", "\n", "url = \"https://www.eiroforum.org/wp-content/uploads/pg-esrf-01.jpg\"\n", "filename = urllib.request.urlretrieve(url)[0]\n", "image = numpy.array(Image.open(filename).convert(\"L\"))\n", "plt.imshow(image)\n", "\n", "h5file = h5py.File(\"data.h5\", mode=\"w\")\n", "h5file[\"copyright\"] = \"P.Ginter/ESRF\"\n", "h5file.attrs[\"url\"] = url\n", "h5file.create_dataset(\"/data\", data=image)\n", "h5file.create_dataset(\"/compressed_data_bitshuffle_lz4\", data=image, **hdf5plugin.Bitshuffle())\n", "h5file.close()\n", "\n", "# Restart kernel once the file is created!" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "nbsphinx": "hidden", "slideshow": { "slide_type": "skip" } }, "outputs": [], "source": [ "%matplotlib inline\n", "from matplotlib import pyplot as plt" ] }, { "cell_type": "markdown", "metadata": { "nbsphinx": "hidden", "slideshow": { "slide_type": "slide" } }, "source": [ "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# hdf5plugin\n", "\n", "[**hdf5plugin**](https://github.com/silx-kit/hdf5plugin) packages a set of [HDF5](https://portal.hdfgroup.org/display/HDF5/) compression filters (namely: blosc, bitshuffle, lz4, FCIDECOMP, ZFP, Zstandard) and makes them usable from the Python programming language through [h5py](https://docs.h5py.org/).\n", "\n", "[**h5py**](https://docs.h5py.org/) is a thin, pythonic wrapper around [HDF5](https://portal.hdfgroup.org/display/HDF5/).\n", "\n", " \n", "\n", " \n", "\n", "Presenter: Thomas VINCENT\n", "\n", "European HDF Users Group Summer 2021, July 7-8, 2021" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "scrolled": false, "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "data": { "text/html": [ "