{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "skip" } }, "source": [ "Abstract\n", "\n", "[**hdf5plugin**](https://github.com/silx-kit/hdf5plugin) is a *Python* package (1) providing a set of [**HDF5**](https://portal.hdfgroup.org/display/HDF5/) compression filters (namely: blosc, bitshuffle, lz4, FCIDECOMP, ZFP, Zstandard) and (2) enabling their use from the *Python* programming language with [**h5py**](https://docs.h5py.org/) a thin, pythonic wrapper around `libHDF5`.\n", "\n", "This presentation illustrates how to use **hdf5plugin** for reading and writing compressed datasets from *Python* and gives an overview of the different HDF5 compression filters it provides.\n", "\n", "License: [CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "nbsphinx": "hidden", "slideshow": { "slide_type": "skip" } }, "outputs": [], "source": [ "# Notebook requirements\n", "# A recent version of Pillow is required!\n", "#%pip install numpy h5py hdf5plugin h5glance rise jupyterlab matplotlib ipympl Pillow" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "nbsphinx": "hidden", "slideshow": { "slide_type": "skip" } }, "outputs": [], "source": [ "%matplotlib inline\n", "\n", "# Creates data.h5 used for demos\n", "from matplotlib import pyplot as plt\n", "import h5py\n", "import hdf5plugin\n", "import numpy\n", "from PIL import Image\n", "import urllib.request\n", "\n", "url = \"https://www.iter.org/doc/www/content/com/Lists/WebText_2014/Attachments/236/platform_view_north_1.jpg\"\n", "filename = urllib.request.urlretrieve(url)[0]\n", "image = numpy.array(Image.open(filename).convert(\"L\"))\n", "plt.imshow(image)\n", "\n", "h5file = h5py.File(\"data.h5\", mode=\"w\")\n", "h5file[\"copyright\"] = \"ITER\"\n", "h5file.attrs[\"url\"] = url\n", "h5file.create_dataset(\"/data\", data=image)\n", "h5file.create_dataset(\n", " \"/compressed_data\",\n", " data=image,\n", " **hdf5plugin.Blosc('lz4', shuffle=hdf5plugin.Blosc.BITSHUFFLE)\n", ")\n", "h5file.close()" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "skip" } }, "source": [ "**Restart kernel once the file is created!**" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "slideshow": { "slide_type": "skip" } }, "outputs": [], "source": [ "import os\n", "os._exit(0) # Makes the kernel restart" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "nbsphinx": "hidden", "slideshow": { "slide_type": "skip" } }, "outputs": [], "source": [ "%matplotlib inline\n", "from matplotlib import pyplot as plt\n", "\n", "def imshow(image):\n", " plt.imshow(image)\n", " plt.colorbar()" ] }, { "cell_type": "markdown", "metadata": { "nbsphinx": "hidden", "slideshow": { "slide_type": "slide" } }, "source": [ "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# hdf5plugin\n", "\n", "[**hdf5plugin**](https://github.com/silx-kit/hdf5plugin) packages a set of [HDF5](https://portal.hdfgroup.org/display/HDF5/) compression filters (namely: blosc, bitshuffle, lz4, FCIDECOMP, ZFP, Zstandard) and makes them usable from the Python programming language through [h5py](https://docs.h5py.org/).\n", "\n", "[**h5py**](https://docs.h5py.org/) is a thin, pythonic wrapper around [HDF5](https://portal.hdfgroup.org/display/HDF5/).\n", "\n", " \n", "\n", " \n", "\n", "Presenter: Thomas VINCENT\n", "\n", "European HDF5 User Group Meeting 2022, May 31, 2022" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "data": { "text/html": [ "