This module is a cython binding to wrap the C SpecFile library.
Documentation for the original C library SpecFile can be found on the ESRF website: The manual for the SpecFile Library
Start by importing SpecFile and instantiate it:
from silx.io.specfile import SpecFile
sf = SpecFile("test.dat")
A SpecFile instance can be accessed like a dictionary to obtain a Scan instance.
If the key is a string representing two values separated by a dot (e.g. "1.2"), they will be treated as the scan number (#S header line) and the scan order:
# get second occurrence of scan "#S 1"
myscan = sf["1.2"]
# access scan data as a numpy array
nlines, ncolumns = myscan.data.shape
If the key is an integer, it will be treated as a 0-based index:
first_scan = sf[0]
second_scan = sf[1]
It is also possible to browse through all scans using SpecFile as an iterator:
for scan in sf:
print(scan.scan_header['S'])
MCA data can be selectively loaded using an instance of MCA provided by Scan:
# Only one MCA line is loaded in memory
second_mca = first_scan.mca[1]
# Iterating trough all MCA in a scan:
for mca_data in first_scan.mca:
print(sum(mca_data))
SpecFile(filename)
Parameters: | filename (string) – Path of the SpecFile to read |
---|
This class wraps the main data and header access functions of the C SpecFile library.
Return the number of scans in the SpecFile
Return the next Scan in a SpecFile each time this method is called.
This usually happens when the python built-in function next() is called with a SpecFile instance as a parameter, or when a SpecFile instance is used as an iterator (e.g. in a for loop).
Return a Scan object.
This special method is called when a SpecFile instance is accessed as a dictionary (e.g. sf[key]).
Parameters: | key (int or str) – 0-based scan index or "n.m" key, where n is the scan number defined on the #S header line and m is the order |
---|---|
Returns: | Scan defined by its 0-based index or its "n.m" key |
Return type: | Scan |
Return number of columns in a scan from the #N header line (without #N and scan number)
Parameters: | scan_index (int) – Unique scan index between 0 and len(self)-1. |
---|---|
Returns: | Number of columns in scan from #N line |
Return type: | int |
Return #S line (without #S and scan number)
Parameters: | scan_index (int) – Unique scan index between 0 and len(self)-1. |
---|---|
Returns: | S line |
Return type: | str |
Returns data for the specified scan index.
Parameters: | scan_index (int) – Unique scan index between 0 and len(self)-1. |
---|---|
Returns: | Complete scan data as a 2D array of doubles |
Return type: | numpy.ndarray |
Returns data column for the specified scan index and column label.
Parameters: |
|
---|---|
Returns: | Data column as a 1D array of doubles |
Return type: | numpy.ndarray |
Return date from #D line
Parameters: | scan_index (int) – Unique scan index between 0 and len(self)-1. |
---|---|
Returns: | Date from #D line |
Return type: | str |
Return list of file header lines.
A file header contains all lines between a #F header line and a #S header line (start of scan). We need to specify a scan number because there can be more than one file header in a given file. A file header applies to all subsequent scans, until a new file header is defined.
Parameters: | scan_index (int) – Unique scan index between 0 and len(self)-1. |
---|---|
Returns: | List of raw file header lines |
Return type: | list of str |
Return one MCA spectrum
Parameters: |
|
---|---|
Returns: | MCA spectrum |
Return type: | 1D numpy array |
Returns scan index from scan number and order.
Parameters: |
|
---|---|
Returns: | Unique scan index |
Return type: | int |
Scan indices are increasing from 0 to len(self)-1 in the order in which they appear in the file. Scan numbers are defined by users and are not necessarily unique. The scan order for a given scan number increments each time the scan number appers in a given file.
Returns list of scan keys (eg ['1.1', '2.1',...]).
Returns: | list of scan keys |
---|---|
Return type: | list of strings |
Return all labels from #L line
Parameters: | scan_index (int) – Unique scan index between 0 and len(self)-1. |
---|---|
Returns: | All labels from #L line |
Return type: | list of strings |
Returns list (1D numpy array) of scan numbers in SpecFile.
Returns: | list of scan numbers (from `` #S`` lines) in the same order as in the original SpecFile (e.g [1, 1, 2, 3, …]). |
---|---|
Return type: | numpy array |
Return MCA calibration in the form \(a + b x + c x²\)
Raise a KeyError if there is no @CALIB line in the scan header.
Parameters: | scan_index (int) – Unique scan index between 0 and len(self)-1. |
---|---|
Returns: | MCA calibration as a list of 3 values \((a, b, c)\) |
Return type: | list of floats |
Return all motor names from #O lines
Parameters: | scan_index (int) – Unique scan index between 0 and len(self)-1. |
---|---|
Returns: | All motor names |
Return type: | list of strings |
Return motor position
Parameters: | scan_index (int) – Unique scan index between 0 and len(self)-1. |
---|---|
Returns: | Specified motor position |
Return type: | double |
Return all motor positions
Parameters: | scan_index (int) – Unique scan index between 0 and len(self)-1. |
---|---|
Returns: | All motor positions |
Return type: | list of double |
Returns scan number from scan index.
Parameters: | scan_index (int) – Unique scan index between 0 and len(self)-1. |
---|---|
Returns: | User defined scan number. |
Return type: | int |
Return number of mca spectra in a scan.
Parameters: | scan_index (int) – Unique scan index between 0 and len(self)-1. |
---|---|
Returns: | Number of mca spectra. |
Return type: | int |
Returns scan order from scan index.
Parameters: | scan_index (int) – Unique scan index between 0 and len(self)-1. |
---|---|
Returns: | Scan order (sequential number incrementing each time a non-unique occurrence of a scan number is encountered). |
Return type: | int |
Return list of scan header lines.
Parameters: | scan_index (int) – Unique scan index between 0 and len(self)-1. |
---|---|
Returns: | List of raw scan header lines |
Return type: | list of str |
Scan(specfile, scan_index)
Parameters: |
|
---|
Interface to access a SpecFile scan
A scan is a block of descriptive header lines followed by a 2D data array.
Following three ways of accessing a scan are equivalent:
sf = SpecFile("/path/to/specfile.dat")
# Explicit class instantiation
scan2 = Scan(sf, scan_index=2)
# 0-based index on a SpecFile object
scan2 = sf[2]
# Using a "n.m" key (scan number starting with 1, scan order)
scan2 = sf["3.1"]
Scan data as a numpy.ndarray with the usual attributes (e.g. data.shape).
Returns a data column
Parameters: | label (str) – Label of data column to retrieve, as defined on the #L line of the scan header. |
---|---|
Returns: | Line data as a 1D array of doubles |
Return type: | numpy.ndarray |
Returns data for a given line of this scan.
Parameters: | line_index (int) – Index of data line to retrieve (starting with 0) |
---|---|
Returns: | Line data as a 1D array of doubles |
Return type: | numpy.ndarray |
List of raw file header lines (as a list of strings).
Dictionary of file header strings, keys without the leading # (e.g. file_header_dict["F"]).
List of raw header lines (as a list of strings).
This includes the file header, the scan header and possibly a MCA header.
Unique scan index 0 - len(specfile)-1
This attribute is implemented as a read-only property as changing its value may cause nasty side-effects (such as loading data from a different scan without updating the header accordingly.
List of data column headers from #L scan header
MCA data in this scan.
Each multichannel analysis is a 1D numpy array. Metadata about MCA data is to be found in mca_header.
Return type: | MCA |
---|
Dictionary of MCA header strings, keys without the leading #@ (e.g. mca_header_dict["CALIB"]).
List of motor names from the #O file header line.
Returns the position for a given motor
Parameters: | name (str) – Name of motor, as defined on the #O line of the file header. |
---|---|
Returns: | Motor position |
Return type: | float |
List of motor positions as floats from the #P scan header line.
First value on #S line (as int)
Order can be > 1 if the same number is repeated in a specfile
Check whether a scan header line exists.
This should be used before attempting to retrieve header information using a C function that may crash with a segmentation fault if the header isn’t defined in the SpecFile.
Parameters: | record (str) – single upper case letter corresponding to the header you want to test (e.g. L for labels) |
---|---|
Returns: | True or False |
Return type: | boolean |
List of raw scan header lines (as a list of strings).
Dictionary of scan header strings, keys without the leading``#`` (e.g. scan_header_dict["S"]). Note: this does not include MCA header lines starting with #@.
MCA(scan)
Parameters: | scan (Scan) – Parent Scan instance |
---|---|
Variables: |
|
This class provides access to Multi-Channel Analysis data stored in a SpecFile scan.
To create a MCA instance, you must provide a parent Scan instance, which in turn will provide a reference to the original SpecFile instance:
sf = SpecFile("/path/to/specfile.dat")
scan2 = Scan(sf, scan_index=2)
mcas_in_scan2 = MCA(scan2)
for i in len(mcas_in_scan2):
mca_data = mcas_in_scan2[i]
... # do some something with mca_data (1D numpy array)
A more pythonic way to do the same work, without having to explicitly instantiate scan and mcas_in_scan, would be:
sf = SpecFile("specfilename.dat")
# scan2 from previous example can be referred to as sf[2]
# mcas_in_scan2 from previous example can be referred to as scan2.mca
for mca_data in sf[2].mca:
... # do some something with mca_data (1D numpy array)
Returns: | Number of mca in Scan |
---|---|
Return type: | int |
Return the next MCA data line each time this method is called.
Returns: | Single MCA |
---|---|
Return type: | 1D numpy array |
Return a single MCA data line
Parameters: | key (int) – 0-based index of MCA within Scan |
---|---|
Returns: | Single MCA |
Return type: | 1D numpy array |
h5py-like API to SpecFile
Specfile data structure exposed by this API:
/
1.1/
title = "…"
start_time = "…"
instrument/
specfile/
file_header = ["…", "…", …]
scan_header = ["…", "…", …]
positioners/
motor_name = value
…
mca_0/
data = …
calibration = …
channels = …
preset_time = …
elapsed_time = …
live_time = …
mca_1/
…
…
measurement/
colname0 = …
colname1 = …
…
mca_0/
data -> /1.1/instrument/mca_0/data
info -> /1.1/instrument/mca_0/
…
2.1/
…
file_header and scan_header are numpy arrays of fixed-length strings containing raw header lines relevant to the scan.
The title is the content of the #S scan header line without the leading #S (e.g "1 ascan ss1vo -4.55687 -0.556875 40 0.2").
The start time is in ISO8601 format ("2016-02-23T22:49:05Z")
All datasets that are not strings store values as float32.
Motor positions (e.g. /1.1/instrument/positioners/motor_name) can be 1D numpy arrays if they are measured as scan data, or else scalars as defined on #P scan header lines. A simple test is done to check if the motor name is also a data column header defined in the #L scan header line.
Scan data (e.g. /1.1/measurement/colname0) is accessed by column, the dataset name colname0 being the column label as defined in the #L scan header line.
MCA data is exposed as a 2D numpy array containing all spectra for a given analyser. The number of analysers is calculated as the number of MCA spectra per scan data line. Demultiplexing is then performed to assign the correct spectra to a given analyser.
MCA calibration is an array of 3 scalars, from the #@CALIB header line. It is identical for all MCA analysers, as there can be only one #@CALIB line per scan.
MCA channels is an array containing all channel numbers. This information is computed from the #@CHANN scan header line (if present), or computed from the shape of the first spectrum in a scan ([0, … len(first_spectrum] - 1]).
Data and groups are accessed in h5py fashion:
from silx.io.spech5 import SpecH5
# Open a SpecFile
sfh5 = SpecH5("test.dat")
# using SpecH5 as a regular group to access scans
scan1group = sfh5["1.1"]
instrument_group = scan1group["instrument"]
# altenative: full path access
measurement_group = sfh5["/1.1/measurement"]
# accessing a scan data column by name as a 1D numpy array
data_array = measurement_group["Pslit HGap"]
# accessing all mca-spectra for one MCA device
mca_0_spectra = measurement_group["mca_0/data"]
SpecH5 and SpecH5Group provide a SpecH5Group.keys() method:
>>> sfh5.keys()
['96.1', '97.1', '98.1']
>>> sfh5['96.1'].keys()
['title', 'start_time', 'instrument', 'measurement']
They can also be treated as iterators:
for scan_group in SpecH5("test.dat"):
dataset_names = [item.name in scan_group["measurement"] if
isinstance(item, SpecH5Dataset)]
print("Found data columns in scan " + scan_group.name)
print(", ".join(dataset_names))
You can test for existence of data or groups:
>>> "/1.1/measurement/Pslit HGap" in sfh5
True
>>> "positioners" in sfh5["/2.1/instrument"]
True
>>> "spam" in sfh5["1.1"]
False
Bases: silx.io.spech5.SpecH5Group
Special SpecH5Group representing the root of a SpecFile.
Parameters: | filename (str) – Path to SpecFile in filesystem |
---|
In addition to all generic SpecH5Group attributes, this class also keeps a reference to the original SpecFile object and has a filename attribute.
Its immediate children are scans, but it also gives access to any group or dataset in the entire SpecFile tree by specifying the full path.
Bases: numpy.ndarray
Emulate h5py.Dataset for a SpecFile object
Parameters: |
|
---|
This class inherits from numpy.ndarray and adds name and value attributes for HDF5 compatibility. value is a reference to the class instance (value = self).
Data is stored in float32 format, unless it is a string.
Bases: object
Emulate h5py.Group for a SpecFile object
Parameters: |
|
---|
Parameters: | key – Path to child element (e.g. "mca_0/info") or full name of group or dataset (e.g. "/2.1/instrument/positioners") |
---|---|
Returns: | True if key refers to a valid member of this group, else False |
Return a SpecH5Group or a SpecH5Dataset if key is a valid name of a group or dataset.
key can be a member of self.keys(), i.e. an immediate child of the group, or a path reaching into subgroups (e.g. "instrument/positioners")
In the special case were this group is the root group, key can start with a / character.
Parameters: | key (str) – Name of member |
---|---|
Raise: | KeyError if key is not a known member of this group. |
Attributes dictionary
Parent SpecH5 object
Full name/path of group
Recursively visit all names in this group and subgroups.
Parameters: | func (function) – Callable (function, method or callable object) |
---|
You supply a callable (function, method or callable object); it will be called exactly once for each link in this group and every group below it. Your callable must conform to the signature:
func(<member name>) => <None or return value>
Returning None continues iteration, returning anything else stops and immediately returns that value from the visit method. No particular order of iteration within groups is guaranteed.
Example:
# Get a list of all contents (groups and datasets) in a SpecFile
mylist = []
f = File('foo.dat')
f.visit(mylist.append)
Recursively visit names and objects in this group.
Parameters: | func (function) – Callable (function, method or callable object) |
---|
You supply a callable (function, method or callable object); it will be called exactly once for each link in this group and every group below it. Your callable must conform to the signature:
func(<member name>, <object>) => <None or return value>
Returning None continues iteration, returning anything else stops and immediately returns that value from the visit method. No particular order of iteration within groups is guaranteed.
Example:
# Get a list of all datasets in a specific scan
mylist = []
def func(name, obj):
if isinstance(obj, SpecH5Dataset):
mylist.append(name)
f = File('foo.dat')
f["1.1"].visititems(func)
Bases: silx.io.spech5.SpecH5Dataset
Special SpecH5Dataset representing a link to a dataset. It works exactly like a regular dataset, but SpecH5Group.visit() and SpecH5Group.visititems() methods will recognize that it is a link and will ignore it.
Bases: silx.io.spech5.SpecH5Group
Special SpecH5Group representing a link to a group.
It works exactly like a regular group but SpecH5Group.visit() and SpecH5Group.visititems() methods will recognize it as a link and will ignore it.
Check if name matches a valid dataset name pattern in a SpecH5.
Parameters: | name (str) – Full name of member |
---|
Check if name matches a valid group name pattern in a SpecH5.
Parameters: | name (str) – Full name of member |
---|
Check if name is a valid link to a dataset in a SpecH5. Return True or False
Parameters: | name (str) – Full name of member |
---|
Check if name is a valid link to a group in a SpecH5. Return True or False
Parameters: | name (str) – Full name of member |
---|
Convert SpecFile date to Iso8601.
Parameters: |
|
---|
Supported formats:
where DDD is the abbreviated weekday, MMM is the month abbreviated name, MM is the month number (zero padded), dd is the weekday number (zero padded) YYYY is the year, hh the hour (zero padded), mm the minute (zero padded) and ss the second (zero padded). All names are expected to be in english.
Examples:
>>> spec_date_to_iso8601("Thu Feb 11 09:54:35 2016")
'2016-02-11T09:54:35'
>>> spec_date_to_iso8601("Sat 2015/03/14 03:53:50")
'2015-03-14T03:53:50'
This module provides functions to convert a SpecFile into a HDF5 file
Parameters: |
|
---|
This is a convenience shortcut to call:
write_spec_to_h5(specfile, h5file, h5path='/',
h5_file_mode="w-", link_type="hard")
Write content of a SpecFile in a HDF5 file.
Parameters: |
|
---|
The structure of the spec data in an HDF5 file is described in the documentation of silx.io.spech5.
Nested python dictionary to HDF5 file conversion
Write a nested dictionary to a HDF5 file, using keys as member names.
Parameters: |
|
---|
Example:
from silx.io.dicttoh5 import dicttoh5
city_area = {
"Europe": {
"France": {
"Isère": {
"Grenoble": "18.44 km2"
},
"Nord": {
"Tourcoing": "15.19 km2"
},
},
},
}
create_ds_args = {'compression': "gzip",
'shuffle': True,
'fletcher32': True}
dicttoh5(city_area, "cities.h5", h5path="/area",
create_dataset_args=create_ds_args)
I/O utility functions
Return a simple string representation of a HDF5 tree structure.
Parameters: |
|
---|---|
Returns: | String representation of an HDF5 tree structure |
Group names and dataset representation are printed preceded by a number of tabulations corresponding to their depth in the tree structure. Datasets are represented as h5py.Dataset objects.
Example:
>>> print(h5ls("Downloads/sample.h5"))
+fields
+fieldB
<HDF5 dataset "z": shape (256, 256), type "<f4">
+fieldE
<HDF5 dataset "x": shape (256, 256), type "<f4">
<HDF5 dataset "y": shape (256, 256), type "<f4">
Saves any number of curves to various formats: Specfile, CSV, txt or npy. All curves must have the same number of points and share the same x values.
Parameters: |
|
---|
When saving to Specfile format, each curve is saved as a separate scan with two data columns (x and y).
CSV and txt formats are similar, except that the txt format allows user defined header and footer text blocks, whereas the CSV format has only a single header line with columns labels separated by field delimiters and no footer. The txt format also allows defining a record separator different from a line break.
The npy format is written with numpy.save and can be read back with numpy.load. If xlabel and ylabels are undefined, data is saved as a regular 2D numpy.ndarray (contatenation of x and y). If both xlabel and ylabels are defined, the data is saved as a numpy.recarray after being transposed and having labels assigned to columns.
Saves one curve to a SpecFile.
The curve is saved as a scan with two data columns. To save multiple curves to a single SpecFile, call this function for each curve by providing the same file handle each time.
Parameters: |
|
---|---|
Returns: | None if close_file is True, else return the file handle. |
numpy.savetxt backport of header and footer arguments from numpy=1.7.0.
Replace with numpy.savetxt when dropping support of numpy < 1.7.0
See numpy.savetxt help: http://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.savetxt.html