specfile: Reading SpecFile

This module is a cython binding to wrap the C SpecFile library, to access SpecFile data within a python program.

Documentation for the original C library SpecFile can be found on the ESRF website: The manual for the SpecFile Library

Examples

Start by importing SpecFile and instantiate it:

from silx.io.specfile import SpecFile

sf = SpecFile("test.dat")

A SpecFile instance can be accessed like a dictionary to obtain a Scan instance.

If the key is a string representing two values separated by a dot (e.g. "1.2"), they will be treated as the scan number (#S header line) and the scan order:

# get second occurrence of scan "#S 1"
myscan = sf["1.2"]

# access scan data as a numpy array
nlines, ncolumns = myscan.data.shape

If the key is an integer, it will be treated as a 0-based index:

first_scan = sf[0]
second_scan = sf[1]

It is also possible to browse through all scans using SpecFile as an iterator:

for scan in sf:
    print(scan.scan_header_dict['S'])

MCA spectra can be selectively loaded using an instance of MCA provided by Scan:

# Only one MCA spectrum is loaded in memory
second_mca = first_scan.mca[1]

# Iterating trough all MCA spectra in a scan:
for mca_data in first_scan.mca:
    print(sum(mca_data))

Classes

Exceptions

class MCA
Parameters:

scan (Scan) – Parent Scan instance
Variables:
  • calibration – MCA calibration \((a, b, c)\) (as in \(a + b x + c x²\)) from #@CALIB scan header.
  • channels – MCA channels list from #@CHANN scan header. In the absence of a #@CHANN header, this attribute is a list [0, …, N-1] where N is the length of the first spectrum. In the absence of MCA spectra, this attribute defaults to None.

This class provides access to Multi-Channel Analysis data. A MCA instance can be indexed to access 1D numpy arrays representing single MCA spectra.

To create a MCA instance, you must provide a parent Scan instance, which in turn will provide a reference to the original SpecFile instance:

sf = SpecFile("/path/to/specfile.dat")
scan2 = Scan(sf, scan_index=2)
mcas_in_scan2 = MCA(scan2)
for i in len(mcas_in_scan2):
    mca_data = mcas_in_scan2[i]
    ... # do some something with mca_data (1D numpy array)

A more pythonic way to do the same work, without having to explicitly instantiate scan and mcas_in_scan, would be:

sf = SpecFile("specfilename.dat")
# scan2 from previous example can be referred to as sf[2]
# mcas_in_scan2 from previous example can be referred to as scan2.mca
for mca_data in sf[2].mca:
    ... # do some something with mca_data (1D numpy array)
class Scan
Parameters:
  • specfile (SpecFile) – Parent SpecFile from which this scan is extracted.
  • scan_index (int) – Unique index defining the scan in the SpecFile

Interface to access a SpecFile scan

A scan is a block of descriptive header lines followed by a 2D data array.

Following three ways of accessing a scan are equivalent:

sf = SpecFile("/path/to/specfile.dat")

# Explicit class instantiation
scan2 = Scan(sf, scan_index=2)

# 0-based index on a SpecFile object
scan2 = sf[2]

# Using a "n.m" key (scan number starting with 1, scan order)
scan2 = sf["3.1"]
data

Scan data as a 2D numpy.ndarray with the usual attributes (e.g. data.shape).

The first index is the detector, the second index is the sample index.

data_column_by_name(self, label)

Returns a data column

Parameters:label (str) – Label of data column to retrieve, as defined on the #L line of the scan header.
Returns:Line data as a 1D array of doubles
Return type:numpy.ndarray
data_line(self, line_index)

Returns data for a given line of this scan.

Note

A data line returned by this method, corresponds to a data line in the original specfile (a series of data points, one per detector). In the data array, this line index corresponds to the index in the second dimension (~ column) of the array.

Parameters:line_index (int) – Index of data line to retrieve (starting with 0)
Returns:Line data as a 1D array of doubles
Return type:numpy.ndarray
file_header

List of raw file header lines (as a list of strings).

file_header_dict

Dictionary of file header strings, keys without the leading # (e.g. file_header_dict["F"]).

header

List of raw header lines (as a list of strings).

This includes the file header, the scan header and possibly a MCA header.

index

Unique scan index 0 - len(specfile)-1

This attribute is implemented as a read-only property as changing its value may cause nasty side-effects (such as loading data from a different scan without updating the header accordingly.

labels

List of data column headers from #L scan header

mca

MCA data in this scan.

Each multichannel analysis is a 1D numpy array. Metadata about MCA data is to be found in mca_header.

Return type:MCA
mca_header_dict

Dictionary of MCA header strings, keys without the leading #@ (e.g. mca_header_dict["CALIB"]).

motor_names

List of motor names from the #O file header line.

motor_position_by_name(self, name)

Returns the position for a given motor

Parameters:name (str) – Name of motor, as defined on the #O line of the file header.
Returns:Motor position
Return type:float
motor_positions

List of motor positions as floats from the #P scan header line.

number

First value on #S line (as int)

order

Order can be > 1 if the same number is repeated in a specfile

record_exists_in_hdr(self, record)

Check whether a scan header line exists.

This should be used before attempting to retrieve header information using a C function that may crash with a segmentation fault if the header isn’t defined in the SpecFile.

Parameters:record (str) – single upper case letter corresponding to the header you want to test (e.g. L for labels)
Returns:True or False
Return type:boolean
scan_header

List of raw scan header lines (as a list of strings).

scan_header_dict

Dictionary of scan header strings, keys without the leading``#`` (e.g. scan_header_dict["S"]). Note: this does not include MCA header lines starting with #@.

exception SfErrColNotFound
exception SfErrFileClose
exception SfErrFileOpen
exception SfErrFileRead
exception SfErrFileWrite
exception SfErrHeaderNotFound
exception SfErrLabelNotFound
exception SfErrLineEmpty
exception SfErrLineNotFound
exception SfErrMcaNotFound
exception SfErrMemoryAlloc
exception SfErrMotorNotFound
exception SfErrPositionNotFound
exception SfErrScanNotFound
exception SfErrUserNotFound
exception SfError

Base exception inherited by all exceptions raised when a C function from the legacy SpecFile library returns an error code.

exception SfNoMcaError

Custom exception raised when SfNoMca() returns -1

class SpecFile(filename)
Parameters:filename – Path of the SpecFile to read

This class wraps the main data and header access functions of the C SpecFile library.

close(self)

Close the file descriptor

columns(self, scan_index)

Return number of columns in a scan from the #N header line (without #N and scan number)

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:Number of columns in scan from #N line
Return type:int
command(self, scan_index)

Return #S line (without #S and scan number)

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:S line
Return type:str
data(self, scan_index)

Returns data for the specified scan index.

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:Complete scan data as a 2D array of doubles
Return type:numpy.ndarray
data_column_by_name(self, scan_index, label)

Returns data column for the specified scan index and column label.

Parameters:
  • scan_index (int) – Unique scan index between 0 and len(self)-1.
  • label (str) – Label of data column, as defined in the #L line of the scan header.
Returns:

Data column as a 1D array of doubles
Return type:

numpy.ndarray
date(self, scan_index=0)

Return date from #D line

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:Date from #D line
Return type:str
file_header(self, scan_index=0)

Return list of file header lines.

A file header contains all lines between a #F header line and a #S header line (start of scan). We need to specify a scan number because there can be more than one file header in a given file. A file header applies to all subsequent scans, until a new file header is defined.

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:List of raw file header lines
Return type:list of str
get_mca(self, scan_index, mca_index)

Return one MCA spectrum

Parameters:
  • scan_index (int) – Unique scan index between 0 and len(self)-1.
  • mca_index (int) – Index of MCA in the scan
Returns:

MCA spectrum
Return type:

1D numpy array
index(self, scan_number, scan_order=1)

Returns scan index from scan number and order.

Parameters:
  • scan_number (int) – Scan number (possibly non-unique).
  • scan_order (int default 1) – Scan order.
Returns:

Unique scan index
Return type:

int

Scan indices are increasing from 0 to len(self)-1 in the order in which they appear in the file. Scan numbers are defined by users and are not necessarily unique. The scan order for a given scan number increments each time the scan number appears in a given file.

keys(self)

Returns list of scan keys (eg ['1.1', '2.1',...]).

Returns:list of scan keys
Return type:list of strings
labels(self, scan_index)

Return all labels from #L line

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:All labels from #L line
Return type:list of strings
list(self)

Returns list (1D numpy array) of scan numbers in SpecFile.

Returns:list of scan numbers (from `` #S`` lines) in the same order as in the original SpecFile (e.g [1, 1, 2, 3, …]).
Return type:numpy array
mca_calibration(self, scan_index)

Return MCA calibration in the form \(a + b x + c x²\)

Raise a KeyError if there is no @CALIB line in the scan header.

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:MCA calibration as a list of 3 values \((a, b, c)\)
Return type:list of floats
motor_names(self, scan_index=0)

Return all motor names from #O lines

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.If not specified, defaults to 0 (meaning the function returns motors names associated with the first scan). This parameter makes a difference only if there are more than on file header in the file, in which case the file header applies to all following scans until a new file header appears.
Returns:All motor names
Return type:list of strings
motor_position_by_name(self, scan_index, name)

Return motor position

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:Specified motor position
Return type:double
motor_positions(self, scan_index)

Return all motor positions

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:All motor positions
Return type:list of double
number(self, scan_index)

Returns scan number from scan index.

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:User defined scan number.
Return type:int
number_of_mca(self, scan_index)

Return number of mca spectra in a scan.

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:Number of mca spectra.
Return type:int
order(self, scan_index)

Returns scan order from scan index.

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:Scan order (sequential number incrementing each time a non-unique occurrence of a scan number is encountered).
Return type:int
scan_header(self, scan_index)

Return list of scan header lines.

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:List of raw scan header lines
Return type:list of str
is_specfile(filename)

Test if a file is a SPEC file, by checking if one of the first two lines starts with #F (SPEC file header) or #S (scan header).

Parameters:filename (str) – File path
Returns:True if file is a SPEC file, False if it is not a SPEC file
Return type:bool
class SpecFile(filename)
Parameters:filename – Path of the SpecFile to read

This class wraps the main data and header access functions of the C SpecFile library.

__len__()

Return the number of scans in the SpecFile

__iter__()

Return the next Scan in a SpecFile each time this method is called.

This usually happens when the python built-in function next() is called with a SpecFile instance as a parameter, or when a SpecFile instance is used as an iterator (e.g. in a for loop).

__getitem__()

Return a Scan object.

This special method is called when a SpecFile instance is accessed as a dictionary (e.g. sf[key]).

Parameters:key (int or str) – 0-based scan index or "n.m" key, where n is the scan number defined on the #S header line and m is the order
Returns:Scan defined by its 0-based index or its "n.m" key
Return type:Scan
close(self)

Close the file descriptor

columns(self, scan_index)

Return number of columns in a scan from the #N header line (without #N and scan number)

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:Number of columns in scan from #N line
Return type:int
command(self, scan_index)

Return #S line (without #S and scan number)

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:S line
Return type:str
data(self, scan_index)

Returns data for the specified scan index.

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:Complete scan data as a 2D array of doubles
Return type:numpy.ndarray
data_column_by_name(self, scan_index, label)

Returns data column for the specified scan index and column label.

Parameters:
  • scan_index (int) – Unique scan index between 0 and len(self)-1.
  • label (str) – Label of data column, as defined in the #L line of the scan header.
Returns:

Data column as a 1D array of doubles
Return type:

numpy.ndarray
date(self, scan_index=0)

Return date from #D line

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:Date from #D line
Return type:str
file_header(self, scan_index=0)

Return list of file header lines.

A file header contains all lines between a #F header line and a #S header line (start of scan). We need to specify a scan number because there can be more than one file header in a given file. A file header applies to all subsequent scans, until a new file header is defined.

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:List of raw file header lines
Return type:list of str
get_mca(self, scan_index, mca_index)

Return one MCA spectrum

Parameters:
  • scan_index (int) – Unique scan index between 0 and len(self)-1.
  • mca_index (int) – Index of MCA in the scan
Returns:

MCA spectrum
Return type:

1D numpy array
index(self, scan_number, scan_order=1)

Returns scan index from scan number and order.

Parameters:
  • scan_number (int) – Scan number (possibly non-unique).
  • scan_order (int default 1) – Scan order.
Returns:

Unique scan index
Return type:

int

Scan indices are increasing from 0 to len(self)-1 in the order in which they appear in the file. Scan numbers are defined by users and are not necessarily unique. The scan order for a given scan number increments each time the scan number appears in a given file.

keys(self)

Returns list of scan keys (eg ['1.1', '2.1',...]).

Returns:list of scan keys
Return type:list of strings
labels(self, scan_index)

Return all labels from #L line

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:All labels from #L line
Return type:list of strings
list(self)

Returns list (1D numpy array) of scan numbers in SpecFile.

Returns:list of scan numbers (from `` #S`` lines) in the same order as in the original SpecFile (e.g [1, 1, 2, 3, …]).
Return type:numpy array
mca_calibration(self, scan_index)

Return MCA calibration in the form \(a + b x + c x²\)

Raise a KeyError if there is no @CALIB line in the scan header.

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:MCA calibration as a list of 3 values \((a, b, c)\)
Return type:list of floats
motor_names(self, scan_index=0)

Return all motor names from #O lines

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.If not specified, defaults to 0 (meaning the function returns motors names associated with the first scan). This parameter makes a difference only if there are more than on file header in the file, in which case the file header applies to all following scans until a new file header appears.
Returns:All motor names
Return type:list of strings
motor_position_by_name(self, scan_index, name)

Return motor position

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:Specified motor position
Return type:double
motor_positions(self, scan_index)

Return all motor positions

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:All motor positions
Return type:list of double
number(self, scan_index)

Returns scan number from scan index.

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:User defined scan number.
Return type:int
number_of_mca(self, scan_index)

Return number of mca spectra in a scan.

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:Number of mca spectra.
Return type:int
order(self, scan_index)

Returns scan order from scan index.

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:Scan order (sequential number incrementing each time a non-unique occurrence of a scan number is encountered).
Return type:int
scan_header(self, scan_index)

Return list of scan header lines.

Parameters:scan_index (int) – Unique scan index between 0 and len(self)-1.
Returns:List of raw scan header lines
Return type:list of str
class Scan
Parameters:
  • specfile (SpecFile) – Parent SpecFile from which this scan is extracted.
  • scan_index (int) – Unique index defining the scan in the SpecFile

Interface to access a SpecFile scan

A scan is a block of descriptive header lines followed by a 2D data array.

Following three ways of accessing a scan are equivalent:

sf = SpecFile("/path/to/specfile.dat")

# Explicit class instantiation
scan2 = Scan(sf, scan_index=2)

# 0-based index on a SpecFile object
scan2 = sf[2]

# Using a "n.m" key (scan number starting with 1, scan order)
scan2 = sf["3.1"]
data

Scan data as a 2D numpy.ndarray with the usual attributes (e.g. data.shape).

The first index is the detector, the second index is the sample index.

data_column_by_name(self, label)

Returns a data column

Parameters:label (str) – Label of data column to retrieve, as defined on the #L line of the scan header.
Returns:Line data as a 1D array of doubles
Return type:numpy.ndarray
data_line(self, line_index)

Returns data for a given line of this scan.

Note

A data line returned by this method, corresponds to a data line in the original specfile (a series of data points, one per detector). In the data array, this line index corresponds to the index in the second dimension (~ column) of the array.

Parameters:line_index (int) – Index of data line to retrieve (starting with 0)
Returns:Line data as a 1D array of doubles
Return type:numpy.ndarray
file_header

List of raw file header lines (as a list of strings).

file_header_dict

Dictionary of file header strings, keys without the leading # (e.g. file_header_dict["F"]).

header

List of raw header lines (as a list of strings).

This includes the file header, the scan header and possibly a MCA header.

index

Unique scan index 0 - len(specfile)-1

This attribute is implemented as a read-only property as changing its value may cause nasty side-effects (such as loading data from a different scan without updating the header accordingly.

labels

List of data column headers from #L scan header

mca

MCA data in this scan.

Each multichannel analysis is a 1D numpy array. Metadata about MCA data is to be found in mca_header.

Return type:MCA
mca_header_dict

Dictionary of MCA header strings, keys without the leading #@ (e.g. mca_header_dict["CALIB"]).

motor_names

List of motor names from the #O file header line.

motor_position_by_name(self, name)

Returns the position for a given motor

Parameters:name (str) – Name of motor, as defined on the #O line of the file header.
Returns:Motor position
Return type:float
motor_positions

List of motor positions as floats from the #P scan header line.

number

First value on #S line (as int)

order

Order can be > 1 if the same number is repeated in a specfile

record_exists_in_hdr(self, record)

Check whether a scan header line exists.

This should be used before attempting to retrieve header information using a C function that may crash with a segmentation fault if the header isn’t defined in the SpecFile.

Parameters:record (str) – single upper case letter corresponding to the header you want to test (e.g. L for labels)
Returns:True or False
Return type:boolean
scan_header

List of raw scan header lines (as a list of strings).

scan_header_dict

Dictionary of scan header strings, keys without the leading``#`` (e.g. scan_header_dict["S"]). Note: this does not include MCA header lines starting with #@.

class MCA
Parameters:

scan (Scan) – Parent Scan instance
Variables:
  • calibration – MCA calibration \((a, b, c)\) (as in \(a + b x + c x²\)) from #@CALIB scan header.
  • channels – MCA channels list from #@CHANN scan header. In the absence of a #@CHANN header, this attribute is a list [0, …, N-1] where N is the length of the first spectrum. In the absence of MCA spectra, this attribute defaults to None.

This class provides access to Multi-Channel Analysis data. A MCA instance can be indexed to access 1D numpy arrays representing single MCA spectra.

To create a MCA instance, you must provide a parent Scan instance, which in turn will provide a reference to the original SpecFile instance:

sf = SpecFile("/path/to/specfile.dat")
scan2 = Scan(sf, scan_index=2)
mcas_in_scan2 = MCA(scan2)
for i in len(mcas_in_scan2):
    mca_data = mcas_in_scan2[i]
    ... # do some something with mca_data (1D numpy array)

A more pythonic way to do the same work, without having to explicitly instantiate scan and mcas_in_scan, would be:

sf = SpecFile("specfilename.dat")
# scan2 from previous example can be referred to as sf[2]
# mcas_in_scan2 from previous example can be referred to as scan2.mca
for mca_data in sf[2].mca:
    ... # do some something with mca_data (1D numpy array)
__len__(self)
Returns:Number of mca in Scan
Return type:int
__iter__(self)

Return the next MCA data line each time this method is called.

Returns:Single MCA
Return type:1D numpy array
__getitem__(self, key)

Return a single MCA data line

Parameters:key (int) – 0-based index of MCA within Scan
Returns:Single MCA
Return type:1D numpy array
class SfError

Base exception inherited by all exceptions raised when a C function from the legacy SpecFile library returns an error code.

class SfErrMemoryAlloc

Bases: silx.io.specfile.SfError, MemoryError

class SfErrFileOpen

Bases: silx.io.specfile.SfError, OSError

class SfErrFileClose

Bases: silx.io.specfile.SfError, OSError

class SfErrFileRead

Bases: silx.io.specfile.SfError, OSError

class SfErrFileWrite

Bases: silx.io.specfile.SfError, OSError

class SfErrLineNotFound

Bases: silx.io.specfile.SfError, KeyError

class SfErrScanNotFound

Bases: silx.io.specfile.SfError, IndexError

class SfErrHeaderNotFound

Bases: silx.io.specfile.SfError, KeyError

class SfErrLabelNotFound

Bases: silx.io.specfile.SfError, KeyError

class SfErrMotorNotFound

Bases: silx.io.specfile.SfError, KeyError

class SfErrPositionNotFound

Bases: silx.io.specfile.SfError, KeyError

class SfErrLineEmpty

Bases: silx.io.specfile.SfError, OSError

class SfErrUserNotFound

Bases: silx.io.specfile.SfError, KeyError

class SfErrColNotFound

Bases: silx.io.specfile.SfError, KeyError

class SfErrMcaNotFound

Bases: silx.io.specfile.SfError, IndexError

class SfNoMcaError

Bases: silx.io.specfile.SfError

Custom exception raised when SfNoMca() returns -1