Open DM3 Images, Spectra, Spectrum-Images and Image-Stacks with pyNSID¶

part of

MSE672: Introduction to Transmission Electron Microscopy

Spring 2026
by Gerd Duscher

Microscopy Facilities
Institute of Advanced Materials & Manufacturing
Materials Science & Engineering
The University of Tennessee, Knoxville

Background and methods to analysis and quantification of data acquired with transmission electron microscopes.

Reading a dm file and translating the data in a python dictionary of sidpy datasets.

The data can be stored in pyNSID style hf5py file to be compatible with all packages in the pycroscopy exosystem.

Because, many other packages and programs for TEM data manipulation are based on the hdf5 file-formats it is relatively easy to convert back and forward between them.

Import packages for figures and¶

Check Installed Packages¶

import sys
import importlib.metadata
def test_package(package_name):
    """Test if package exists and returns version or -1"""
    try:
        version = importlib.metadata.version(package_name)
    except importlib.metadata.PackageNotFoundError:
        version = '-1'
    return version

# pyTEMlib setup ------------------
if test_package('pyTEMlib') < '0.2026.1.0':
    print('installing pyTEMlib')
    !{sys.executable} -m pip install --upgrade pyTEMlib
# ------------------------------
print('done')

installing pyTEMlib
Requirement already satisfied: pyTEMlib in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (0.2025.12.1)
Requirement already satisfied: scipy in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyTEMlib) (1.16.3)
Requirement already satisfied: numpy in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyTEMlib) (2.3.5)
Requirement already satisfied: pillow in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyTEMlib) (12.0.0)
Requirement already satisfied: ase in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyTEMlib) (3.27.0)
Requirement already satisfied: tqdm in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyTEMlib) (4.67.1)
Requirement already satisfied: plotly in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyTEMlib) (6.5.0)
Requirement already satisfied: pandas in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyTEMlib) (2.3.3)
Requirement already satisfied: requests in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyTEMlib) (2.32.5)
Requirement already satisfied: lxml in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyTEMlib) (5.3.0)
Requirement already satisfied: ipympl in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyTEMlib) (0.9.8)
Requirement already satisfied: spglib in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyTEMlib) (2.6.0)
Requirement already satisfied: simpleitk in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyTEMlib) (2.5.2)
Requirement already satisfied: scikit-image in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyTEMlib) (0.25.2)
Requirement already satisfied: scikit-learn in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyTEMlib) (1.7.2)
Requirement already satisfied: pyNSID>=0.0.7 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyTEMlib) (0.0.7.2)
Requirement already satisfied: sidpy>=0.12.7 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyTEMlib) (0.12.7)
Requirement already satisfied: SciFiReaders>=0.12.4 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyTEMlib) (0.12.4)
Requirement already satisfied: xraylib in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyTEMlib) (4.2.0)
Requirement already satisfied: toolz in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyNSID>=0.0.7->pyTEMlib) (1.1.0)
Requirement already satisfied: cytoolz in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyNSID>=0.0.7->pyTEMlib) (1.0.1)
Requirement already satisfied: dask>=0.10 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyNSID>=0.0.7->pyTEMlib) (2025.11.0)
Requirement already satisfied: h5py>=2.6.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyNSID>=0.0.7->pyTEMlib) (3.15.1)
Requirement already satisfied: six in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pyNSID>=0.0.7->pyTEMlib) (1.17.0)
Requirement already satisfied: click>=8.1 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from dask>=0.10->pyNSID>=0.0.7->pyTEMlib) (8.2.1)
Requirement already satisfied: cloudpickle>=3.0.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from dask>=0.10->pyNSID>=0.0.7->pyTEMlib) (3.1.2)
Requirement already satisfied: fsspec>=2021.09.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from dask>=0.10->pyNSID>=0.0.7->pyTEMlib) (2025.10.0)
Requirement already satisfied: packaging>=20.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from dask>=0.10->pyNSID>=0.0.7->pyTEMlib) (25.0)
Requirement already satisfied: partd>=1.4.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from dask>=0.10->pyNSID>=0.0.7->pyTEMlib) (1.4.2)
Requirement already satisfied: pyyaml>=5.3.1 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from dask>=0.10->pyNSID>=0.0.7->pyTEMlib) (6.0.3)
Requirement already satisfied: colorama in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from click>=8.1->dask>=0.10->pyNSID>=0.0.7->pyTEMlib) (0.4.6)
Requirement already satisfied: locket in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from partd>=1.4.0->dask>=0.10->pyNSID>=0.0.7->pyTEMlib) (1.0.0)
Requirement already satisfied: setuptools in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from SciFiReaders>=0.12.4->pyTEMlib) (80.9.0)
Requirement already satisfied: numba in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from SciFiReaders>=0.12.4->pyTEMlib) (0.62.1)
Requirement already satisfied: ipython>=7.1.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from SciFiReaders>=0.12.4->pyTEMlib) (9.7.0)
Requirement already satisfied: pyUSID in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from SciFiReaders>=0.12.4->pyTEMlib) (0.0.12)
Requirement already satisfied: gdown in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from SciFiReaders>=0.12.4->pyTEMlib) (5.2.0)
Requirement already satisfied: mrcfile in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from SciFiReaders>=0.12.4->pyTEMlib) (1.5.4)
Requirement already satisfied: pycroscopy-gwyfile in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from SciFiReaders>=0.12.4->pyTEMlib) (1.0.2)
Requirement already satisfied: decorator>=4.3.2 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from ipython>=7.1.0->SciFiReaders>=0.12.4->pyTEMlib) (5.2.1)
Requirement already satisfied: ipython-pygments-lexers>=1.0.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from ipython>=7.1.0->SciFiReaders>=0.12.4->pyTEMlib) (1.1.1)
Requirement already satisfied: jedi>=0.18.1 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from ipython>=7.1.0->SciFiReaders>=0.12.4->pyTEMlib) (0.19.2)
Requirement already satisfied: matplotlib-inline>=0.1.5 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from ipython>=7.1.0->SciFiReaders>=0.12.4->pyTEMlib) (0.2.1)
Requirement already satisfied: prompt_toolkit<3.1.0,>=3.0.41 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from ipython>=7.1.0->SciFiReaders>=0.12.4->pyTEMlib) (3.0.52)
Requirement already satisfied: pygments>=2.11.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from ipython>=7.1.0->SciFiReaders>=0.12.4->pyTEMlib) (2.19.2)
Requirement already satisfied: stack_data>=0.6.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from ipython>=7.1.0->SciFiReaders>=0.12.4->pyTEMlib) (0.6.3)
Requirement already satisfied: traitlets>=5.13.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from ipython>=7.1.0->SciFiReaders>=0.12.4->pyTEMlib) (5.14.3)
Requirement already satisfied: wcwidth in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from prompt_toolkit<3.1.0,>=3.0.41->ipython>=7.1.0->SciFiReaders>=0.12.4->pyTEMlib) (0.2.13)
Requirement already satisfied: parso<0.9.0,>=0.8.4 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from jedi>=0.18.1->ipython>=7.1.0->SciFiReaders>=0.12.4->pyTEMlib) (0.8.5)
Requirement already satisfied: matplotlib>=2.0.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from sidpy>=0.12.7->pyTEMlib) (3.10.7)
Requirement already satisfied: distributed>=2.0.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from sidpy>=0.12.7->pyTEMlib) (2025.11.0)
Requirement already satisfied: psutil in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from sidpy>=0.12.7->pyTEMlib) (7.0.0)
Requirement already satisfied: joblib>=0.11.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from sidpy>=0.12.7->pyTEMlib) (1.5.2)
Requirement already satisfied: ipywidgets in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from sidpy>=0.12.7->pyTEMlib) (8.1.7)
Requirement already satisfied: ipykernel in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from sidpy>=0.12.7->pyTEMlib) (6.31.0)
Requirement already satisfied: dill in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from sidpy>=0.12.7->pyTEMlib) (0.4.0)
Requirement already satisfied: jinja2>=2.10.3 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from distributed>=2.0.0->sidpy>=0.12.7->pyTEMlib) (3.1.6)
Requirement already satisfied: msgpack>=1.0.2 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from distributed>=2.0.0->sidpy>=0.12.7->pyTEMlib) (1.1.1)
Requirement already satisfied: sortedcontainers>=2.0.5 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from distributed>=2.0.0->sidpy>=0.12.7->pyTEMlib) (2.4.0)
Requirement already satisfied: tblib>=1.6.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from distributed>=2.0.0->sidpy>=0.12.7->pyTEMlib) (3.1.0)
Requirement already satisfied: tornado>=6.2.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from distributed>=2.0.0->sidpy>=0.12.7->pyTEMlib) (6.5.1)
Requirement already satisfied: urllib3>=1.26.5 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from distributed>=2.0.0->sidpy>=0.12.7->pyTEMlib) (2.6.1)
Requirement already satisfied: zict>=3.0.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from distributed>=2.0.0->sidpy>=0.12.7->pyTEMlib) (3.0.0)
Requirement already satisfied: MarkupSafe>=2.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from jinja2>=2.10.3->distributed>=2.0.0->sidpy>=0.12.7->pyTEMlib) (3.0.2)
Requirement already satisfied: contourpy>=1.0.1 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from matplotlib>=2.0.0->sidpy>=0.12.7->pyTEMlib) (1.3.3)
Requirement already satisfied: cycler>=0.10 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from matplotlib>=2.0.0->sidpy>=0.12.7->pyTEMlib) (0.11.0)
Requirement already satisfied: fonttools>=4.22.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from matplotlib>=2.0.0->sidpy>=0.12.7->pyTEMlib) (4.61.0)
Requirement already satisfied: kiwisolver>=1.3.1 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from matplotlib>=2.0.0->sidpy>=0.12.7->pyTEMlib) (1.4.9)
Requirement already satisfied: pyparsing>=3 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from matplotlib>=2.0.0->sidpy>=0.12.7->pyTEMlib) (3.2.5)
Requirement already satisfied: python-dateutil>=2.7 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from matplotlib>=2.0.0->sidpy>=0.12.7->pyTEMlib) (2.9.0.post0)
Requirement already satisfied: executing>=1.2.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from stack_data>=0.6.0->ipython>=7.1.0->SciFiReaders>=0.12.4->pyTEMlib) (2.2.1)
Requirement already satisfied: asttokens>=2.1.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from stack_data>=0.6.0->ipython>=7.1.0->SciFiReaders>=0.12.4->pyTEMlib) (3.0.0)
Requirement already satisfied: pure_eval in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from stack_data>=0.6.0->ipython>=7.1.0->SciFiReaders>=0.12.4->pyTEMlib) (0.2.3)
Requirement already satisfied: beautifulsoup4 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from gdown->SciFiReaders>=0.12.4->pyTEMlib) (4.14.2)
Requirement already satisfied: filelock in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from gdown->SciFiReaders>=0.12.4->pyTEMlib) (3.20.0)
Requirement already satisfied: soupsieve>1.2 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from beautifulsoup4->gdown->SciFiReaders>=0.12.4->pyTEMlib) (2.5)
Requirement already satisfied: typing-extensions>=4.0.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from beautifulsoup4->gdown->SciFiReaders>=0.12.4->pyTEMlib) (4.15.0)
Requirement already satisfied: comm>=0.1.1 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from ipykernel->sidpy>=0.12.7->pyTEMlib) (0.2.3)
Requirement already satisfied: debugpy>=1.6.5 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from ipykernel->sidpy>=0.12.7->pyTEMlib) (1.8.16)
Requirement already satisfied: jupyter-client>=8.0.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from ipykernel->sidpy>=0.12.7->pyTEMlib) (8.6.3)
Requirement already satisfied: jupyter-core!=5.0.*,>=4.12 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from ipykernel->sidpy>=0.12.7->pyTEMlib) (5.9.1)
Requirement already satisfied: nest-asyncio>=1.4 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from ipykernel->sidpy>=0.12.7->pyTEMlib) (1.6.0)
Requirement already satisfied: pyzmq>=25 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from ipykernel->sidpy>=0.12.7->pyTEMlib) (27.1.0)
Requirement already satisfied: platformdirs>=2.5 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from jupyter-core!=5.0.*,>=4.12->ipykernel->sidpy>=0.12.7->pyTEMlib) (4.5.0)
Requirement already satisfied: widgetsnbextension~=4.0.14 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from ipywidgets->sidpy>=0.12.7->pyTEMlib) (4.0.14)
Requirement already satisfied: jupyterlab_widgets~=3.0.15 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from ipywidgets->sidpy>=0.12.7->pyTEMlib) (3.0.15)
Requirement already satisfied: llvmlite<0.46,>=0.45.0dev0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from numba->SciFiReaders>=0.12.4->pyTEMlib) (0.45.1)
Requirement already satisfied: pytz>=2020.1 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pandas->pyTEMlib) (2025.2)
Requirement already satisfied: tzdata>=2022.7 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from pandas->pyTEMlib) (2025.2)
Requirement already satisfied: narwhals>=1.15.1 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from plotly->pyTEMlib) (2.7.0)
Requirement already satisfied: charset_normalizer<4,>=2 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from requests->pyTEMlib) (3.4.4)
Requirement already satisfied: idna<4,>=2.5 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from requests->pyTEMlib) (3.11)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from requests->pyTEMlib) (2025.11.12)
Requirement already satisfied: PySocks!=1.5.7,>=1.5.6 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from requests[socks]->gdown->SciFiReaders>=0.12.4->pyTEMlib) (1.7.1)
Requirement already satisfied: networkx>=3.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from scikit-image->pyTEMlib) (3.6.1)
Requirement already satisfied: imageio!=2.35.0,>=2.33 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from scikit-image->pyTEMlib) (2.37.2)
Requirement already satisfied: tifffile>=2022.8.12 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from scikit-image->pyTEMlib) (2025.10.4)
Requirement already satisfied: lazy-loader>=0.4 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from scikit-image->pyTEMlib) (0.4)
Requirement already satisfied: threadpoolctl>=3.1.0 in c:\users\gduscher\appdata\local\anaconda3\lib\site-packages (from scikit-learn->pyTEMlib) (3.5.0)
done

Load the plotting and figure packages¶

%matplotlib widget
import matplotlib.pylab as plt
import numpy as np
import sys

import pyTEMlib

import sidpy
import pyNSID
import h5py

if 'google.colab' in sys.modules:
    from google.colab import output
    output.enable_custom_widget_manager()
    from google.colab import drive
    drive.mount("/content/drive")

# For archiving reasons it is a good idea to print the version numbers out at this point
print('pyTEM version: ',pyTEMlib.__version__)
__notebook__='CH1_04-Reading_File'
__notebook_version__='2024_01_09'

pyTEM version:  0.2025.12.1

Open a file¶

This function opens a hfd5 file in the pyNSID style which enables you to keep track of your data analysis.

Please see the Installation notebook for installation.

We want to consolidate files into one dataset that belongs together. For example a spectrum image dataset consists of:

Survey image,
EELS spectra
Z-contrast image acquired simultaneously with the spectra.

So load the top dataset first in the above example the survey image.

Please note that the plotting routine of matplotlib was introduced in Matplotlib and Numpy for Micrographs notebook.

Use the file p1-3hr.dm3 from TEM_data directory for a practice run

# ------ Input ------- #
load_example = True
# -------------------- #

# Open file widget and select file which will be opened in code cell below
if not load_example:
    drive_directory = pyTEMlib.file_tools.get_last_path()
    file_widget = pyTEMlib.file_tools.FileWidget(drive_directory)

if load_example:
    file_name = '../example_data/p1-3-hr3.dm3'
    datasets = pyTEMlib.file_tools .open_file(file_name)
    main_dataset = datasets[list(datasets.keys())[0]]
else:
    main_dataset = file_widget.selected_dataset
    datasets = file_widget.datasets

view = main_dataset.plot()

Use the selection tool on left of the image and select a part of the image and then a part of the colorbar.

Data Structure¶

The data themselves reside in a python dictionary and the selected sidpy dataset which we name main_dataset.

The current_dataset has additional information stored as attributes which can be accessed through their name.

print(main_dataset)
main_dataset

sidpy.Dataset of type IMAGE with:
 dask.array<array, shape=(2048, 2048), dtype=int32, chunksize=(2048, 2048), chunktype=numpy.ndarray>
 data contains: intensity (counts)
 and Dimensions: 
x:  distance (nm) of size (2048,)
y:  distance (nm) of size (2048,)
 with metadata: ['experiment', 'filename']

print(f'size of current dataset is {main_dataset.shape}')

size of current dataset is (2048, 2048)

The current_dataset has additional information stored as attributes which can be accessed through their name.

There are two dictionaries within that attributes:

metadata
original_metadata

which contain additional information about the data

print('title: ', main_dataset.title)
print('data type: ', main_dataset.data_type)

for key in datasets:
    print(key)
    print(datasets[key].original_metadata.keys())
    
main_dataset.metadata

title:  p1-3-hr3
data type:  DataType.IMAGE
Channel_000
dict_keys(['ImageData', 'ImageTags', 'Name', 'UniqueID', 'DM', 'original_filename', 'ApplicationBounds', 'DocumentObjectList', 'DocumentTags', 'HasWindowPosition', 'Image Behavior', 'ImageSourceList', 'InImageMode', 'MinVersionList', 'NextDocumentObjectID', 'Page Behavior', 'PageSetup', 'SentinelList', 'Thumbnails', 'WindowPosition'])

{'experiment': {'single_exposure_time': 0.0,
  'exposure_time': 1.0,
  'convergence_angle': 0.0,
  'collection_angle': 0.0,
  'number_of_frames': 1,
  'microscope': 'Libra 200 MC',
  'acceleration_voltage': 199990.28125},
 'filename': '../example_data/p1-3-hr3.dm3'}

Data Structure¶

The datasets variable is a dictionary (like a directory in a file system) which containes contains datasets.

Below I show how to access one of those datasets with a pull down menu.

chooser = pyTEMlib.file_tools.ChooseDataset(datasets)

current_dataset = chooser.dataset
view = current_dataset.plot()

An important attribute in current_dataset is the original_metadata group, where all the original metadata of your file reside in the attributes. This is usually a long list for dm3 files.

current_dataset.original_metadata.keys()

dict_keys(['ImageData', 'ImageTags', 'Name', 'UniqueID', 'DM', 'original_filename', 'ApplicationBounds', 'DocumentObjectList', 'DocumentTags', 'HasWindowPosition', 'Image Behavior', 'ImageSourceList', 'InImageMode', 'MinVersionList', 'NextDocumentObjectID', 'Page Behavior', 'PageSetup', 'SentinelList', 'Thumbnails', 'WindowPosition'])

The original_metadata attribute has all information stored from the orginal file.

No information will get lost

for key,value in current_dataset.original_metadata.items():
    print(key, value)
print(current_dataset.h5_dataset)

ImageData {'Calibrations': {'Brightness': {'Origin': 0.0, 'Scale': 1.0, 'Units': ''}, 'Dimension': {'0': {'Origin': 0.0, 'Scale': 0.03894666209816933, 'Units': 'nm'}, '1': {'Origin': 0.0, 'Scale': 0.03894666209816933, 'Units': 'nm'}}, 'DisplayCalibratedUnits': 1}, 'Data': 'read', 'DataType': 7, 'Dimensions': {'0': 2048, '1': 2048}, 'PixelDepth': 4}
ImageTags {'Acquisition': {'Device': {'Active Size (pixels)': [2048, 2048], 'Camera Number': 0, 'CCD': {'Pixel Size (um)': [14.0, 14.0]}, 'Configuration': {'Transpose': {'Diagonal Flip': 0, 'Horizontal Flip': 1, 'Vertical Flip': 0}}, 'Name': 'US1000XP 1', 'Source': 'US1000XP 1'}, 'Frame': {'Area': {'Transform': {'Class Name': 'cm_acquisitiontransform_list', 'Transform List': {'0': {'Binning': [1, 1], 'Class Name': 'cm_acquisitiontransform', 'Sub Area Adjust': [0, 0, 0, 0], 'Transpose': {'Diagonal Flip': 0, 'Horizontal Flip': 1, 'Vertical Flip': 0}}}}}, 'CCD': {'Pixel Size (um)': [14.0, 14.0]}, 'Intensity': {'Range': {'Bias (counts)': 250.0, 'Dark Current (counts/s)': 0.0, 'Dark Level (counts)': 250.0, 'Maximum Value (counts)': 65535.0, 'Minimum Value (counts)': 0.0, 'Saturation Level (counts)': 65785.0}, 'Transform': {'Class Name': 'cm_valuetransform_list', 'Transform List': {'0': {'Class Name': 'cm_valuetransform_affine', 'Offset': 250.0, 'Scale': 1.0}, '1': {'ADC Max': 65535.0, 'ADC Min': 0.0, 'Class Name': 'cm_valuetransform_adc'}}}}, 'Reference Images': {'Dark': {'Mean (counts)': 251.31182837486267, 'Standard Deviation (counts)': 3.28830754005297}}, 'Sequence': {'Acquisition Start Time (epoch)': 1528991109318.0, 'Exposure Start (ns)': 130828979.37529667, 'Exposure Time (ns)': 1000000000.0, 'Frame Index': 0, 'Frame Start (ns)': 130828979.37529667, 'Frame Time (ns)': 3482273464.1519103, 'Readout Time (ns)': 3482273464.1519103}}, 'Parameters': {'Acquisition Write Flags': 4294967295, 'Base Detector': {'Class Name': 'cm_namedcameradetectorparameterset', 'Name': 'default'}, 'Detector': {'continuous': 0, 'exposure (s)': 1.0, 'hbin': 1, 'height': 2048, 'left': 0, 'top': 0, 'vbin': 1, 'width': 2048}, 'Environment': {'Mode Name': 'Imaging'}, 'High Level': {'Acquisition Buffer Size': 0, 'Antiblooming': 0, 'Binning': [1, 1], 'CCD Read Area': [0, 0, 2048, 2048], 'CCD Read Ports': 1, 'Choose Number Of Frame Shutters Automatically': 1, 'Class Name': 'cm_camera_highlevelparameters', 'Continuous Readout': 0, 'Corrections': 817, 'Corrections Mask': 817, 'Exposure (s)': 1.0, 'Number Of Frame Shutters': 1, 'Processing': 'Gain Normalized', 'Quality Level': 1, 'Read Frame Style': 0, 'Read Mode': 0, 'Secondary Shutter Post Exposure Compensation (s)': 0.0, 'Secondary Shutter Pre Exposure Compensation (s)': 0.0, 'Shutter': {'Primary Shutter States': 0, 'Primary Shutter States Mask': 0, 'Secondary Shutter States': 0, 'Secondary Shutter States Mask': 0, 'Shutter Exposure': 0, 'Shutter Index': 0}, 'Shutter Post Exposure Compensation (s)': 0.0, 'Shutter Pre Exposure Compensation (s)': 0.0, 'Transform': {'Diagonal Flip': 0, 'Horizontal Flip': 0, 'Vertical Flip': 0}}, 'Objects': {'0': {'Class Name': 'cm_autoexpose_acquireimage', 'Do Auto Expose': 0, 'Maximum Exposure (s)': 5.0, 'Minimum Exposure (s)': 0.1, 'Target Intensity (%)': 50.0, 'Test Exposure (s)': 0.1}, '1': {'Class Name': 'cm_imgproc_finalcombine', 'Frame Combine Style': 'Copy', 'Parameter 1': 1.0}, '2': {'Class Name': 'cm_stdviewerimagedisplayer', 'Do Auto Zoom': 0, 'Screen Relative Position': [1.0, 0.0], 'Version': 33947648, 'View Name': 'Frame', 'Viewer Class': 'acquire', 'Window Relative Position': [1.0, 0.0], 'Zoom': 1.0}, '3': {'Class Name': 'cm_imgproc_histogram'}}, 'Parameter Set Name': 'Record', 'Parameter Set Tag Path': 'Imaging:Acquire:Record', 'Version': 33947648}}, 'DataBar': {'Acquisition Date': '6/14/2018', 'Acquisition Time': '11:45:13 AM', 'Acquisition Time (OS)': 1.3173464713128774e+17, 'Binning': 1, 'Custom elements': {}, 'Device Name': 'US1000XP 1', 'Exposure Number': 13083663, 'Exposure Time (s)': 1.0}, 'Microscope Info': {'Actual Magnification': 359465.9784890286, 'Cs(mm)': 2.2, 'Emission Current (A)': 230.0, 'Formatted Indicated Mag': '315kx', 'Formatted Voltage': '200.0kV', 'HT Extrapolated': 0, 'Illumination Mode': 'TEM', 'Imaging Mode': 'Image Mag', 'Indicated Magnification': 315000.0, 'Items': {'0': {'Data Type': 20, 'Label': 'Specimen', 'Tag path': 'Microscope Info:Specimen', 'Value': 'Fe-9Cr(0.3Y)-3E10(17)-475C'}, '1': {'Data Type': 20, 'Label': 'Operator', 'Tag path': 'Microscope Info:Operator', 'Value': 'Tengfei Yang'}, '2': {'Data Type': 20, 'Label': 'Microscope', 'Tag path': 'Microscope Info:Microscope', 'Value': 'Libra 200 MC'}}, 'Magnification Interpolated': 0, 'Microscope': 'Libra 200 MC', 'Name': 'Libra COM', 'Operation Mode': 'IMAGING', 'Operator': 'Tengfei Yang', 'Probe Current (nA)': 0.0, 'Probe Size (nm)': 0.0, 'Specimen': 'Fe-9Cr(0.3Y)-3E10(17)-475C', 'STEM Camera Length': 94.49999779462814, 'Voltage': 199990.28125}}
Name p1-3-hr3
UniqueID {'0': 3677084, '1': 1068567676, '2': 1927893732, '3': 2094808766}
DM {'dm_version': 3, 'file_size': 17382688, 'full_file_name': '../example_data/p1-3-hr3.dm3'}
original_filename ../example_data/p1-3-hr3.dm3
ApplicationBounds [0, 0, 1465, 2236]
DocumentObjectList {'0': {'AnnotationGroupList': {'0': {'AnnotationType': 31, 'BackgroundColor': [0, 0, 0], 'BackgroundMode': 1, 'FillMode': 1, 'Font': {'Attributes': 7, 'FamilyName': 'Arial Narrow', 'Size': 85}, 'ForegroundColor': [-1, -1, -1], 'HasBackground': 1, 'IsMoveable': 1, 'IsResizable': 1, 'IsSelectable': 1, 'IsTranslatable': 1, 'IsVisible': 1, 'ObjectTags': {}, 'Rectangle': [1768.0, 128.0, 1920.0, 1088.0], 'TextOffsetH': 1.0, 'TextOffsetV': 1.0, 'TextWidth': 195.18429565429688, 'UniqueID': 9}}, 'AnnotationType': 20, 'BackgroundColor': [-1, -1, -1], 'BackgroundMode': 2, 'FillMode': 1, 'ForegroundColor': [-1, 0, -32640], 'HasBackground': 0, 'ImageDisplayInfo': {'BrightColor': [-1, -1, -1], 'Brightness': 0.5, 'CaptionOn': 0, 'CaptionSize': 10, 'CLUT': [[0, 0, 0], [257, 257, 257], [514, 514, 514], [771, 771, 771], [1028, 1028, 1028], [1285, 1285, 1285], [1542, 1542, 1542], [1799, 1799, 1799], [2056, 2056, 2056], [2313, 2313, 2313], [2570, 2570, 2570], [2827, 2827, 2827], [3084, 3084, 3084], [3341, 3341, 3341], [3598, 3598, 3598], [3855, 3855, 3855], [4112, 4112, 4112], [4369, 4369, 4369], [4626, 4626, 4626], [4883, 4883, 4883], [5140, 5140, 5140], [5397, 5397, 5397], [5654, 5654, 5654], [5911, 5911, 5911], [6168, 6168, 6168], [6425, 6425, 6425], [6682, 6682, 6682], [6939, 6939, 6939], [7196, 7196, 7196], [7453, 7453, 7453], [7710, 7710, 7710], [7967, 7967, 7967], [8224, 8224, 8224], [8481, 8481, 8481], [8738, 8738, 8738], [8995, 8995, 8995], [9252, 9252, 9252], [9509, 9509, 9509], [9766, 9766, 9766], [10023, 10023, 10023], [10280, 10280, 10280], [10537, 10537, 10537], [10794, 10794, 10794], [11051, 11051, 11051], [11308, 11308, 11308], [11565, 11565, 11565], [11822, 11822, 11822], [12079, 12079, 12079], [12336, 12336, 12336], [12593, 12593, 12593], [12850, 12850, 12850], [13107, 13107, 13107], [13364, 13364, 13364], [13621, 13621, 13621], [13878, 13878, 13878], [14135, 14135, 14135], [14392, 14392, 14392], [14649, 14649, 14649], [14906, 14906, 14906], [15163, 15163, 15163], [15420, 15420, 15420], [15677, 15677, 15677], [15934, 15934, 15934], [16191, 16191, 16191], [16448, 16448, 16448], [16705, 16705, 16705], [16962, 16962, 16962], [17219, 17219, 17219], [17476, 17476, 17476], [17733, 17733, 17733], [17990, 17990, 17990], [18247, 18247, 18247], [18504, 18504, 18504], [18761, 18761, 18761], [19018, 19018, 19018], [19275, 19275, 19275], [19532, 19532, 19532], [19789, 19789, 19789], [20046, 20046, 20046], [20303, 20303, 20303], [20560, 20560, 20560], [20817, 20817, 20817], [21074, 21074, 21074], [21331, 21331, 21331], [21588, 21588, 21588], [21845, 21845, 21845], [22102, 22102, 22102], [22359, 22359, 22359], [22616, 22616, 22616], [22873, 22873, 22873], [23130, 23130, 23130], [23387, 23387, 23387], [23644, 23644, 23644], [23901, 23901, 23901], [24158, 24158, 24158], [24415, 24415, 24415], [24672, 24672, 24672], [24929, 24929, 24929], [25186, 25186, 25186], [25443, 25443, 25443], [25700, 25700, 25700], [25957, 25957, 25957], [26214, 26214, 26214], [26471, 26471, 26471], [26728, 26728, 26728], [26985, 26985, 26985], [27242, 27242, 27242], [27499, 27499, 27499], [27756, 27756, 27756], [28013, 28013, 28013], [28270, 28270, 28270], [28527, 28527, 28527], [28784, 28784, 28784], [29041, 29041, 29041], [29298, 29298, 29298], [29555, 29555, 29555], [29812, 29812, 29812], [30069, 30069, 30069], [30326, 30326, 30326], [30583, 30583, 30583], [30840, 30840, 30840], [31097, 31097, 31097], [31354, 31354, 31354], [31611, 31611, 31611], [31868, 31868, 31868], [32125, 32125, 32125], [32382, 32382, 32382], [32639, 32639, 32639], [-32640, -32640, -32640], [-32383, -32383, -32383], [-32126, -32126, -32126], [-31869, -31869, -31869], [-31612, -31612, -31612], [-31355, -31355, -31355], [-31098, -31098, -31098], [-30841, -30841, -30841], [-30584, -30584, -30584], [-30327, -30327, -30327], [-30070, -30070, -30070], [-29813, -29813, -29813], [-29556, -29556, -29556], [-29299, -29299, -29299], [-29042, -29042, -29042], [-28785, -28785, -28785], [-28528, -28528, -28528], [-28271, -28271, -28271], [-28014, -28014, -28014], [-27757, -27757, -27757], [-27500, -27500, -27500], [-27243, -27243, -27243], [-26986, -26986, -26986], [-26729, -26729, -26729], [-26472, -26472, -26472], [-26215, -26215, -26215], [-25958, -25958, -25958], [-25701, -25701, -25701], [-25444, -25444, -25444], [-25187, -25187, -25187], [-24930, -24930, -24930], [-24673, -24673, -24673], [-24416, -24416, -24416], [-24159, -24159, -24159], [-23902, -23902, -23902], [-23645, -23645, -23645], [-23388, -23388, -23388], [-23131, -23131, -23131], [-22874, -22874, -22874], [-22617, -22617, -22617], [-22360, -22360, -22360], [-22103, -22103, -22103], [-21846, -21846, -21846], [-21589, -21589, -21589], [-21332, -21332, -21332], [-21075, -21075, -21075], [-20818, -20818, -20818], [-20561, -20561, -20561], [-20304, -20304, -20304], [-20047, -20047, -20047], [-19790, -19790, -19790], [-19533, -19533, -19533], [-19276, -19276, -19276], [-19019, -19019, -19019], [-18762, -18762, -18762], [-18505, -18505, -18505], [-18248, -18248, -18248], [-17991, -17991, -17991], [-17734, -17734, -17734], [-17477, -17477, -17477], [-17220, -17220, -17220], [-16963, -16963, -16963], [-16706, -16706, -16706], [-16449, -16449, -16449], [-16192, -16192, -16192], [-15935, -15935, -15935], [-15678, -15678, -15678], [-15421, -15421, -15421], [-15164, -15164, -15164], [-14907, -14907, -14907], [-14650, -14650, -14650], [-14393, -14393, -14393], [-14136, -14136, -14136], [-13879, -13879, -13879], [-13622, -13622, -13622], [-13365, -13365, -13365], [-13108, -13108, -13108], [-12851, -12851, -12851], [-12594, -12594, -12594], [-12337, -12337, -12337], [-12080, -12080, -12080], [-11823, -11823, -11823], [-11566, -11566, -11566], [-11309, -11309, -11309], [-11052, -11052, -11052], [-10795, -10795, -10795], [-10538, -10538, -10538], [-10281, -10281, -10281], [-10024, -10024, -10024], [-9767, -9767, -9767], [-9510, -9510, -9510], [-9253, -9253, -9253], [-8996, -8996, -8996], [-8739, -8739, -8739], [-8482, -8482, -8482], [-8225, -8225, -8225], [-7968, -7968, -7968], [-7711, -7711, -7711], [-7454, -7454, -7454], [-7197, -7197, -7197], [-6940, -6940, -6940], [-6683, -6683, -6683], [-6426, -6426, -6426], [-6169, -6169, -6169], [-5912, -5912, -5912], [-5655, -5655, -5655], [-5398, -5398, -5398], [-5141, -5141, -5141], [-4884, -4884, -4884], [-4627, -4627, -4627], [-4370, -4370, -4370], [-4113, -4113, -4113], [-3856, -3856, -3856], [-3599, -3599, -3599], [-3342, -3342, -3342], [-3085, -3085, -3085], [-2828, -2828, -2828], [-2571, -2571, -2571], [-2314, -2314, -2314], [-2057, -2057, -2057], [-1800, -1800, -1800], [-1543, -1543, -1543], [-1286, -1286, -1286], [-1029, -1029, -1029], [-772, -772, -772], [-515, -515, -515], [-258, -258, -258], [-1, -1, -1]], 'CLUTName': 'Greyscale', 'ComplexMode': 4, 'ComplexRange': 1000.0, 'Contrast': 0.5, 'ContrastMode': 1, 'DimensionLabels': {'0': ''}, 'DoAutoSurvey': 1, 'EstimatedMax': 656.0, 'EstimatedMaxTrimPercentage': 0.0010000000474974513, 'EstimatedMin': 2354.0, 'EstimatedMinTrimPercentage': 0.0010000000474974513, 'Gamma': 0.5, 'HighLimit': 2107.03125, 'HiLimitContrastDeltaTriggerPercentage': 0.0, 'IsInverted': 0, 'LowLimit': 718.9080810546875, 'LowLimitContrastDeltaTriggerPercentage': 0.0, 'MainSliceId': {'0': 0}, 'MinimumContrast': 0.0, 'RangeAdjust': 1.0, 'SparseSurvey_GridSize': 16, 'SparseSurvey_NumberPixels': 32, 'SparseSurvey_UseNumberPixels': 1, 'SurveyTechique': 2}, 'ImageDisplayType': 1, 'ImageSource': 0, 'IsMoveable': 1, 'IsResizable': 1, 'IsSelectable': 1, 'IsTranslatable': 1, 'IsVisible': 1, 'ObjectTags': {}, 'Rectangle': [0.0, 0.0, 1427.0, 1427.0], 'UniqueID': 8}}
DocumentTags {}
HasWindowPosition 1
Image Behavior {'DoIntegralZoom': 0, 'ImageDisplayBounds': [0.0, 0.0, 1427.0, 1427.0], 'IsZoomedToWindow': 1, 'UnscaledTransform': {'Offset': [0.0, 0.0], 'Scale': [1.0, 1.0]}, 'ViewDisplayID': 8, 'WindowRect': [0.0, 0.0, 1427.0, 1427.0], 'ZoomAndMoveTransform': {'Offset': [0.0, 0.0], 'Scale': [1.0, 1.0]}}
ImageSourceList {'0': {'ClassName': 'ImageSource:Simple', 'Id': {'0': 0}, 'ImageRef': 1}}
InImageMode 1
MinVersionList {'0': {'RequiredVersion': 50659328}}
NextDocumentObjectID 10
Page Behavior {'DoIntegralZoom': 0, 'DrawMargins': 1, 'DrawPaper': 1, 'IsFixedInPageMode': 0, 'IsZoomedToWindow': 1, 'LayedOut': 0, 'PageTransform': {'Offset': [0.0, 0.0], 'Scale': [1.0, 1.0]}, 'RestoreImageDisplayBounds': [0.0, 0.0, 2048.0, 2048.0], 'RestoreImageDisplayID': 8, 'TargetDisplayID': 4294967295}
PageSetup {'General': [1, 1000, 8500, 11000, 1000, 1000, -1000, -1000], 'Win32': b'\x06\x00\x00\x004!\x00\x00\xf8*\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xe8\x03\x00\x00\xe8\x03\x00\x00\xe8\x03\x00\x00\xe8\x03\x00\x00\x00\x00\x00\x00\x01\x00\x01\x00\x01\x00\x01\x00\x01\x00\x1b\x10', 'Win32_DevModeW': b'S\x00e\x00n\x00d\x00 \x00T\x00o\x00 \x00O\x00n\x00e\x00N\x00o\x00t\x00e\x00 \x002\x000\x001\x000\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x04\x00\x06\xdc\x00\x0c\x03\x03\xff\x81\x03\x01\x00\x01\x00\xea\no\x08d\x00\x01\x00\x07\x00\xfd\xff\x02\x00\x01\x00X\x02\x01\x00\x00\x00L\x00e\x00t\x00t\x00e\x00r\x00 \x008\x00.\x005\x00"\x00x\x001\x001\x00"\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00\x01\x00\x00\x00\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00DINU"\x00\xd0\x00\x0c\x03\x00\x00\xc2\xac\x90Q\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00\x00\x00\x07\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xd0\x00\x00\x00SMTJ\x00\x00\x00\x00\x10\x00\xc0\x00S\x00e\x00n\x00d\x00 \x00T\x00o\x00 \x00M\x00i\x00c\x00r\x00o\x00s\x00o\x00f\x00t\x00 \x00O\x00n\x00e\x00N\x00o\x00t\x00e\x00 \x002\x000\x001\x000\x00 \x00D\x00r\x00i\x00v\x00e\x00r\x00\x00\x00RESDLL\x00UniresDLL\x00PaperSize\x00LETTER\x00Orientation\x00PORTRAIT\x00Resolution\x00DPI600\x00ColorMode\x0024bpp\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', 'Win32_DevNamesW': b'\x04\x00*\x00?\x00\x00\x00S\x00e\x00n\x00d\x00 \x00T\x00o\x00 \x00M\x00i\x00c\x00r\x00o\x00s\x00o\x00f\x00t\x00 \x00O\x00n\x00e\x00N\x00o\x00t\x00e\x00 \x002\x000\x001\x000\x00 \x00D\x00r\x00i\x00v\x00e\x00r\x00\x00\x00S\x00e\x00n\x00d\x00 \x00T\x00o\x00 \x00O\x00n\x00e\x00N\x00o\x00t\x00e\x00 \x002\x000\x001\x000\x00\x00\x00n\x00u\x00l\x00:\x00\x00\x00'}
SentinelList {}
Thumbnails {'0': {'ImageIndex': 0, 'SourceSize_Pixels': [1427, 1427]}}
WindowPosition [30, 801, 1457, 2228]
None

Any python object will provide a help.

help(current_dataset)

Fetching long content....

All attributes of a python object can be viewed with the * dir* command.

As above: too much information for normal use, but it is there if needed.

dir(current_dataset)

['A',
 'T',
 '_Array__chunks',
 '_Array__name',
 '_Dataset__protected',
 '_Dataset__rearrange_axes',
 '_Dataset__reduce_dimensions',
 '_Dataset__validate_dim',
 '__abs__',
 '__add__',
 '__and__',
 '__array__',
 '__array_function__',
 '__array_priority__',
 '__array_ufunc__',
 '__await__',
 '__bool__',
 '__class__',
 '__complex__',
 '__dask_graph__',
 '__dask_keys__',
 '__dask_layers__',
 '__dask_optimize__',
 '__dask_postcompute__',
 '__dask_postpersist__',
 '__dask_scheduler__',
 '__dask_tokenize__',
 '__deepcopy__',
 '__delattr__',
 '__dict__',
 '__dir__',
 '__div__',
 '__divmod__',
 '__doc__',
 '__eq__',
 '__firstlineno__',
 '__float__',
 '__floordiv__',
 '__format__',
 '__ge__',
 '__getattribute__',
 '__getitem__',
 '__getstate__',
 '__gt__',
 '__hash__',
 '__index__',
 '__init__',
 '__init_subclass__',
 '__int__',
 '__invert__',
 '__iter__',
 '__le__',
 '__len__',
 '__long__',
 '__lshift__',
 '__lt__',
 '__matmul__',
 '__mod__',
 '__module__',
 '__mul__',
 '__ne__',
 '__neg__',
 '__new__',
 '__nonzero__',
 '__or__',
 '__pos__',
 '__pow__',
 '__radd__',
 '__rand__',
 '__rdiv__',
 '__rdivmod__',
 '__reduce__',
 '__reduce_ex__',
 '__repr__',
 '__rfloordiv__',
 '__rlshift__',
 '__rmatmul__',
 '__rmod__',
 '__rmul__',
 '__ror__',
 '__rpow__',
 '__rrshift__',
 '__rshift__',
 '__rsub__',
 '__rtruediv__',
 '__rxor__',
 '__setattr__',
 '__setitem__',
 '__sizeof__',
 '__slots__',
 '__static_attributes__',
 '__str__',
 '__sub__',
 '__subclasshook__',
 '__truediv__',
 '__weakref__',
 '__xor__',
 '_axes',
 '_cached_keys',
 '_chunks',
 '_closest_point',
 '_data_type',
 '_elemwise',
 '_griddata_transform',
 '_h5_dataset',
 '_key_array',
 '_meta',
 '_metadata',
 '_min_dist',
 '_modality',
 '_name',
 '_original_metadata',
 '_quantity',
 '_rebuild',
 '_repr_html_',
 '_reset_cache',
 '_scalarfunc',
 '_source',
 '_structures',
 '_title',
 '_units',
 '_variance',
 '_vindex',
 'abs',
 'add_provenance',
 'add_structure',
 'adjust_axis',
 'all',
 'angle',
 'any',
 'argmax',
 'argmin',
 'argtopk',
 'astype',
 'blocks',
 'choose',
 'chunks',
 'chunksize',
 'clip',
 'compute',
 'compute_chunk_sizes',
 'conj',
 'copy',
 'cumprod',
 'cumsum',
 'dask',
 'data_descriptor',
 'data_type',
 'del_dimension',
 'dim_0',
 'dim_1',
 'dot',
 'dtype',
 'fft',
 'flatten',
 'flatten_complex',
 'fold',
 'from_array',
 'get_dimension_by_number',
 'get_dimension_slope',
 'get_dimensions_by_type',
 'get_dimensions_types',
 'get_extent',
 'get_image_dims',
 'get_spectral_dims',
 'h5_dataset',
 'hdf_close',
 'imag',
 'itemsize',
 'labels',
 'like_data',
 'map_blocks',
 'map_overlap',
 'max',
 'mean',
 'metadata',
 'min',
 'modality',
 'moment',
 'name',
 'nbytes',
 'ndim',
 'nonzero',
 'npartitions',
 'numblocks',
 'original_metadata',
 'partitions',
 'persist',
 'plot',
 'point_cloud',
 'prod',
 'provenance',
 'quantity',
 'ravel',
 'real',
 'rechunk',
 'reduce_dims',
 'rename_dimension',
 'repeat',
 'reshape',
 'round',
 'set_dimension',
 'set_thumbnail',
 'shape',
 'shuffle',
 'size',
 'source',
 'squeeze',
 'std',
 'store',
 'structures',
 'sum',
 'swapaxes',
 'title',
 'to_backend',
 'to_dask_dataframe',
 'to_delayed',
 'to_hdf5',
 'to_svg',
 'to_tiledb',
 'to_zarr',
 'topk',
 'trace',
 'transpose',
 'unfold',
 'units',
 'var',
 'variance',
 'view',
 'view_metadata',
 'view_original_metadata',
 'vindex',
 'visualize',
 'x',
 'y']

Adding Data¶

To add another dataset that belongs to this measurement we will use the h5_add_channel from file_tools in the pyTEMlib package.

Here is how we add a channel there.

We can also add a new measurement group (add_measurement in pyTEMlib) for similar datasets.

This is equivalent to making a new directory in a file structure on your computer.

datasets['Copied_of_Channel_000'] = current_dataset.copy()

We use above functions to add the content of a (random) data-file to the current file.

This is important if you for example want to add a Z-contrast or survey-image to a spectrum image.

Therefore, these functions enable you to collect the data from different files that belong together.

datasets.keys()

dict_keys(['Channel_000', 'Copied_of_Channel_000'])

Adding additional information¶

Similarly, we can add a whole new measurement group or a structure group.

This function will be contained in the KinsCat package of pyTEMlib.

If you loaded the example image, with graphite and ZnO both are viewed in the [1,1,1] zone axis.

import ase
                                                                                 
graphite = pyTEMlib.crystal_tools.structure_by_name('Graphite')
print(graphite)

Atoms(symbols='C4', pbc=False, cell=[[2.46772414, 0.0, 0.0], [-1.2338620699999996, 2.1371117947721068, 0.0], [0.0, 0.0, 6.711]])

current_dataset.structures['Crystal_000'] = graphite
                                                            
zinc_oxide = pyTEMlib.crystal_tools.structure_by_name('ZnO')
current_dataset.structures['ZnO'] = zinc_oxide

Keeping Track of Analysis and Results¶

A notebook is notorious for getting confusing, especially if one uses different notebooks for different task, but store them in the same file.

If you like a result of your calculation, log it.

Use the datasets dictionary to add a analysed and/or modified dataset. Make sure the metadata contain all the necessary information, so that you will know later what you did.

The convention in this class will be to call the dataset Log_000.

new_dataset = current_dataset.T
new_dataset.metadata = {'analysis': 'Nothing', 'name': 'Nothing'}
datasets['Log_000'] = new_dataset

An example for a log¶

We log the Fourier Transform of the image we loaded

First we perform the calculation

fft_image = current_dataset.fft().abs()
fft_image = np.log(60+fft_image)

view = fft_image.plot()

Now that we like this we log it.

Please note that just saving the fourier transform would not be good as we also need the scale and such.

fft_image.title = 'FFT Gamma corrected'
fft_image.metadata = {'analysis': 'fft'}
datasets['Log_001'] = fft_image

view = fft_image.plot()

We added quite a few datasets to our dictionary.

Let’s have a look

chooser = pyTEMlib.file_tools.ChooseDataset(datasets)

view = chooser.dataset.plot()

Save Datasets to hf5_file¶

Write all datasets to one h5_file, which we then close immediatedly

h5_group = pyTEMlib.file_tools.save_dataset(datasets, filename='./nix.hf5')

Cannot overwrite file. Using:  nix-1.hf5

C:\Users\gduscher\AppData\Local\anaconda3\Lib\site-packages\pyNSID\io\hdf_io.py:111: UserWarning: main_data_name should not contain the "-" character. Reformatted name from:p1-3-hr3 to p1_3_hr3
  warn('main_data_name should not contain the "-" character. Reformatted'
C:\Users\gduscher\AppData\Local\anaconda3\Lib\site-packages\pyNSID\io\hdf_utils.py:381: FutureWarning: validate_h5_dimension may be removed in a future version
  warn('validate_h5_dimension may be removed in a future version',
C:\Users\gduscher\AppData\Local\anaconda3\Lib\site-packages\pyNSID\io\hdf_io.py:111: UserWarning: main_data_name should not contain the "-" character. Reformatted name from:p1-3-hr3 to p1_3_hr3
  warn('main_data_name should not contain the "-" character. Reformatted'
C:\Users\gduscher\AppData\Local\anaconda3\Lib\site-packages\pyNSID\io\hdf_utils.py:381: FutureWarning: validate_h5_dimension may be removed in a future version
  warn('validate_h5_dimension may be removed in a future version',
C:\Users\gduscher\AppData\Local\anaconda3\Lib\site-packages\pyNSID\io\hdf_io.py:111: UserWarning: main_data_name should not contain the "-" character. Reformatted name from:Transposed_p1-3-hr3 to Transposed_p1_3_hr3
  warn('main_data_name should not contain the "-" character. Reformatted'
C:\Users\gduscher\AppData\Local\anaconda3\Lib\site-packages\pyNSID\io\hdf_utils.py:381: FutureWarning: validate_h5_dimension may be removed in a future version
  warn('validate_h5_dimension may be removed in a future version',
C:\Users\gduscher\AppData\Local\anaconda3\Lib\site-packages\pyNSID\io\hdf_utils.py:381: FutureWarning: validate_h5_dimension may be removed in a future version
  warn('validate_h5_dimension may be removed in a future version',

Close the file

h5_group.file.close()

Open h5_file¶

Open the h5_file that we just created

datasets2= pyTEMlib.file_tools.open_file(filename='./nix-1.hf5')

chooser = pyTEMlib.file_tools.ChooseDataset(datasets2)

Short check if we got the data right¶

we print the tree and we plot the data

view = chooser.dataset.plot()

Back Matplotlib and Numpy for Micrographs
Next: Overview
Chapter 1: Introduction
List of Content: Front

Open DM3 Images, Spectra, Spectrum-Images and Image-Stacks with pyNSID

Open DM3 Images, Spectra, Spectrum-Images and Image-Stacks with pyNSID¶

Import packages for figures and¶

Check Installed Packages¶

Load the plotting and figure packages¶

Open a file¶

Data Structure¶

Data Structure¶

Adding Data¶

Adding additional information¶

Keeping Track of Analysis and Results¶

An example for a log¶

Save Datasets to hf5_file¶

Open h5_file¶

Short check if we got the data right¶

Navigation¶