ScanImage Tiff Reader

Download

Pre-built binaries are available for 64-bit Windows, OS X, or Linux.

Target Version Download
Windows x64 1.3 Download
OS X 1.3 Download
Linux 1.3 Download

About

The ScanImageTiffReader is a command line interface and C library that reads data from Tiff and BigTiff files recorded using ScanImage. It was written with performance in mind and provides access to ScanImage-specific metadata. This library should actually work with most tiff files, but as of now we don’t support compressed or tiled data.

Cross-platform language bindings are available for Python (2.7.14 and 3.6), Matlab (2016b+), or Julia (1.0). Those are available from their respective package managers. See the links for more information.

Both ScanImage and this reader are products of Vidrio Technologies. If you have questions or need support feel free to email.

Examples

Each example shows how to extract raw image data from the file.

Command Line Interface

ScanImageTiffReader image data my.tif > my.raw

C Library

#include <tiff.reader.api.h>
const size_t bytes_per_pixel[]={1,2,4,8,1,2,4,8,4,8};
ScanImageTiffReader reader=ScanImageTiffReader_Open("my.tif");
struct nd shape=ScanImageTiffReader_GetShape(reader);
size_t nbytes=bytes_per_pixel[shape->type]*shape.strides[shape.ndim];
void *data=malloc(nbytes);
ScanImageTiffReader_GetData(reader,data,nbytes);
ScanImageTiffReader_Close(reader);

ScanImage Tiff Files

ScanImage records images, volumes and video as Tiff or BigTiff files. For the most part Tiff and BigTiff are similar, but they are not the same; BigTiff enables storage of larger (>4GB) data sets. In addition to the image data, ScanImage stores metadata describing the microscope configuration and settings used during the acquisition. These are stored in the file itself.

Some of this metadata is accessible using standard Tiff readers. The tiff format provides for data fields (tags) which can be used to attach data to describing each frame. Past versions of ScanImage would store a copy the metadata in “image description” tag for each frame. Much of the metadata is redundant, and this can lead to longer load times and significant storage overhead in some cases.

Fortunately, the tiff format is very flexible and allows us to easily store the part of the metadata that doesn’t change over time in a dedicated block of the file. However, this metadata block is only accessible if you know where to look. The tiff files can still be read by any reader conforming to the baseline tiff specification, but those readers will not be aware of the ScanImage metadata block. The ScanImageTiffReader knows how to extract this metadata and also provides fast access to the images themselves.

ScanImage tiff files store the following kinds of data:

Kind Description
data The image data itself
metadata Frame-invariant metadata such as the microscope configuration and region of interest descriptions.
descriptions Frame-varying metadata such as timestamps and I2C data.

The metadata sections themselves are encoded as either matlab evaluable strings or json.

The ScanImage documentation has a very detailed description of how data is stored in a ScanImage Tiff.

C Library

The C API is provided as a static or shared library. It requires no dependencies. We try to statically link wherever possible.

Open/Close

Use ScanImageTiffReader_Open() to open a file. When you’re done call ScanImageTiffReader_Close() to close any open file handles and cleanup any other resources in use.

Error handling

When errors occur, functions usually return a 0. Any error messages will be contained in the log field of the ScanImageTiffReader context object. Normally the log string is NULL; only after an error does it point to a null-terminated string.

Image data

To read the image data,

  1. Determine the size of the buffer you’ll need to hold the raw data using ScanImageTiffReader_GetDataSizeBytes().
  2. Allocate the memory you need.
  3. Read using ScanImageTiffReader_GetData(). This loads the entire volume into memory. There’s no option for reading just a subset of the data.

One nice thing about ScanImageTiffReader_GetData() is that many asynchronous read requests are made at once. This typically means that read commands will be coallesced and queued by the operating system so that very high read bandwidths can be acheived.

Metadata

Reading metadata follows a similar pattern. To read the frame-invariant metadata:

  1. Determine the size of the buffer you need using ScanImageTiffReader_GetMetadataSizeBytes().
  2. Read it using ScanImageTiffReader_GetMetadata().

The frame-varying metadata is read using a similar pattern. There are two sets of functions for reading the frame varying data:

Command Line Interface

ScanImageTiffReader

The command line tool is designed to make inspecting files simple. It also allows extraction of the image data into a raw file.

Basic Commands
Command Description
ScanImageTiffReader descriptions Print all image descriptions.
with descriptions --frame <id> Print one image description.
ScanImageTiffReader metadata Print the ScanImage metadata.
ScanImageTiffReader data Print the raw image data as a byte stream.
ScanImageTiffReader help Print detailed help for a sub-command.

Usage

From the command line:

ScanImageTiffReader

will output a basic help message:

Usage: ScanImageTiffReader <command> [<args>]

These are the available commands:
   descriptions     Extract the contents of a the image description tag(s).
                    If you do not specify which frame, then all the image
                    descriptions are output.
          image     Do stuff with image data.
       metadata     Extract the ScanImage metadata section from the file.
           help     Print detailed help for the specified sub-command.

Sub-commands

Each sub-command has it’s own arguments. The available sub-commands are listed below.

help <subcommand>

Returns more detailed information for each sub-command.

descriptions [--frame <index>] <input tiff> [<output file>]

Extract header data. ScanImage stores (a lot) of information about the state of the microscope in each frame. This command will extract all the information for all the frames. If you use --frame only that frame’s header will be output.

metadata <input tiff> [<output file>]

Extracts the metadata section. ScanImage stores some non-frame varying data in the Tiff file, which is what this command extracts.

Historically, this data has had several different formats. Some are human readible text, but some are binary formats that are difficult to interpret outside of Matlab. ScanImage provides a matlab-specific tool for decoding the binary formats.

image bytes <input file>

Return the size of the raw image data in bytes.

image raw <input file> <output file>

Write raw bytes to an output file for the entire volume. The pixels are written in row-major order (C-style), so for an array with dimensions [w,h,d] the index of a pixel at (x,y,z) is x+y*w+z*w*h.

image shape <input file>

Print the dimensions and pixel type of the volume in the tiff stack.

Performance

Compared to many common BigTiff readers, the ScanImageTiffReader is fast. When done right, reading a Tiff file can have very low overhead. One should expect that the read performance is roughly the same as the bandwidth-limiting bottleneck, usually the hard-drive used for storage.

Thanks to solid-state storage and aggressive caching of files by the operating systems, read speeds of 500 MB/s or greater are very acheivable. Realizing this bandwidth improvement can reduce read times by an order of magnitude or more.

Reading from an SSD drive (Samsung 840 Pro),

_images/drive_stats.png

Benchmark speeds for the storage drive used here.

_images/file_explorer.png

A 6GB BigTiff file produced by ScanImage.

A roughly 6GB file should take ~11.4 seconds to read. Using the ScanImageTiffReader:

_images/julia_si.png

Time to read a stack using the ScanImageTiffReader API in Julia. The choice of language doesn’t make a significant difference.

We can compute the effective read bandwidth by dividing the total byte size of the file (6.277 GB) by the amount of time it took to read the file. In the Julia example above, this comes out to 430 MB/s.

Due to caching by the operating system, we sometimes exceed the expected speed of the hard drive. The behavior of this kind of file-system caching might vary between operating systems.