CNMF#

Accessor: cnmf

class mesmerize_core.CNMFExtensions(s)#

Extensions for managing CNMF output data

Parameters:

s (Series) –

get_cnmf_memmap(mode='r')#

Get the CNMF C-order memmap. This should NOT be used for viewing the movie frames use caiman.get_input_movie() for that purpose.

Parameters:

mode (str) –

passed to numpy.memmap

one of: {‘r+’, ‘r’, ‘w+’, ‘c’}

The file is opened in this mode:

’r’

Open existing file for reading only.

’r+’

Open existing file for reading and writing.

’w+’

Create or overwrite existing file for reading and writing.

’c’

Copy-on-write: assignments affect data in memory, but changes are not saved to disk. The file on disk is read-only.

Returns:

numpy memmap array used for CNMF

Return type:

np.ndarray

get_output_path()#

Get the path to the cnmf hdf5 output file.

Note: You generally want to work with the other extensions instead of directly using the hdf5 file.

Returns:

full path to the caiman-format CNMF hdf5 output file

Return type:

Path

get_output(return_copy=True)#
Parameters:

return_copy (bool) –

if True returns a copy of the cached value in memory.
if False returns the same object as the cached value in memory, not recommend this could result in strange unexpected behavior.
In general you want a copy of the cached value.

Returns:

Returns the Caiman CNMF object

Return type:

CNMF

Examples

Load the CNMF model with estimates from the hdf5 file.

from mesmerize_core import load_batch

df = load_batch("/path/to/batch_dataframe_file.pickle")

# assume the 0th index is a cnmf item
cnmf_obj = df.iloc[0].cnmf.get_output()

# see some estimates
print(cnmf_obj.estimates.C)
print(cnmf_obj.estimates.f)
get_masks(component_indices=None, threshold=0.01, return_copy=True)#
Get binary masks of the spatial components at the given component_indices.
Created from CNMF.estimates.A
Parameters:

  • component_indices (str or np.ndarray, optional) –

    indices of the components to include
    if not provided, None, or "all" uses all components
    if "good" uses good components, i.e. Estimates.idx_components
    if "bad" uses bad components, i.e. Estimates.idx_components_bad
    if np.ndarray, uses the indices in the provided array

  • threshold (float) – threshold

  • return_copy (bool) –

    if True returns a copy of the cached value in memory.
    if False returns the same object as the cached value in memory, not recommend this could result in strange unexpected behavior.
    In general you want a copy of the cached value.

Returns:

shape is [dim_0, dim_1, n_components]

Return type:

np.ndarray

get_contours(component_indices=None, swap_dim=True, return_copy=True)#

Get the contour and center of mass for each spatial footprint

Parameters:

  • component_indices (str or np.ndarray, optional) –

    indices of the components to include
    if not provided, None, or "all" uses all components
    if "good" uses good components, i.e. Estimates.idx_components
    if "bad" uses bad components, i.e. Estimates.idx_components_bad
    if np.ndarray, uses the indices in the provided array

  • swap_dim (bool) – swap the x and y coordinates, use if the contours don’t align with the cells in your image

  • return_copy (bool) –

    if True returns a copy of the cached value in memory.
    if False returns the same object as the cached value in memory, not recommend this could result in strange unexpected behavior.
    In general you want a copy of the cached value.

Returns:

(List[coordinates array], List[centers of masses array])
each array of coordinates is 2D, [xs, ys]
each center of mass is [x, y]

Return type:

Tuple[List[np.ndarray], List[np.ndarray]]

Examples

This example loads the input movie and contours, and plots them with fastplotlib

from mesmerize_core import load_batch

# needs fastplotlib and must be run in a notebook
from fastplotlib import Plot
from ipywidgets import IntSlider, VBox

df = load_batch("/path/to/batch_dataframe_file.pickle")

# assuming the 0th index is a cnmf item
movie = df.iloc[0].caiman.get_input_movie()
contours, coms = df.iloc[0].cnmf.get_contours()

# the following requires fastplotlib and must be run in a new notebook cell
slider = IntSlider(value=0, min=0, max=movie.shape[0] - 1, step=1)
plot = Plot()

image_graphic = plot.image(movie[0].T, cmap="gnuplot2")
# note the movie frame is transposed, this is sometimes requires to get the contours to align

for coor in contours:
    # line data has to be 3D
    zs = np.ones(coor.shape[0])  # this will place it above the image graphic
    c3d = [coor[:, 0], coor[:, 1], zs]
    coors_3d = np.dstack(c3d)[0]

    # make all the lines red, [R, G, B, A] array
    colors = np.vstack([[1., 0., 0., 0.7]] * coors_3d.shape[0])
    plot.line(data=coors_3d, colors=colors)

previous_slider_value = 0
def update_frame():  # runs on each rendering cycle
    if slider.value == previous_slider_value:
        return
    image_graphic.update_data(data=movie[slider.value].T)

plot.add_animations([update_frame])

VBox([plot.show(), slider])
get_temporal(component_indices=None, add_background=False, return_copy=True)#

Get the temporal components for this CNMF item, basically CNMF.estimates.C

Parameters:

  • component_indices (str or np.ndarray, optional) –

    indices of the components to include
    if not provided, None, or "all" uses all components
    if "good" uses good components, i.e. Estimates.idx_components
    if "bad" uses bad components, i.e. Estimates.idx_components_bad
    if np.ndarray, uses the indices in the provided array

  • add_background (bool) – if True, add the temporal background, cnmf.estimates.C + cnmf.estimates.f

  • return_copy (bool) –

    if True returns a copy of the cached value in memory.
    if False returns the same object as the cached value in memory, not recommend this could result in strange unexpected behavior.
    In general you want a copy of the cached value.

Returns:

shape is [n_components, n_frames]

Return type:

np.ndarray

Examples

Plot the temporal components as a heatmap

from mesmerize_core import load_batch
from seaborn import heatmap

df = load_batch("/path/to/batch_dataframe_file.pickle")

# assumes 0th index is a cnmf batch item
temporal = df.iloc[0].cnmf.get_temporal()

heatmap(temporal)
get_rcm(component_indices=None, temporal_components=None, return_copy=False)#

Return the reconstructed movie with no background, i.e. A ⊗ C, as a LazyArray. This is an array that performs lazy computation of the reconstructed movie only upon indexing.

Parameters:

  • component_indices (optional, Union[np.ndarray, str]) –

    indices of the components to include
    if np.ndarray, uses these indices in the provided array
    if "good" uses good components, i.e. cnmf.estimates.idx_components
    if "bad" uses bad components, i.e. cnmf.estimates.idx_components_bad
    if not provided, None, or "all" uses all components

  • temporal_components (optional, np.ndarray) –

    temporal components to use as C for computing reconstructed movie.

    uses cnmf.estimates.C if not provided
    useful if you want to create the reconstructed movie using dF/Fo, z-scored data, etc.

  • return_copy (bool, default False) –

    if True returns a copy of the cached value in memory.
    if False returns the same object as the cached value in memory
    False is used by default when returning LazyArrays for technical reasons

Returns:

shape is [n_frames, x_dims, y_dims]

Return type:

LazyArray

Examples

This example uses fastplotlib to display the reconstructed movie from a CNMF item that has already been run.

fastplotlib code must be run in a notebook
See the demo notebooks for more detailed examples.
from mesmerize_core import *
from fastplotlib.widgets import ImageWidget

# load existing batch
df = load_batch("/path/to/batch.pickle")

# get the reconstructed movie as LazyArray
# assumes the last index, `-1`, is a cnmf item
# uses only the "good" components
rcm = df.iloc[-1].cnmf.get_rcm(component_indices="good")

# view with ImageWidget
iw = ImageWidget(data=rcm)
iw.show()
get_rcb()#

Return the reconstructed background, (b ⊗ f)

Returns:

shape is [n_frames, x_dims, y_dims]

Return type:

LazyArray

Examples

This example uses fastplotlib to display the reconstructed movie from a CNMF item that has already been run.

fastplotlib code must be run in a notebook
See the demo notebooks for more detailed examples.
from mesmerize_core import *
from fastplotlib.widgets import ImageWidget

# load existing batch
df = load_batch("/path/to/batch.pickle")

# get the reconstructed background as a LazyArray
# assumes the last index, `-1`, is a cnmf item
rcb = df.iloc[-1].cnmf.get_rcb()

# view with ImageWidget
iw = ImageWidget(data=rcb)
iw.show()
get_residuals()#

Return residuals, Y - (A ⊗ C) - (b ⊗ f)

Returns:

shape is [n_frames, x_dims, y_dims]

Return type:

LazyArray

Examples

This example uses fastplotlib to display the reconstructed movie from a CNMF item that has already been run.

fastplotlib code must be run in a notebook
See the demo notebooks for more detailed examples.
from mesmerize_core import *
from fastplotlib.widgets import ImageWidget

# load existing batch
df = load_batch("/path/to/batch.pickle")

# get the reconstructed background as a LazyArray
# assumes the last index, `-1`, is a cnmf item
residuals = df.iloc[-1].cnmf.get_residuals()

# view with ImageWidget
iw = ImageWidget(data=residuals)
iw.show()
run_detrend_dfof(quantileMin=8, frames_window=500, flag_auto=True, use_fast=False, use_residuals=True, detrend_only=False)#
Uses caiman’s detrend_df_f.
call CNMF.get_detrend_dfof() to get the values.
Sets CNMF.estimates.F_dff

Warning

Overwrites the existing cnmf hdf5 output file for this batch item

Parameters:

  • quantileMin (float) – quantile used to estimate the baseline (values in [0,100]) used only if ‘flag_auto’ is False, i.e. ignored by default

  • frames_window (int) – number of frames for computing running quantile

  • flag_auto (bool) – flag for determining quantile automatically

  • use_fast (bool) – flag for using approximate fast percentile filtering

  • detrend_only (bool) – flag for only subtracting baseline and not normalizing by it. Used in 1p data processing where baseline fluorescence cannot be determined.

  • use_residuals (bool) –

Return type:

None

Notes

invalidates the cache for this batch item.

get_detrend_dfof(component_indices=None, return_copy=True)#

Get the detrended dF/F0 curves after calling run_detrend_dfof. Basically CNMF.estimates.F_dff.

Parameters:

  • component_indices (str or np.ndarray, optional) –

    indices of the components to include
    if not provided, None, or "all" uses all components
    if "good" uses good components, i.e. Estimates.idx_components
    if "bad" uses bad components, i.e. Estimates.idx_components_bad
    if np.ndarray, uses the indices in the provided array

  • return_copy (bool) –

    if True returns a copy of the cached value in memory.
    if False returns the same object as the cached value in memory, not recommend this could result in strange unexpected behavior.
    In general you want a copy of the cached value.

Returns:

shape is [n_components, n_frames]

Return type:

np.ndarray

run_eval(params)#

Run component evaluation. This basically changes the indices for good and bad components.

Warning

Overwrites the existing cnmf hdf5 output file for this batch item

Parameters:

params (dict) –

dict of parameters for component evaluation

parameter

details

SNR_lowest

float, minimum accepted SNR value

cnn_lowest

float, minimum accepted value for CNN classifier

gSig_range

List[int, int] or None, range for gSig scale for CNN classifier

min_SNR

float, transient SNR threshold

min_cnn_thr

float, threshold for CNN classifier

rval_lowest

float, minimum accepted space correlation

rval_thr

float, space correlation threshold

use_cnn

bool, use CNN based classifier

use_ecc

bool, flag for eccentricity based filtering

max_ecc

float, max eccentricity

Return type:

None

Notes

invalidates the cache for this batch item.

get_good_components()#

get the good component indices, Estimates.idx_components

Returns:

array of ints, indices of good components

Return type:

np.ndarray

get_bad_components()#

get the bad component indices, Estimates.idx_components_bad

Returns:

array of ints, indices of bad components

Return type:

np.ndarray

Lazy Arrays#

These are returned by the respective cnmf extensions (see above), get_rcm(), get_rcb(), and get_residuals(). They make it possible to view large arrays which would otherwise be larger than RAM.

class mesmerize_core.arrays.LazyArrayRCM(spatial, temporal, frame_dims)#

LazyArray for reconstructed movie, i.e. A ⊗ C

Parameters:

  • spatial (ndarray) –

  • temporal (ndarray) –

  • frame_dims (Tuple[int, int]) –

property n_frames: int#

int number of frames

property shape: Tuple[int, int, int]#

Tuple[int] (n_frames, dims_x, dims_y)

property dtype: str#

str data type

property min: float#

int min value of the array if it were fully computed

property max: float#

float max value of the array if it were fully computed

class mesmerize_core.arrays.LazyArrayRCB(spatial, temporal, frame_dims)#

Lazy array for reconstructed background, i.e. b ⊗ f

Parameters:

  • spatial (ndarray) –

  • temporal (ndarray) –

  • frame_dims (Tuple[int, int]) –

class mesmerize_core.arrays.LazyArrayResiduals(raw_movie, rcm, rcb, timeout=10)#

Lazy array for residuals, i.e. Y - (A ⊗ C) - (b ⊗ f)

Parameters:
property dtype: str#

str data type

property shape: Tuple[int, int, int]#

Tuple[int] (n_frames, dims_x, dims_y)

property n_frames: int#

int number of frames

property min: float#

int min value of the array if it were fully computed

property max: float#

float max value of the array if it were fully computed