Quick start guide

This tutorial will show you how to utilize the ‘livecellx’ package to extract sinlge cell trajectories from long live-cell imaging data by segmenting and tracking single cells, and then compute single cell features in a latent space.

# import some common libraries
import os
import glob
import os.path
import numpy as np
import json, random, cv2
from cellpose import models
from cellpose.io import imread
import matplotlib.pyplot as plt

from tqdm import tqdm
from pathlib import Path
from skimage import measure
from PIL import Image, ImageSequence

# from livecellx import core
from livecellx import core
from livecellx.core import datasets
from livecellx.core import SingleCellTrajectory, SingleCellStatic
from livecellx.core.datasets import LiveCellImageDataset, SingleImageDataset

2025-04-13 17:22:20.256829: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2025-04-13 17:22:20.256857: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2025-04-13 17:22:20.257925: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-04-13 17:22:20.262883: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-04-13 17:22:20.824490: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT

Loading single cells from sample data

LiveCellImageDataset, SingleCellImageDataset from livecellx.core.datasets allow users to load images with ease, without reading directly into memories. Both of two classes are designed to help manage and interact with datasets of live cell images. SingleCellImageDataset takes a single image from the memory and makes it a single time point dataset, which can be handy when you would like to process imaging datasets.

Now, we download sample datasets and prepare the single cell datasets from the mask and DIC datasets. This is the step where we convert our image data into a format that’s easy to work with for segmenting and tracking single-cells.

from skimage.measure import regionprops
from livecellx.core.io_sc import prep_scs_from_mask_dataset
from livecellx.sample_data import tutorial_three_image_sys

dic_dataset, mask_dataset = tutorial_three_image_sys()
single_cells = prep_scs_from_mask_dataset(mask_dataset, dic_dataset)

|-----> Downloading data to datasets/test_data_STAV-A549.zip
|-----> Data already exists at datasets/test_data_STAV-A549.zip
|-----> Extracting data to datasets
3 png img file paths loaded;
3 tif img file paths loaded;

100%|██████████| 3/3 [00:00<00:00, 12.01it/s]

We want to ensure that the preparation of single cell datasets was successful, and each dataset has the associated mask data.

for sc in single_cells:
    assert sc.mask_dataset

Check the total number of single cell datasets prepared.

len(single_cells)

42

Organizing single cells by timeframe

We aim to organize the single cell datasets based on the timeframes they belong to.

single_cells_by_time = {}
for cell in single_cells:
    if cell.timeframe not in single_cells_by_time:
        single_cells_by_time[cell.timeframe] = []
    single_cells_by_time[cell.timeframe].append(cell)

Now, let’s have a confirmation on how many single cell datasets we have for each timeframe. This provides a quick overview of the data distribution across time.

for time in single_cells_by_time:
    print(time, len(single_cells_by_time[time]))

0 13
2 15
1 14

Visualizing one single cell

To better understand the single cell data we have, let’s visualize one of the single cells using different representations.

sc = single_cells[1]

#fig, axes = plt.subplots(1, 4, figsize=(10, 5))
#sc.show(ax=axes[0])  # show raw image
#sc.show_mask(ax=axes[1])  # show mask
#sc.show_contour_img(ax=axes[2])  # show contour image
#sc.show_contour_mask(ax=axes[3])  # show contour mask

Compact Visualization using show_panel to visualize all the above representations and cropped images in a single call.

sc.show_panel(padding=50, figsize=(15, 5))

array([<Axes: title={'center': 'img'}>, <Axes: title={'center': 'mask'}>,
       <Axes: title={'center': 'contour_img'}>,
       <Axes: title={'center': 'contour_mask'}>,
       <Axes: title={'center': 'label_crop'}>,
       <Axes: title={'center': 'img_crop'}>], dtype=object)

Preprocessing cell image

In this block, preprocessing is performed on the cell image to ensure better accuracy and effectiveness in further analysis. This involves normalizing the original image (normalize_img_to_uint8), correcting the background using two different algorithms (correct_background_bisplrep and correct_background_median_gamma), and ensuring the image data is suitable for visualization and analysis.

import scipy
import numpy as np
from skimage import morphology
import scipy.interpolate as interp

from livecellx.preprocess.utils import normalize_img_to_uint8
from livecellx.preprocess.correct_bg import correct_background_bisplrep, correct_background_median_gamma


# Selecting a single cell from the dataset
sc = single_cells[0].copy()

padding_size = 30
# Showing the cropped image of the single cell
# sc.show(crop=True)
# Obtaining the cropped image of the single cell
sc_img = sc.get_img_crop(padding=padding_size)
original_sc = sc_img

# Normalizing the image and correcting the background using the B-spline representation method
bisplrep_sc_img = normalize_img_to_uint8(sc_img)
bisplrep_sc_img = correct_background_bisplrep(sc_img, sample_step=5, s=1e20)

# Normalizing the image and correcting the background using median gamma correction method
gamma_sc_img = sc.get_img_crop(padding=padding_size)
gamma_sc_img = normalize_img_to_uint8(sc_img)
gamma_sc_img = correct_background_median_gamma(sc_img, disk_size=2)

/home/ken67/anaconda3/envs/livecellx/lib/python3.9/site-packages/skimage/filters/rank/generic.py:268: UserWarning: Bad rank filter performance is expected due to a large number of bins (34545), equivalent to an approximate bitdepth of 15.1.
  image, footprint, out, mask, n_bins = _preprocess_input(image, footprint,

Visualizing background correction results

This block visualizes the results of the background correction to provide a clear comparison of the original image and the corrected images using two different background correction algorithms.

from mpl_toolkits.axes_grid1 import make_axes_locatable
def add_colorbar(im, ax, fig):
    divider = make_axes_locatable(ax)
    cax = divider.append_axes("right", size="3%", pad=0.05)
    fig.colorbar(im, cax=cax, orientation="vertical")

fig, axes = plt.subplots(1, 2, figsize=(15, 5))
im = axes[0].imshow(original_sc)
# show colorbar
add_colorbar(im, axes[0], fig)
axes[0].set_title("original")

im = axes[1].imshow(gamma_sc_img)
add_colorbar(im, axes[1], fig)
axes[1].set_title("corrected: gamma correction")

Text(0.5, 1.0, 'corrected: gamma correction')

We select Cells for Analysis

sc1 = single_cells[1]
sc2 = single_cells[2]

Calculating cell features

Now we’ll calculate a variety of features for sc1.

from livecellx.trajectory.feature_extractors import compute_skimage_regionprops, compute_haralick_features

skimage_features = compute_skimage_regionprops(sc1)
sc1.add_feature("skimage", skimage_features)

# Optionally, you can also calculate Haralick texture features for the cell
# Uncomment the below lines to calculate Haralick features

# haralick_features = compute_haralick_features(sc1)
# sc1.add_feature("haralick", haralick_features)

These features will provide quantitative information about the cell which could be used in many ways, including in tracking and analysis tasks.

sc1.get_feature_pd_series()

skimage_area                                               9375.0
skimage_area_bbox                                         20916.0
skimage_area_convex                                        9728.0
skimage_area_filled                                        9375.0
skimage_axis_major_length                              199.544403
                                             ...
skimage_perimeter                                      472.457936
skimage_perimeter_crofton                              450.599728
skimage_solidity                                         0.963713
t                                                             0.0
sc_id                        80188ade-4049-45bc-bacc-70415d188a6f
Length: 145, dtype: object

We show bounding box of the cell

sc2.bbox

array([ 400, 1158,  479, 1269])

We calculate overlap between two single cells

sc1.compute_iou(sc2), sc1.compute_overlap_percent(sc2)

(0.0, 0.0)

Tracking based on single cells

In this section, we leverage the SORT (Simple, Online and Realtime Tracking) algorithm to track cells across different frames based on their bounding boxes. This helps in understanding the movement and transformation of cells over time.

from typing import List
from livecellx.track.sort_tracker_utils import (
    gen_SORT_detections_input_from_contours,
    update_traj_collection_by_SORT_tracker_detection,
    track_SORT_bbox_from_contours,
    track_SORT_bbox_from_scs
)


sctc = track_SORT_bbox_from_scs(single_cells, dic_dataset, mask_dataset=mask_dataset, max_age=0, min_hits=0)

SORT tracking...: 100%|██████████| 3/3 [00:00<00:00, 189.30it/s]

Generating movies

Now that we have tracked the cells, we can generate movies to visualize the tracking process. This block also checks for any cells that were discarded or newly detected during the tracking process which can be informative for understanding the tracking performance and cellular dynamics.

new_scs = []
for _, sct in sctc:
    new_scs.extend(sct.get_all_scs())
new_scs_ids = {sc.id for sc in new_scs}
old_scs_ids = {sc.id for sc in single_cells}
for sc in single_cells:
    if not (sc.id in new_scs_ids):
        print("sc:", sc.id, "is discarded in the tracking process")
        # sc.show_panel()

for sc in new_scs:
    sc_id = sc.id
    if not (sc_id in old_scs_ids):
        print("sc:", sc_id, "is a new cell at time", sc.timeframe)
        # sc.show_panel()

We create directory to store movies

!mkdir -p "./notebook_results/general_tutorial/track_movies/"

Having processed the tracking data, generating trajectory movies aids in understanding cell movement and behavior over time. In this block, we generate and save a movie for each cell trajectory using the generate_single_trajectory_movie function.

for track_id, sct in sctc:
    sct.show_on_grid(nr=1, nc=3, enhance_contrast_factor=2, cmap="viridis", dims=[200, 300])
    sct.show_on_grid(nr=1, nc=3, enhance_contrast_factor=2, cmap="viridis")
    break

from livecellx.track.movie import generate_single_trajectory_movie

for track_id, traj in sctc:
    generate_single_trajectory_movie(traj, save_path=f"./notebook_results/general_tutorial/track_movies/track_{track_id}.gif");

Trajectory Length Histogram

sctc.histogram_traj_length()

<Axes: xlabel='Trajectory Length', ylabel='count'>

Visualizing first frame of each trajectory

By visualizing the first frame of each trajectory, we get a glimpse of the initial state of each cell.

for traj in sctc.get_all_trajectories()[0:2]:
    track_id = traj.track_id
    print("track_id=", track_id)
    traj.timeframe_to_single_cell[list(traj.timeframe_to_single_cell.keys())[0]].show_panel(padding=15, figsize=(20, 5))
    plt.show()

track_id= 13.0

track_id= 12.0

The Kernel crashed while executing code in the current cell or a previous cell.

Please review the code in the cell(s) to identify a possible cause of the failure.

Click <a href='https://aka.ms/vscodeJupyterKernelCrash'>here</a> for more info.

View Jupyter <a href='command:jupyter.viewOutput'>log</a> for further details.

import necessary library for the napari viewer

%gui qt
from skimage import data
from livecellx.core.napari_visualizer import NapariVisualizer

import numpy as np
from napari.viewer import Viewer
from livecellx.plot.visualizer import Visualizer
from livecellx.core.single_cell import SingleCellStatic, SingleCellTrajectory, SingleCellTrajectoryCollection

Manually or programmatically correct trajectories via SCTOperator

In this part, we are setting up Napari to visualize the DIC images alongside the tracked cell trajectories.

import napari
from livecellx.core.sct_operator import SctOperator, create_sct_napari_ui, create_scs_edit_viewer, create_scts_operator_viewer
viewer = napari.view_image(dic_dataset.to_dask(), name="dic_image", cache=True)
shape_layer = NapariVisualizer.gen_trajectories_shapes(sctc, viewer, contour_sample_num=20)
shape_layer.mode = "select"

If you would like to apply our correct segmentation model, you need to load a model.

from livecellx.core.sc_seg_operator import ScSegOperator, create_sc_seg_napari_ui
# # replace the path below with your downdloaded CSN checkpoint path
# ckpt_path = r"./notebook_results/csn_models/model_v11_epoch=3282-test_loss=2.3688.ckpt"
# ScSegOperator.load_default_csn_model(path=ckpt_path, cuda=True);

sct_operator = SctOperator(sctc, shape_layer, viewer)
# sct_operator.setup_shape_layer(shape_layer)
create_sct_napari_ui(sct_operator)

sct_operator.select_shape(None)

Select the cell you want to correct and operate. Below picture shows the cell with index 14 selected connects with trajectory 2 and becomes part of it after changing to mode connect.

Manually call methods of sct_operator to connect/disconnect/add relations

# sct_operator.connect_two_scts()
# sct_operator.disconnect_sct()
# sct_operator.add_mother_daughter_relation()

After change to mode disconnect, select on the cell you want to disconnect. Below picture shows the cell with index 14 selected disconnects with trajectory 2 and becomes a new trajectory.

Delete something in Napari and run the following block to observe if the corresponding keys have been deleted from the trajectory collection

sct_operator.traj_collection.track_id_to_trajectory.keys()

sc_operator = sct_operator.edit_selected_sc()

The right bottom panel Sc Operator shows operation you can adopt to edit a single cell.

cur_properties = sct_operator.shape_layer.current_properties
cur_properties["sc"]
cur_properties["sc"][0].show_panel()

We can check the current number of single cells and cells selected

len(sct_operator.get_all_scs()), len(sct_operator.select_info)

If there are any selected single cell trajectories (SCTs) intend to be removed, this can be done using the delete_selected_sct() method.

sct_operator.delete_selected_sct()