Annotation by clicking single cells in the Napari UI

In this tutorial, we will explore the process of annotating single cells within the Napari User Interface (UI), which is an essential step to leverage machine learning algorithms for further analysis, such as mitosis detection.

# import some common libraries
import os
import glob
import os.path
import numpy as np
import json, random, cv2
from cellpose import models
from import imread
import matplotlib.pyplot as plt

from tqdm import tqdm
from pathlib import Path
from skimage import measure
from PIL import Image, ImageSequence

# from livecellx import segment
from livecellx import core
from livecellx.core import datasets
from livecellx.core import SingleCellTrajectory, SingleCellStatic
from livecellx.core.datasets import LiveCellImageDataset, SingleImageDataset
from livecellx.sample_data import tutorial_three_image_sys
import napari
napari.__version__  # should be 0.4.10 or greater

Loading data

Before the annotation process, it’s essential to load the data - either from pre-prepared masks(prep_scs_from_mask_dataset) or a specific JSON file containing single cell information. We showcase two examples: how to load data from a particular dataset named EBSS data, 24 hours and how to load a sample tutorial 3-image dataset.

In the first example (EBSS data), we define the paths for our dataset and mask dataset. We then create a LiveCellImageDataset object for the mask data and a dictionary time2url to map indices to the file paths of the DIC images. We again use LiveCellImageDataset to hold our DIC dataset.

EBSS data, 24 hours

# dataset_dir_path = Path(
#     "../datasets/EBSS_Starvation/tif_STAV-A549_VIM_24hours_NoTreat_NA_YL_Ti2e_2022-12-21/XY16/"
# )

# mask_dataset_path = Path("../datasets/EBSS_Starvation/tif_STAV-A549_VIM_24hours_NoTreat_NA_YL_Ti2e_2022-12-21/out/XY16/seg")

# mask_dataset = LiveCellImageDataset(mask_dataset_path, ext="png")
# time2url = sorted(glob.glob(str((Path(dataset_dir_path) / Path("*_DIC.tif")))))
# time2url = {i: path for i, path in enumerate(time2url)}
# dic_dataset = LiveCellImageDataset(time2url=time2url, ext="tif")

Loading sample tutorial 3-image data

In the second example, we call tutorial_three_image_sys to get the DIC and mask datasets. We then use prep_scs_from_mask_dataset to prepare the single cell data.

from livecellx.core.io_sc import prep_scs_from_mask_dataset

dic_dataset, mask_dataset = tutorial_three_image_sys()
single_cells = prep_scs_from_mask_dataset(mask_dataset, dic_dataset)
|-----> Downloading data to datasets\
|-----> Data already exists at datasets\
|-----> Extracting data to datasets
3 png img file paths loaded;
3 tif img file paths loaded;
100%|██████████| 3/3 [00:08<00:00,  2.95s/it]

Loading your own dataset

The last commented line is a placeholder for loading your own dataset using LiveCellImageDataset.

# dic_dataset = LiveCellImageDataset(...)
Skip if you do not plan to apply CSN
load CSN model if you would like to use it to correct segmentation
# from livecellx.core.sc_seg_operator import ScSegOperator, create_sc_seg_napari_ui

# replace the path below with your downdloaded CSN checkpoint path
# ckpt_path = r"./notebook_results/csn_models/model_v11_epoch=3282-test_loss=2.3688.ckpt"
# ScSegOperator.load_default_csn_model(path=ckpt_path, cuda=True);

Annotating by clicking single cells in Napari

Napari provides an intuitive interface that allows you to manually segment and annotate cells within images. In this part, we will delve into the process of using the Napari UI for single-cell annotation.

from livecellx.core.sct_operator import create_scs_edit_viewer, SctOperator, create_scs_edit_viewer_by_interval, _get_viewer_sct_operator
from livecellx.core.single_cell import create_sctc_from_scs, filter_sctc_by_time_span

Manual segmentation and annotation

You may or may not have segmentation ready. If you do not have segmentation masks, you may do annotation manually by starting with an empty list of single cell objects. All you need to do is to load an LiveCellImageDataset object.

Setting up the annotation interface

The function create_scs_edit_viewer_by_interval facilitates the loading of shapes within a specified time window, ranging from t to t + interval. During the annotation process, you can:

Press n to move the window forward. Press b to move the window backward. Press m to load from your current step slice.

# sct_operator = create_scs_edit_viewer(single_cells, img_dataset = dic_dataset)
span_interval = 10
# For pre-segmented data:
# viewer = create_scs_edit_viewer_by_interval(single_cells, img_dataset=dic_dataset, span_interval = 10, viewer = None)

# For manual segmentation (when no segmentation masks are available):
viewer = create_scs_edit_viewer_by_interval([], img_dataset=dic_dataset, span_interval = 10, viewer = None)
|-----> A new SCTC object with size 0 is created by subsetting the original sctc with time span (0, 10)
c:\Users\13290\AppData\Local\Programs\Python\Python39\lib\site-packages\napari\utils\colormaps\ UserWarning: Given color input is empty. Converting input to a white color array.


[ ]:
Run all stop here!

Validating the Annotations

After manual annotation, it’s vital to ensure that your annotations are saved correctly and can be accessed programmatically. Here, we’re retrieving the Single Cell Trajectory (SCT) operator from the viewer, which contains all the annotation information.

sct_operator = _get_viewer_sct_operator(viewer)

If you manually annotate, you shall add mask first. Once you finish drawing mask, you need to save it. You finnally click clear sc operator on SCT Operator panel: