EM Mask Generation

Short overview of what this module does and links to usage.

• Install & Usage: See the module's README and scripts in the GitHub repository for the most up-to-date instructions.
• Design Choices: See Design Choices for the "why" behind the "what".

What is the EM Mask Generation module?

In this module, we generate EM masks that label which parts of a volume are brain and which parts are non-brain tissue. Think of it as a tissue-labelling step: we want a clean mask that says "this voxel belongs to brain or ventral nerve cord" vs "this is resin, trachea, or something we don’t plan to segment".

For now, the examples are based on Drosophila larva FIBSEM volumes (downsampled), but the ideas are general.

We work with two classes:

• Brain – any tissue that forms the brain or ventral nerve cord.
• Non-brain – resin, tracheal membranes, cutting artefacts, or any regions of the volume we do not want to segment further.

Figure 1. Brain vs non-brain mask generation in EM volumes. Top left: raw downsampled FIBSEM slice annotated with brain, resin, and trachea. Bottom left: classical DoG/texture-based masking, which separates biological tissue from background but also includes non-brain structures. Top right: manually drawn brain mask (ground truth). Bottom right: prediction from a simple 3D U-Net trained on these labels.

Why do we need these masks?

Most downstream steps in Catena (neuron segmentation, synapses, mitochondria, neurotransmitters, etc) only care about brain tissue. Resin and other non-brain structures just:

• waste GPU time and memory,
• could introduce false positives (e.g. “neuron segments” on resin),
• and make evaluation and proofreading more painful.

A good brain vs non-brain mask lets us:

• restrict all heavy models and post-processing to relevant voxels only,
• standardise crops and RoIs across modules,
• and quickly visualise which parts of a huge EM volume are actually “in play” for connectomics.

In short: these masks are a cheap but powerful way to ignore what we don’t care about.

How do we generate these masks?

We provide two ways to obtain a brain vs non-brain mask.

1. Conventional computer vision

As a lightweight, dependency-free baseline we start with classical CV:

• apply a Difference of Gaussians (DoG) filter, and
• run a local texture / structure analysis to build masks.

The pure DoG masks are not great on their own, but the texture-based masking can give a decent first separation of “background resin” from “biological tissue”. However, this approach cannot easily distinguish other non-brain structures such as tracheal membranes. It is a good starting point, but not the whole story.

Warning

The DoG/texture-based masks require manual finetuning of hyper-parameters such as block_size, texture_sigma, edge_sigma and minimum_region_size. Check design notes for more details

2. Machine learning

To go beyond that, we train a very simple 3D U-Net on a downsampled EM volume with semantic labels for brain and non-brain:

• implemented with MONAI,
• minimal data augmentation to encourage some generalisation to other EM datasets,
• trained in pixel space (we do not encode voxel size explicitly).

At inference, this model generalises well (though of course not perfectly) to completely unseen EM volumes. The predicted masks already do a good job at separating brain tissue from everything else and are much better at handling structures like tracheal membranes.

A post-processing step should follow after inference to:

• fill small gaps,
• slightly grow/erode the mask, and
• make sure it overlays cleanly on the EM volume.

An example post-processing script is provided in this module.