NA-MIC Project Weeks
Back to Projects List
Development of Deep Learning Segmentation for Spines with Metastatic Disease
Key Investigators
- Ron Alkalay (Beth Israel Deaconess Medical Center)
- Curtis Lisle (KnowledgeVis,LLC)
- Andres Diaz-Pinto (Kings College Longon)
- Andras Lasso (Queens University)
Project Description
We have labeled CT data sets for lumbar and/or thoracic as well as full spine columns for patients at baseline. For a good # of patients, we also have 3 and 6m follow-up CT, but these are not yet labeled. It would be great to get some help/advice regarding how to speed up the segmentation for the labeling and extraction of volume information from the masks. The segmented volumes are needed for the analytical and computational modeling pipeline as part of a collaboration with MIT. Lytic spines present a special segmentation challenge because the interior of vertibrae have degraded and appear like surrounding tissue in CT scans.
Objective
- Objective A. Setup MONAI-based neural network training on the labeled scans.
- Objective B. Investigate using MONAILabel as a possibly better way to annotate new scans.
- Objective C. Train a DNN on the labeled data. Can this model be used in MONAILabel?
- Objective D. Find a path forward to better annotation using DL-assisted segmentation.
Approach and Plan
- Start with publically available VerSe spine dataset.
- Evaluate MONAILabel from Andres using model trained on VerSe as a segmentation tool for these lytic spines. How good is segmentation?
- Consider fine-tuning model in MONAILabel with lytic spines from our dataset
- Train model on only our lytic spines for comparison
- Install MONAILabel on GPU hardware at BI to setup for better annotation after project week
Progress and Next Steps
- Received permission to share a few cases with collaborators
- Acquired GPU hardware. Evaluated installing Linux, WSL, and Conda-on-windows. Settled on Windows
- Andres Diaz-Pinto used the VerSe spine dataset to create a MONAILabel vertebrae segmentation app
- Curt was able to install and run MONAILabel locally
- Andras Lasso helped with a script to fill internal holes in the vertebrae
- Segmentation Editor performance is improved with the VTK9 version of Slicer and Andras engineering fixes
- Curt started preparing for ML training of a network using Ron’s improved annotations, but this is still in progress.
- Next Steps: Improve segmentations for the remaining annotated spines and use in MONAILabel.
- Next Steps: identify any other collaborators interested to continue working on this project; eventually expanding to include nearby muscle and disks
Spine segmentation protocol for training data generation
This protocol can be used to quickly and accurately segmentat the spine (each vertebra in a separate segment; no internal holes), which can be used as training data for deep learning based segmentation:
- Define a bone threshold using Threshold effect and use that as a mask (do not Apply the threshold). Choose a threshold value that only selects bones, not soft tissues. Cancellous bone inside the vertebrae will be missed, but those holes will be filled in a later step.
- Paint a seed in each vertebra with a different color in a sagittal slice. Paint seed in an axial slice for one of the vertebrae.
- Use “Grow from seeds effect” to segment the entire spine automatically. Adjust seeds as needed, then finalize the segmentation by clicking Apply.
- Fill holes inside the vertebrae by copy-pasting this script into the Python console.
Illustrations
Here is an example of how the degraded vertebrae look in CT scans and the challenges of segmentation in Slicer. Note the interior holes because of the heterogeneous nature of the lytic bone.
Because the bone in the spines is degraded from cancer, it is difficult to get a good segmentation. Here is the result of several hours of manual segmentation:
During the project week, we received consulting assistance on segmentation techniques from Andras Lasso and Rudolf Bumm. After Andras’ wrote us a post-processing script, the final result of registration is much better. These annotations can be used for neural net training:
Here is the MONAILabel server and Slicer Module running on Curt’s workstation. The app created annotations of vertebrae automatically created by a neural network trained using the VerSe spine dataset. This app works for interactively improving the segmentation through editing (using the deepedit algorithm):
Development was started for training a standalone neural network model, also using the MONAI framework. Here is a sample training image showing a portion of the spine vertibrae and the corresponding label going into the neural network for training. This shows why it was critical to achieve solid annotations for the interior portions of the vertebrae: