Review model suggestions on 3D scans.
Keep every decision on record.

1. 

   01 Scan slice

   Axial slice 300 of the real volume, windowed: the model's input, with the one box it was given.
2. 

   02 Model features

   How this model layer was computed

   SAM 2.1's image encoder turns the slice into 256 channels on a 64 x 64 grid. This is their first principal component (17% of the variance), at its native 16 px blocks. A projection, not a layer.

3. 

   03 Model scores

   How this model layer was computed

   The mask decoder's output, -25 to +8. Logits are not probabilities.

4. 

   04 Model proposal

   Where the logits are above zero: SAM 2.1's outline of the bottle wall, dashed.
5. 

   148146140139

   05 Published label

   Instance 138, the pressure bottle, as published by the dataset's annotators (solid), with neighbouring parts' published instance outlines (thin). IoU with the proposal: 0.82.
   Labels on the picture: 148 · 146 · 140 · 139

6. Worked history on public scans. Names, times and version IDs are examples.

   • Aerospace · V5scan fba9a5cfSAM 2.1 proposal 68d34799, on its own branchpublished label 138 1c7047f3, merged
   • dataset main, tag dv-sample-01, read by the training input preview

   06 Record

   The history for this scan: the proposal stays on its own branch, each published human label joins the dataset line on its own, and a tag pins the version a training job reads.

CT and instance masks: Gruber et al., Fraunhofer EZRT, doi 10.5281/zenodo.10651746, CC BY 4.0 (opens in a new tab). Model: SAM 2.1 hiera-tiny, run by Nitsor outside the product. Modified by Nitsor: windowed, cropped, outlines and model outputs drawn over it.

1. 

   01 Scan slice

   Slice 64 of a public chest CT, lung window. The same patient the four radiologists outlined. The box given to the model is the readers' combined extent plus 3 mm, derived from their outlines.
2. 

   02 Model features

   How this model layer was computed

   SAM 2.1's image encoder, first principal component of 256 channels (25% of the variance), native 16 px blocks.

3. 

   03 Model scores

   How this model layer was computed

   SAM 2.1's decoder output for one box around the nodule. Logits, not probabilities.

4. 

   04 Lung lobes, a second model

   How this model layer was computed

   A second model, a different task: lung lobes. TotalSegmentator's softmax for the class 'left lower lobe' on the same field, per voxel. It does not look for the nodule.

5. 

   05 Model proposal

   SAM 2.1's outline of the nodule, dashed. IoU with the four readers: 0.79, 0.80, 0.84, 0.84.
6. 

   06 Four published outlines

   Four radiologists' outlines, kept separate, over their agreement (grey, 1 to 4). Longest diameter of the 2-of-4 consensus: 32.8 mm, derived from the outlines.

7. Worked history on public scans. Names, times and version IDs are examples.

   • Healthcare · LIDC-0003scan 58628ab1SAM 2.1 proposal 96cab4b8, on its own branchreader 1 aa3e882e, mergedreader 2 9e9a9479, mergedreader 3 55e99aaa, mergedreader 4 1a30ac07, merged
   • dataset main, tag dv-sample-01, read by the training input preview

   07 Record

   The history for this scan: the proposal stays on its own branch, each published human label joins the dataset line on its own, and a tag pins the version a training job reads.

32.8 mm, longest diameter of the 2-of-4 consensus, derived from the published outlines

CT and reader outlines: Armato et al., "Data From LIDC-IDRI", TCIA, doi 10.7937/K9/TCIA.2015.LO9QL9SX, CC BY 3.0 (opens in a new tab). Models: SAM 2.1 hiera-tiny and TotalSegmentator v2, run by Nitsor outside the product. Modified by Nitsor: windowed, cropped, outlines and model outputs drawn over it.

1. 

   01 MRI slice

   A sagittal T2 image from SPIDER case 101, slice 8 across. The display keeps the scan's own voxel spacing; the image and the masks share one geometry.
2. 

   02 Published masks

   Solid white: the published vertebra (01 to 06) and disc (201 to 206) masks on this slice. The dataset made them semi-automatically, then its annotators reviewed and manually corrected them. The numbers count from the bottom up; they are not anatomical names.

Lumbar MRI and reference masks: SPIDER, van der Graaf et al., Radboud University Medical Center, doi 10.5281/zenodo.10159290, CC BY 4.0 (opens in a new tab). Modified by Nitsor: cropped, and the published masks drawn as outlines. No model output is drawn on this scan.

1. 

   01 Scan slice

   The equatorial slice of a real orange CT: rind, pulp segments and the central column.
2. 

   02 Model features

   How this model layer was computed

   SAM 2.1's image encoder, first principal component of 256 channels (32% of the variance), at its native blocks.

3. 

   03 Model scores

   How this model layer was computed

   The decoder's output for one box, -9 to +7. Logits are not probabilities.

4. 

   04 Model proposal

   SAM 2.1's recorded proposal, long dashes. It takes in the rind; the authors' scripted pulp mask stops inside it.
5. 

   05 Authors' scripted masks

   Short dashes: the pulp and oil-gland masks, the authors' scripted segmentation (thresholding and morphology, parameters set by hand for each fruit and checked by eye), shipped with the scans; no person drew them. IoU 0.76 compares SAM 2.1 with the scripted pulp mask, not with a human label.

Citrus CT and its scripted masks: Amézquita, Quigley, Ophelders, Seymour, Munch, Chitwood, doi 10.5061/dryad.34tmpg4n6, CC0 (opens in a new tab). Model: SAM 2.1 hiera-tiny, run by Nitsor outside the product. Modified by Nitsor: cropped, and SAM 2.1's proposal and the authors' scripted masks drawn as dashed outlines.