Skip to main content
The NCI Community Hub will be retiring in May 2024. For more information please visit the NCIHub Retirement Page:https://ncihub.cancer.gov/groups/ncihubshutdown/overview
close

Study FDA-MSKCC 14-head Microscope

by Brandon D. Gallas

On Nov. 3, 2017 Brandon Gallas and Qi Gong visited MSKCC to help Yukako Yagi conduct a study using eeDAP on a 14-head microscope. There were 11 pathologists participating. There were 40 regions of interest (ROIs) where they were asked to detect and mark the locations of mitotic figures. For each ROI, there were candidate mitotic figures identified during a preliminary study. There were 128 candidate mitotic figures in total. The participants were asked to classify these candidates as mitotic figures or not. The study was completed in 2 hours!

The counts and classifications were done without discussion under instructions to make decisions independently, and the classifications were confidence scores as opposed to binary yes/no classifications: a 0-100 scale, threshold=50. We believe that there is more information in multi-level scores that can lead to smaller studies or better statistical power. Most of the candidate MFs were identified during a preliminary image curation study where 4 pathologists counted and marked the locations of MFs arising from the same 40 ROIs. We call them “candidate” MFs because only 21 of 92 were unanimously identified. The distribution of agreement results across the candidates reflects a spectrum of candidates from obvious and likely MFs to subtle and improbable MFs. In fact, in order to include candidate MFs with even lower pathologist agreement, we asked one of the image curation pathologists to re-evaluate 12 of the original 40 ROIs. These ROIs had only one candidate or none identified during the original image curation. For this re-evaluation we asked the image curation pathologist to lower his threshold for identifying MFs. He identified 36 more candidates. This spectrum allows an efficient and effective evaluation of an AI algorithm.

  • Here is a picture of all the pathologists working on the 14-head microscope
  • Here is an abstract submitted (but not accepted) to the USCAP 2018 annual meeting. It outlines the basics: the preliminary study and the 14-head study. USCAP2018abstractGallas3-submitted.docx (7 MB, uploaded by Brandon D. Gallas 6 years 4 months ago)
  • Here is a summary of the study written for the participants. FDA-MSKCCmultiHead-2.docx (25 KB, uploaded by Brandon D. Gallas 6 years 4 months ago)
  • Here is the case report form. We used paper this time, hoping to have a cloud solution next time. caseReportForm-v2.docx (6 MB, uploaded by Brandon D. Gallas 6 years 4 months ago)
  • Here are quick-access instructions for running the preliminary study instructions_mcsMSKCC.docx (2 MB, uploaded by Brandon D. Gallas 6 years 4 months ago)
  • Here are instructions for running eeDAP with ImageScope to collect image annotations saveImageScopeAnnotationsTaskWorkflow.docx (17 KB, uploaded by Brandon D. Gallas 6 years 4 months ago)
  • Here is the definition of a mitotic figure and the region of interest used for the studies mitosisDefinition.docx (67 KB, uploaded by Brandon D. Gallas 6 years 4 months ago)
  • Here is the observer consent form for the study observerConsentForm_mcsMSKCC-2.docx (28 KB, uploaded by Brandon D. Gallas 6 years 4 months ago)
  • Here is the observer registration form [File

mskcc14headStudy.png

Created on , Last modified on