26 July 2021 Update
We are pleased to announce some recent accomplishments of the High-throughput truthing project (HTT) and request your assistance to maintain progress.
The HTT project’s objective is to create a validation dataset fit for a regulatory purpose. Pursuing this objective is expected to inform regulatory frameworks and be instructive to others to develop their own validation datasets. For more detailed information about the project and/or to register as a pathologist-data-collector, please visit https://ncihub.org/groups/eedapstudies/.
Please contact Katherine Elfer (Katherine.Elfer@fda.hhs.gov), Project Coordinator, or Brandon Gallas (Brandon.Gallas@fda.hhs.gov), Project Lead, with any questions or to join our efforts.
Upcoming activities
8/6/2021: Discussion about patient sampling for HTT pivotal study
- Lead: Victor Garcia is a new FDA fellow with an MD (board ready) and a Clinical Informatics Fellowship.
- Contact Katherine Elfer or Brandon Gallas to join the conversation
8/8/2021: Presentation at the American Statistical Society Joint Statistical Meetings
- “Pathologist Agreement from Quantitative Measurements: a Pilot Study”
- In coordination with this presentation and the corresponding proceedings paper, we intend to make public a GitHub repository (name “HTT”) with an R data package containing the pathologist annotations.
- LINK to Full Presentation Information
- LINK to Conference Program
8/12/2021: FDA Annual Summer Student Scientific Poster Day
- 9:30 – 11:30am EDT
- Two Posters from our summer interns:
- Title 1: Developing a controlled workflow for clinical studies to collect pathologist annotations
- Title 2: Communication Methods and Workflows to Develop and Implement High Throughput Truthing of Pathologist Annotations as a Reference Standard for Validating Artificial Intelligence in Digital Pathology.
- Link to full abstracts
- Link to Register for full FDA Webcast & Poster Event
Summer 2021-Fall 2021: Recruiting Pathologists for on-site Data Collection
- We are looking for pathologists to provide annotations using the eeDAP system
- Data Collection Sites; Yale New Haven Land Stony Brook Medicine (early Fall 2021)
- LINK to full recruitment and information
Recent accomplishments
7/2021: Statistical Methods Presentation
- Si Wen, Statistician FDA/CDRH/OSEL/Division of Imaging, Diagnostics, and Software Reliability
- Title: Quantitative Agreement Analysis for HTT Pilot Study Data
- Presentation video and GitHub repo with methods coming and will be linked HERE
- Related manuscript: Preprint on arXiv – 2107.08891. It has been submitted to Statistics in Biopharmaceutical Research.
7/2021: Shipped eeDAP system to Yale for microscope-mode data collection
- eeDAP Github Repo
- Looking for pathologists to provide annotations
- Upgraded eeDAP hardware for speed
- Resolved third-party software version conflicts (Bioformats, Java, and Matlab – https://github.com/DIDSR/eeDAP/issues/89 )