==25 August 2021 Update== ~~26 July 2021 Update~~ We are pleased to announce some recent accomplishments of the High-throughput truthing project (HTT). The HTT project’s objective is to create a validation dataset fit for a regulatory purpose. Pursuing this objective is expected to inform regulatory frameworks and be instructive to others to develop their own validation datasets. For more detailed information about the project and/or to register as a pathologist-data-collector, please visit [https://ncihub.org/groups/eedapstudies/]. Please contact Katherine Elfer (Katherine.Elfer@fda.hhs.gov), Project Coordinator, or Brandon Gallas (Brandon.Gallas@fda.hhs.gov), Project Lead, with any questions or to join our efforts. ---- =Upcoming activities= ====9/3/2021: Discussion about patient sampling for HTT pivotal study and CME Training==== * Lead: Victor Garcia, MD, is a new FDA fellow, board certified in internal medicine and recently completed a Clinical Informatics Fellowship. ====Summer 2021-Fall 2021: Recruiting Pathologists for on-site Data Collection==== * We are looking for pathologists to provide annotations using the eeDAP system * Data Collection Sites; Yale New Haven * [HTTrecruiting LINK to full recruitment and information] ---- =Recent accomplishments= ====8/19/2021: FDA Summer Intern Festival - Google Summer of Code Video ==== * 2021 Google Summer of Code Intern Peddi Sai Varshith recorded a video of his work with the High Truthing Project and integrating hardware with caMicroscope * Video and Project Summary: [https://ncihub.org/groups/eedapstudies/wiki/Presentation2021:PeddiSaiVarshith:caMicroscopeHardwareIntegrationup HERE] ====8/12/2021: FDA Annual Summer Student Scientific Poster Day==== * 9:30 - 11:30am EDT * Two Posters from our summer interns: * Title 1: Developing a controlled workflow for clinical studies to collect pathologist annotations * Title 2: Communication Methods and Workflows to Develop and Implement High Throughput Truthing of Pathologist Annotations as a Reference Standard for Validating Artificial Intelligence in Digital Pathology. * [Presentation:ORISESummerStudentScientificPosterDay2021 Link to posters and full abstracts] *[https://collaborationcqpub1.fda.gov/content/connect/c1/7/en/events/event/shared/default_template/event_registration.html?sco-id=241882471 Link to Register for full FDA Webcast & Poster Event] ====8/8/2021: Public Release of the HTT !Github Repository==== * Contains primary data objects (with documentation) and supporting functions * Contains Pathology Informatics R markdown file with code to process data and create all figures * Repository: [https://github.com/DIDSR/HTT] ====8/8/2021: Presentation at the American Statistical Society Joint Statistical Meetings==== * “Pathologist Agreement from Quantitative Measurements: a Pilot Study” * In coordination with this presentation and the corresponding proceedings paper, we intend to make public a !GitHub repository (name "HTT") with an R data package containing the pathologist annotations. * [Presentation:JSMPathologistAgreementfromQuantitativeMeasurements:aPilotStudy LINK to Full Recordings and Abstract] * [https://ww2.amstat.org/meetings/jsm/2021/onlineprogram/AbstractDetails.cfm?abstractid=318980 LINK to Conference Program] ====7/2021: Statistical Methods Presentation==== * Si Wen, Statistician FDA/CDRH/OSEL/Division of Imaging, Diagnostics, and Software Reliability * Title: Quantitative Agreement Analysis for HTT Pilot Study Data * [Presentation:WenQuantitativeAgreementAnalysisforHTTPilotStudyData Watch HERE] * !GitHub repository with methods: [https://github.com/DIDSR/ANOVA.MRMC.LOA] * Related manuscript: Preprint on [https://arxiv.org/abs/2107.08891 arXiv - 2107.08891]. It has been submitted to Statistics in Biopharmaceutical Research. ====7/2021: Shipped eeDAP system to Yale for microscope-mode data collection==== * [https://github.com/DIDSR/eeDAP eeDAP Github Repo] * Microscope professionally cleaned and ready for data collection * Looking for pathologists to provide annotations * Upgraded eeDAP hardware for speed * Resolved third-party software version conflicts (Bioformats, Java, and Matlab - [https://github.com/DIDSR/eeDAP/issues/89] ) ====7/2021: We have executed two research collaboration agreements (RCA) for slide sourcing.==== * We are looking for at least one other US site for slide sourcing * We have an established RCA template * [HTTAdminRCA:ResearchCollaborationAgreements LINK to RCA Abstracts and Template] ====6/2021: Hired five summer research assistants to support related research==== * Victor Garcia, MD MS: Stony Brook Medicine * Sarah Dudgeon, MPH (back to join us for data collection!): CORE Center for Computational Health New Haven Hospital * John Paul (JP) Philips: University of Chicago, Dept. of Physics * Phoebe Qian: Mc.Clean High School, VA * James Setty: Wall High School, D.C.