Back to Presentations and Publications Main Page

Pathology Informatics Summit 2022

Garcia_PI_Summit_2022_Slides_-_20220506.pdf (3 MB, uploaded by Victor Garcia 1 year 11 months ago)

Introduction

The High Throughput Truthing project aims to develop a dataset of stromal tumor-infiltrating lymphocytes (sTIL) density estimates in hematoxylin and eosin stained invasive breast cancer biopsy specimens fit for a regulatory purpose 1,2. The dataset is derived from a pilot study of 640 unique regions of interest (ROIs) derived from 64 whole slide images. After collecting annotations for our pilot study, we observed greater interobserver variability among sTIL density evaluations than desired. To improve annotation quality, we used the pilot data to create custom training materials.

Method

We categorized ROIs based on their mean sTIL density as “10% or less”, “11% to 40%”, “greater than 40%”. We selected 72 unique ROIs from those with the highest and lowest variance. Each ROI was discussed in a group setting by at least three members of our expert panel, one clinical scientist and seven board-certified pathologists who are project collaborators and trained in sTIL scoring. In a series of eight one-hour sessions, our panel reviewed each ROI and provided verbal estimates of stromal percentage and sTIL density, and comments on features that confound the sTIL assessment. We then compiled experts’ scores and comments into a reference document.

Results

The expert discussions found a need for additional instructions on sTIL assessment. Challenges were encountered for both stromal percentage and sTIL density assessment. These include evaluating the stromal percentage with respect to the whole ROI, distinguishing fibroblasts and degenerated tumor cell nuclei from lymphocytes, and excluding adipose tissue as tumor-associated stroma.

Conclusions

Our approach demonstrates a method by which training materials can be developed using a focus group approach based on pilot study data. From selected ROIs, we will create two data sets: a training set and a proficiency test set. The training set offers participants real-time, descriptive and quantitative feedback based on our compiled expert commentary to solidify their sTIL density knowledge, while the proficiency test set requires participants to demonstrate their competency in sTIL assessment before proceeding with data annotation. These materials will train crowd-sourced pathologists to help create a large algorithm validation dataset, while also improving sTIL evaluation in clinical practice.

Victor Garcia
- Division of Imaging, Diagnostics, and Software Reliability, OSEL/CDRH/FDA, White Oak, MD
Katherine Elfer
- Division of Imaging, Diagnostics, and Software Reliability, OSEL/CDRH/FDA, White Oak, MD
Anna Ehinger
- Department of Genetics and Pathology, Laboratory Medicine Region Skåne, Lund, Sweden; Department of Clinical Sciences Lund, Division of Oncology, Lund University, Lund, Sweden
Dieter Peeters
- Department of Pathology, Sint-Maarten Hospital, Mechelen, Belgium; Department of Biomedical Sciences, University of Antwerp, Antwerp, Belgium
Xiaoxian Li
- Department of Pathology and Laboratory Medicine, Emory University, Atlanta, GA
Amy Ly
- Department of Pathology, Massachusetts General Hospital, Boston, MA
Bruce Werness
- Inova Health System Department of Pathology, Falls Church, VA; Arrivebio LLC, San Francisco, CA
Matthew Hanna
- Memorial Sloan Kettering Cancer Center, New York
Kim Blenman
- Department of Internal Medicine, Section of Medical Oncology, School of Medicine and Yale Cancer Center, and Department of Computer Science, School of Engineering and Applied Sciences, Yale University, New Haven, CT
Roberto Salgado
- Division of Research, Peter Mac Callum Cancer Centre, Melbourne, Australia; Department of Pathology, GZA-ZNA Hospitals, Antwerp, Belgium
Brandon D. Gallas
- Division of Imaging, Diagnostics, and Software Reliability, OSEL/CDRH/FDA, White Oak, MD

Created on 13 Apr 2022, Last modified on 30 Dec 2022