Oncology Software / Hardware Design
Digital Pathology
I remember the day I was assigned to the Research Platform (RP). I had just spent a year battling with our Service Platform developers over the implementation of Salesforce HealthCloud—a journey detailed elsewhere. By the time I transitioned to the RP, I was thrilled to discover it was entirely hand-rolled, supported by a team of top-notch creative developers.
John Philip, a 30-year veteran of MSK, research scientist, and software developer, onboarded me into Digital Pathology. He ended his pitch with a simple yet audacious statement: “Basically, we need to retire the microscope.”
The idea of retiring the microscope seemed so outrageous, I couldn’t wait to dive in.
One of MSK’s Histology Labs
Extracting a specimen in a Gross Lab
Over the next several sprints, I immersed myself in the world of histology and accessions, learned about the staggering number of digitizers MSK employs (more than any other research facility in the world), and conducted interviews with pathologists—both fellows and attendings.
One of my earliest requests to John was to introduce me to the “curmudgeons”—seasoned attendings who are notoriously resistant to technology.
When he hesitated, I assured him: “If I can convert them, we’ve done it.”
Key Insights from Pathologists
- A Deep Love for Microscopes: Most pathologists own high-end microscopes, with many keeping one at home—a seamless extension of their professional selves.
- Open to AI (with Conditions): Surprisingly, they were more receptive to AI analysis than expected but insisted on human verification for all results.
- Frustration with Logging In: They hate logging into multiple systems. If not at their personal terminal, it can take up to 20 minutes just to get set up.
- Blindfolded Gestures: For years, MSK researchers were forcing mice into the hands of Pathologists – when they really wanted to retain the learned body memory from years at the microscope
- Delegation of Accession Planning: Fellows handle accession planning, but attendings insist on reviewing all cases themselves.
- Reluctance to Let Fellows Use AI: Attendings strictly prohibit fellows from using AI tools, preferring to oversee and verify their work.
- Rapid Oscillation Between Roles: When reviewing cases initially diagnosed by fellows, attendings seamlessly shift between analysis and teaching, training their mental models in real-time.
- Technical Lag Times Are the Real Enemy: The curmudgeons weren’t opposed to digital implementations; they despised the technical lag. DICOM and other medical images, often 3GB in size, exacerbate this frustration.
- Apple Loyalty: Every pathologist I spoke to owned iPhones and Apple products—none used Android.
While these insights are just the surface, they provided critical direction for my designs and prototypes. Some of the most remarkable findings are proprietary and can’t be shared here, but they served as the driving force behind my approach.
Observing a signout with a leader in cancer research
Conclusion
Below are fragmented snapshots of the Design Thinking process that shaped Digital Pathology. They represent the journey of translating these insights into tangible, user-focused solutions. The ultimate goal was to ensure that the microscope’s digital successor was not just functional but embraced as an indispensable tool by even the most skeptical pathologists.
Analyzing Cerner’s CoPath
CoPath is a accession management system pathologist uses to record diagnosis. One of my first stops was to go through the application with an expert to understand the data points, their work environment, and look for pain.
Contextual Inquiry
All of the Pathologists were excited to be interviewed. I would select a day and spend 2 to 3 hours observing and recording. Then, I would go to a coffee shop document my thoughts and ideas. Here is a minute sample of the hundred or so hours of footage.
Research Analysis
I imported the recordings into Dovetail to quantify the observations using a rubric for intuition.
Alignment Artifacts
From the inteviews and observations, I crafted journeys and timelines – detailing every step of the process.
Sketches
While watching and documenting, I created sketches of the process, interactions, and hardware / software solutions.
Design and Redesign, and Prototype
I created a mi-fi composites in Figma using our MSK Design System. These were then delivered to the team for discussion. Again, and again, and again.
Design and Redesign, and Prototype
I created a mi-fi composites to prototype in Figma using our MSK Design System. I designed these for an iPad and tested them in situ with the fellows and attending pathologists.
Results
All the pathologists loved the iPad solution. It eliminated a need for multiple applications, the resolution was outstanding, and the interface was intuitive. The next step was for me to produce a augmented reality version – using a fabricated headset and their iPhones.
Unfortunately, the department couldn’t support this direction because of funding …
Pivot
Throughout this discussion, we became aware of a worklist application being developed by our colleague in another division. Ajaz had already created the backend data handshake with CoPath and was able to ingest information in realtime. We could also push the ultra-hi res images into the application – employing our AI analysis bots. He was close to being finished and he never considered doing something specifially for diagnosis.
It was a perfect fit.
I updated the designs and created an online prototype (using SalesForce’s Lightning Design System) … with robust Data Models!
A cancer patient’s journey starts with a diagnosis. This diagnosis is
many times done on a tissue embedded ina glass slide viewed in a microscope by
pathologists. I was asked to build a digital infrastructure to allow diagnosis using the
scanned image ofthe glass slide. John was a key driver of the final solution. He
interviewed 20+ people, ranging from pathologists to support staff. He created clear
clickable prototypes that we were able to test rapidly with the various staff. He also
created understandable wireframes that allowed our developers to build quickly. The
software is on pace for 100% adoption by end of 2025 with 100+ pathologists. With this
implementation, MSK will be able to eliminate most ofthe physical microscopes and
allow the pathologists to work remotely to render diagnosis on a traditional computer
screen. This software infrastructure enabled MSK to be a pioneer in large scale digital
pathology, a practice now slowly being deployed at other hospitals.
Coda
The online version with Ajaz’s worklist has been a success – and we’re looking at 100% adoption in 2025. Since the design and research was completed, I moved onto another project in the Research Platform – but I had always wanted to build out a version of our app in a native iOS – to take advantage of the M processors incredible power.
In the years since, I taught myself native *OS – and put an app into the App Store <low_key>yayyyyy</low_key>
The OS Platform ships with frameworks to handle medical images – so, I simply didn’t want to create a simple prototype; I wanted to fabricate the real deal in an app. And since the Vision Pro was launched – this has been an itch that I could not scratch.
So, out of the “love of the game” I decided to build it myself … and name it “Vangelos” after Evan Stamelos – the manager of Informatics & Digital Pathology!
