Quantitative Medical Image Analysis in the Cloud using Big Data Approaches

• Introduction
  • Quantitative?
  • Big?
• Small Data vs Big Data

The Tools

• Spark Image Layer
• Quantitative Search Engine for Images

The Science

• Academic Projects
• Commercial Projects
• Outlook / Developments

• 83% of Harvard Radiologists thought this scan looked perfectly normal
• http://www.npr.org/sections/health-shots/2013/02/11/171409656/why-even-radiologists-can-miss-a-gorilla-hiding-in-plain-sight
• called inattentional blindness
  • since the focus was on counting cancerous lung nodules
  • standard sanity checks were never made

Human vision system is imperfect

Which center square seems brighter?

Are the intensities constant in the image?

• There is a complex relationship between the macroscopic bone strength and the microscopic cellular networks inside of them.
• Examining the individual cells can give us insights into what makes for healthy and pathological bone growth

1. Count how many cells are in the bone slice
2. Ignore the ones that are ‘too big’ or shaped ‘strangely’
3. Are there more on the right side or left side?
4. Are the ones on the right or left bigger, top or bottom?

• Do it all over again for 96 more samples, this time with 2000 slices instead of just one!

• Genetic studies require hundreds to thousands of samples

• 

• For this study, the difference between 717 and 1200 samples is the difference between finding the links and finding nothing.

• Now again with 1090 samples!
• 

• Those metrics were quantitative and could be easily visually extracted from the images

• What happens if you have softer metrics

• 

• How aligned are these cells?
• Is the group on the left more or less aligned than the right?
• errr?

As the amount of data increases

But how do radiologists / experts scale

• Expensively
  • Median salary $468,594 $\rightarrow$ $245 / hour
• Poorly
  • Reading twice as fast $\rightarrow$ 2.5X as many errors

X-Ray

• Swiss Light Source (SRXTM) images at (>1000fps) $\rightarrow$ 8GB/s, diffraction patterns (cSAXS) at 30GB/s
• Nanoscopium (Soleil), 10TB/day, 10-500GB file sizes, very heterogenous data

Optical

• Light-sheet microscopy (see talk of Jeremy Freeman) produces images $\rightarrow$ 500MB/s
• High-speed confocal images at (>200fps) $\rightarrow$ 78Mb/s

Geospatial

• New satellite projects (Skybox, etc) will measure >10 petabytes of images a year

Personal

• GoPro 4 Black - 60MB/s (3840 x 2160 x 30fps) for $600
• fps1000 - 400MB/s (640 x 480 x 840 fps) for $400

The breakdown of science into component tasks has changed from a time perspective over the last years and how we expect it to change by 2020.

• Experiment Design (Pink)
• Data Management (Blue)
• Measurement Time (Green)
• Post Processing (Purple)

The primary trends to focus on are the rapid decrease in measurement time and increasing importantance of post-processing.

The breakdown of science into component tasks has changed from a time perspective over the last years and how we expect it to change by 2020.

• Experiment Design (Pink)
• Data Management (Blue)
• Measurement Time (Green)
• Post Processing (Purple)

The primary trends to focus on are the rapid decrease in measurement time and increasing importantance of post-processing.

If you looked at one 1000 x 1000 sized image every second

It would take you
139 hours to browse through a terabyte of data.

Big Data is a very popular term today. There is a ton of hype and seemingly money is being thrown at any project which includes these words. First to get over the biggest misconception big data isn't about how much data you have, it is how you work with it

Before we can think about Big Data we need to define what it's replacing, Small Data.

Small Data

The 0815 approach, using standard tools and lots of clicking

• Hand Identification $\rightarrow$ 30s / object
• 30-40k objects per sample
• One Sample in 6.25 weeks
• 1300 samples $\rightarrow$ 120 man-years

Advantages

• Hand verification of every sample, visual identification of errors and problems

Disadvantages

• Biased by user
• Questionable reproducibility
• Time-consuming
• Exploration challenging
• Data versioning is difficult

• ImageJ macro for segmentation
• Python script for shape analysis
• Paraview macro for network and connectivity
• Python script to pool results
• MySQL Database storing results (5 minutes / query)

1.5 man-years

Advantages

• Faster than manual methods
• Less bias
• More reproducible

Disadvantages

• Compartmentalized analysis
• Time-consuming
• Complex interdependent workflow
• Fragile (machine crashes, job failures)
• Expensive - Measure twice, cut once, not exploratory

Correctness

The most important job for any piece of analysis is to be correct.

• A powerful testing framework is essential
• Avoid repetition of code which leads to inconsistencies
• Use compilers to find mistakes rather than users

Easily understood, changed, and used

Almost all image processing tasks require a number of people to evaluate and implement them and are almost always moving targets

• Flexible, modular structure that enables replacing specific pieces

Fast

The last of the major priorities is speed which covers both scalability, raw performance, and development time.

• Long waits for processing discourages exploration
• Manual access to data on separeate disks is a huge speed barrier
• Real-time image processing requires millisecond latencies
• Implementing new ideas can be done quickly

• Rather than building an analysis as quickly as possible and then trying to hack it to scale up to large datasets
  • chose the framework first
  • then start making the necessary tools.
• Google, Amazon, Yahoo, and many other companies have made huge in-roads into these problems
• The real need is a fast, flexible framework for robustly, scalably performing complicated analyses, a sort of Excel for big imaging data.

Apache Spark and Hadoop 2

The two frameworks provide a free out of the box solution for

• scaling to >10000 computers
• storing and processing exabytes of data
• fault tolerance
  • 2/3rds of computers can crash and a request still accurately finishes
• hardware and software platform indpendence (Mac, Windows, Linux)

These frameworks are really cool and Spark has a big vocabulary, but flatMap, filter, aggregate, join, groupBy, and fold still do not sound like anything I want to do to an image.

I want to

• filter out noise, segment, choose regions of interest
• contour, component label
• measure, count, and analyze
• …

Spark Image Layer

• Developed at 4Quant, ETH Zurich, and Paul Scherrer Institut
• The Spark Image Layer is a Domain Specific Language for Microscopy for Spark.
• It converts common imaging tasks into coarse-grained Spark operations

We have developed a number of commands for SIL handling standard image processing tasks

Fully exensible with

$$\downarrow \textrm{Perform analysis on a every image}$$

$$\downarrow \textrm{Filter out the most anisotropic cells}$$

The query is processed by the SQL parsing engine

• sent to the database where it can be combined with locally available private data
• sent to the image layer for all image processing components to be integrated
• sent to the Spark Core so specific map, reduce, fold operations can be generated
• sent to the master node where it can be distributed across the entire cluster / cloud

Once processed into Spark commands the query can be executed on multiple machines using multiple data storage environments in the background

• Data-locality ensures analysis is run efficiently on the same machine it is stored on
• Network communication fully managed
• Fault tolerance of nodes ensured

For many traditional scientific studies, the sample count is fairly low 10-100, for such low counts, if we look at the costs (estimated from our experience with cortical bone microstructure and rheology)

• Development costs are very high for Big
• Traditionally students are much cheaper than the 150kCHF / year used in this model
• Medium Data is a good compromise

As studies get bigger (>100 samples), we see that these costs begin to radically shift.

• The high development costs in Medium and Big $\rightarrow$ 0
• The high human costs of Small and Medium $\propto$ $Samples$

Instead of

We have

• Healthy

• Sick

A spinoff - 4Quant: From images to insight

• Quantitative Search Machine for Images
  • Find all patients with livers larger than 10cm diameter
  • Count the number of highly anisotropic nuclei in myeloma patients
• Custom Analysis Solutions
  • Custom-tailored software to solve your problems
• One Stop Shop
  • Measurement, analysis, and statistical analysis

Education / Training

• Consulting
  • Advice on imaging techniques, analysis possibilities
• Education
  • Workshops on Image Analysis
  • Courses / Training
  • Quantitative Big Imaging
• Contact Us: contact@4quant.com

• Flavio Trolese

• Dr. Prat Das Kanungo

• Dr. Ana Balan

• Prof. Marco Stampanoni

• TOMCAT Group
• Radiology Team at the Universitaetspital Basel
• Keith Cheng from Penn State Medical Center
• AIT at PSI and Scientific Computer at ETH

We are interested in partnerships and collaborations

Learn more at

• 4Quant: From Images to Statistics - http://www.4quant.com
• Quantitative Big Imaging Course at ETH Zurich
• X-Ray Imaging Group at ETH Zurich - http://bit.ly/1gD8wKb