Tumor Sensitive Matching Flow: An Approach for Ovarian Cancer Metastasis Detection and Segmentation Jianfei Liu1 , Shijun Wang1 , Marius G. Linguraru2 , and Ronald M. Summers1 1

Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, Radiology and Imaging Sciences, National Institutes of Health Clinical Center Bethesda, MD 20892, USA 2 Sheikh Zayed Institute for Pediatric Surgical Innovation, Children’s National Medical Center, Washington DC 20010, USA

Abstract. Accurately detecting and segmenting ovarian cancer metastases can have potentially great clinical impact on diagnosis and treatment. The routine machine learning strategies to locate ovarian tumors work poorly because the tumors spread randomly to the entire abdomen. We propose a tumor sensitive matching flow (TSMF) to identify metastasiscaused shape variance between patient organs and atlas. TSMF juxtaposes the role of feature computation/classification, and TSMF vectors highlight tumor regions while dampening all other areas. Therefore, metastases can be accurately located by choosing areas with large TSMF vectors, and segmented by exploiting the level set algorithm on these regions. The proposed algorithm was validated on contrast-enhanced CT data from 11 patients with 26 metastases. 84.6% of metastases were successfully detected, and false positive per patient was 1.2. The volume overlap of the segmented metastases was 63 ± 5.6%, the Dice coefficient was 77 ± 4.2%, and the average surface distance was 3.9 ± 0.95mm. Keywords: Ovarian Cancer Metastases, Computer-Aided Detection, Tumor Sensitive Matching Flow

1

Introduction

Detecting and segmenting ovarian cancer metastases enhance the prognosis and treatment of women with ovarian cancer because 75% of them have tumors that have already spread at the time of diagnosis[9]. The metastasis detection manifests many challenges, including 1) variability in shapes and locations among individuals, 2) indistinctive intensity profile in comparison with surrounding tissues, 3) abnormal shapes of human organs compressed by metastases. Unpredictable locations of metastases hinder existing detection algorithms[12, 13, 4] to classify ovarian tumors despite the fact that salient tumor classifiers can be trained on annotated datasets. Manual annotation and classifier training are usually time-consuming. Moreover, most detection algorithms[1, 3, 6] focus on

2

Jianfei liu, et al.

(a)

(b)

(c)

Fig. 1: (a) Significant shape variance between the registered atlas (purple) and the patient’s liver (green); (b) optical flow methods can track the shape variance, which is mapped to the patient’s liver. Blue to red represents small to large displacements; (c) tumor sensitive matching flow highlights the displacements at tumor regions while dampening all other deformed areas.

finding lesions or tumors inside organs. Our purpose is instead to locate exterior ovarian cancer metastases attached to organs. Therefore, metastasis detection without training would be desirable. Metastases can be alternatively located by measuring local shape variance between patient data and atlas because they often push organs to deform. Image registration serves this purpose. For instance, free-form deformation registration[10] exploits the spline model to track non-rigid motion. Fig. 1a illustrates the shape variance between the registered atlas liver (purple) and the patient’s liver (green). The difference is partly due to attached metastases (red), but mainly caused by the variability among individuals. Therefore, shape comparison solely based on image registration is unreliable. Optical flow methods[2, 7] can compute relative image displacements between registered and patient livers, as shown in Fig. 1b. The amounts of image displacements are color mapped to the patient’s liver. Red areas contain large displacements, which match actual shape variance. However, the individual variability dominates the image displacements while metastases take minor effects. In this paper, we study the problem of eliminating shape variance caused by individual variability while keeping the variance due to metastases, so as to identify them. We propose a tumor sensitive matching flow (TSMF) to integrate local tumor classifier into optical flow computation. Tumor-like regions are emphasized during flow computation, while all other areas are suppressed. Fig. 1c illustrates the results of TSMF, where red regions correspond to the locations of metastases. Finally, we can place level set seeds in these areas to segment them.

2

Methodology

The flow of our method for detecting and segment metastasis is described in Fig. 2. It consists of three major steps: shape descriptor construction, tumor sensitive matching flow computation, and metastasis segmentation.

Tumor Sensitive Matching Flow

3

Fig. 2: Tumor sensitive matching flow method for detecting and segmenting metastasis. 2.1

Shape Descriptor Construction

Shape is the primary feature for radiologists to detect metastasis. The purpose of this step is to build shape descriptors for both patient images and reference CT images (scanned from healthy persons), depicted in top left and center images of Fig. 2. Liver and spleen are segmented using the method in Linguraru[5] from patient images because ovarian cancer metastases frequently attach to them. Distance transform[8] is then performed on the segmented organs to build the distance field, which is the shape descriptor for the patient data. Similar process is performed on one reference CT dataset. The Reference CT images are first registered with the patient images[10]. The registration parameters are then used to transform a probabilistic atlas, shown in top right image of Fig. 2, to the patient coordinate. The registered atlas thus covers the possible spatial ranges that the healthy organ would have. Another distance field is calculated from the registered atlas and used as the shape descriptor for the reference images. Therefore, we obtain two pairs of datasets: patient and registered reference images, and patient and atlas distance fields. 2.2

Tumor Sensitive Matching Flow (TSMF) Computation

The TSMF computation is the key to accurately identify metastases by comparing two pairs of datasets from the previous step. Let Ip (x, y, z) and Ia (x, y, z)

4

Jianfei liu, et al.

be the patient images and reference images, and Dp (x, y, z) and Da (x, y, z) be − their corresponding distance fields, with → u = (ux , uy , uz ) be the TSMF vector at point (x, y, z). Similar to optical flow computation[2, 7], the TSMF computation can be formulated as a global energy function within a minimization framework. → E(− w) =

ZZ (x,y,z)∈R3

Ψ ((Ia (x + ux , y + uy , z + uz ) − Ip (x, y, z))2 ) | {z } Intensity Constancy

+ βG(x, y, z)Ψ ((∇Ia (x + ux , y + uy , z + uz ) − ∇Ip (x, y, z))2 ) | {z } Gradient Constancy

(1)

+ γG(x, y, z)Ψ ((Da (x + ux , y + uy , z + uz ) − Dp (x, y, z))2 ) {z } | Distance Constancy + αΨ (|∇ux |2 + |∇uy |2 + |∇uz |2 )dxdydz | {z } Flow Smoothness

√ where Ψ (x2 ) = x2 + 2 ,  = 0.001 is a modified L1 norm and allows the computation to handle non-Gaussian deviations of the matching criterion. α, β, and γ are constants to balance different components. G(x, y, z) is a metastasislikelihood equation for estimating the probability of the metastasis existence at point (x, y, z). The larger the value of G(x, y, z), the more influence the distance and gradient constancy terms will conduct in the local flow computation. Therefore, flow vectors are magnified at the locations where metastasis is more likely to exist. Next, we clarify the definition of G(x, y, z). In Fig. 2.2, we notice that the intensity values of the metastasis are slightly lower than the liver and its region is approximately homogeneous. A Gaussian kernel is used to model the intensity distribution of metastases, and we experimentally determine that µm = 1060HU and σm = 20HU are the average and standard deviation from one representative dataset. The metastasis also generates the local concavity of the liver, and it can be measured by S(x, y, z) = Da (x, y, z)−Dp (x, y, z). MoreFig. 3: The analysis of metas- over, it stays at the exterior of the liver, and tasis properties to design a thus G(x, y, z) should be a piecewise function that highlights metastases at organ’s boundmetastasis-likelihood function. aries. Let Ω and Ω be the segmented organs and non-organ regions, respectively. ∂Ω is the organ boundary. G(x, y, z) is defined as,  0.01 for (x, y, z) ∈ Ω  m 2 ) ) for (x, y, z) ∈ ∂Ω G(x, y, z) = S(x, y, z)/exp( σσm )(1 + ( µ−µ (2) σ m  0.01 × (Dmax − Dp (x, y, z)) for (x, y, z) ∈ Ω

Tumor Sensitive Matching Flow

5

Here, Dmax is the largest distance value at the patient distance field, and µ and σ are mean and deviation of the intensity values of the non-organ pixels adjacent to the current boundary point in the patient images. Eq. 2 indicates that the likelihood of the metastasis existence remains a small value in the non-organ regions and gradually decreases towards the organ. The likelihood significantly increases if the local boundary has large shape change as well as the intensity values remains homogeneous within the metastasis intensity level. Therefore, Eq. 2 is sensitive to the metastases attaching to organs. Minimization. Eq. 1 is non-trivial to be minimized because it is a highly nonlinear and non-convex equation. In order to ease the description, we define the following abbreviations[7]. ∆I = Ia (x + ux , y + uy , z + uz ) − Ip (x, y, z) ∆D = Da (x + ux , y + uy , z + uz ) − Dp (x, y, z) ∆(∂x I) = ∂x Ia (x + ux , y + uy , z + uz ) − ∂x Ip (x, y, z)

(3)

∆(∂y I) = ∂y Ia (x + ux , y + uy , z + uz ) − ∂y Ip (x, y, z) ∆(∂z I) = ∂z Ia (x + ux , y + uy , z + uz ) − ∂z Ip (x, y, z) In terms of the calculus of variations, the Euler-Lagrange equation regarding to x component is expressed as Ψ 0 ((∆I)2 )∂x Ia ∆I + βG(x, y, z)Ψ 0 ((∆(∂x I))2 + (∆(∂y I))2 + (∆(∂z I))2 ) (∂xx Ia ∆(∂x I) + ∂xy Ia ∆(∂y I) + ∂xz Ia ∆(∂z I)) + γG(x, y, z)Ψ 0 ((∆D)2 )∂x Da ∆D (4) − αdiv(Ψ 0 (|∇ux |2 + |∇uy |2 + |∇uz |2 )∇ux ) = 0

The equations of y and z components can be similarly derived. However, Eq. 4 − is still nonlinear in its argument → u. Multi-scale analysis is an efficient approach to handle non-convexity of Eq. 4 as the solution in the coarse scale can better approximate the global minimum. Volume pyramids are constructed to simulate scale space on patient and reference images as well as their distance fields. Sampling rate 0.75 is used to ensure the smooth transition between different scales. Sequential linearization[2, 7] is another numerical strategy to remove nonlinearity in Eq. 4. It is represented as two nested fixed-point iterations. Assuming k be pyramid level and l be the outer iteration index, k,l k,l k,l k,l (∆I)k,l+1 = (∆I)k,l + (∂x Ia )k,l duk,l x + (∂y Ia ) duy + (∂z Ia ) duz

(5)

− k,l k,l k,l k,l k,l where → u k,l+1 = (uk,l x + dux , uy + duy , uz + duz ). Accordingly, non-linearity at ∆I is iteratively removed, and the same strategy can be performed on other abbreviations in Eq. 3. Let the inner iteration index be m, the purpose of the − inner iteration is to manipulate all Ψ 0 (∗) operators only relying on d→ u k,l,m when → − k,l,m+1 du is being estimated. Therefore, Eq. 4 is finally converted into a linear equation after two nested iterations. Successive over-relaxation method[14] is employed to minimize the massive liner system over the entire volume. After two nested iterations exceed predefined

6

Jianfei liu, et al.

values at the current pyramid level, the solutions are used as the initialization for the next pyramid level through bilinear interpolation. TSMF field is generated after the computation is accomplished at the finest pyramid level. Fig. 1c shows the final results. 2.3

Metastasis Segmentation

Level set method based on fast marching[11] is employed to segment metastases due to its accuracy. The key to extracting metastases successfully is the determination of a set of seed points. TSMF field makes the task of seed determination tractable. Potential metastasis regions are first extracted by selecting organ surfaces with the length of flow vectors larger than 15mm. Connecting graphs are then built on the selected surfaces to determine the number of connected regions, in other words, the number of potential metastases. If the number of vertices of a connected region exceeds 100 (approximately 100mm2 area on Fig. 4: The process of seed point deter- the organ surface), it is evenly split. Fig. 2.3 illustrates the process of mination, where the metastasis is represented as a red sphere and the liver seed point determination within a is shown in blue. A transitional point connected region. Let p = (x, y, z) be q is determined by moving a surface a surface point, represented as a green point p with half length of the flow vec- point. We first compute the transix, yˆ, zˆ) = (x+ u2x , y+ tor, indicated as white arrows. The seed tional point q = (ˆ uy uz point is determined by choosing the cen- 2 , z + 2 ) in yellow. A set of transiter points of a set of transitional points tional points can bePobtained, and its center point is q = i q, indicated as satisfying the intensity requirement. a black point. If µm − σm < I(q) < µm + σm , q is chosen as a seed point. Otherwise, search its adjacent points and select neighbored points that fulfill the intensity range. If none of them satisfy the condition, this region is rejected. The determined seed points are then imported into the fast marching approach and metastases are finally segmented.

3

Experimental Results

TSMF algorithm was tested on 11 abdominal contrast-enhanced CT datasets generated by Siemens 64-detector CT scanner. Slice thicknesses was 1mm. Each dataset has at least one ovarian cancer metastasis. Retrospective analysis of these images was inspected by our Institutional Reviewer Board. 26 metastases

Tumor Sensitive Matching Flow

7

in the selected datasets were annotated by an experienced radiologist and used as the ground-truth. Their size (the maximum diameter) range is 4.0-49.9mm. 22 metastases were attached to the liver, and the remaining 4 were touched to the spleen. It takes 20 minutes to process one patient. Fig. 5 illustrates the results from four patients, corresponding to four rows. Ground-truth metastases are illustrated in the right column, TSMF fields are given in the center column, and our segmentation results are shown in the right. The first patient in Fig. 5 has a metastasis in the right side of the liver. The TSMF field accurately tracks the shape change caused by this metastasis. As a result, the metastasis is identified and segmented in the right column. There is one false positive on the gallbladder. The intensity profile and shape of the gallbladder as computed by the TSMF are similar to those of the metastasis. The similar result was observed in the second patient. In the third patient, the spleen is also attached to a metastasis and TSMF can still locate and segment it correctly. The fourth patient is a challenging case because most organs in the left abdomen were removed. Metastasis (A) located at the left abdomen is attached to the liver only at one slice. Because the shape change is minor, TSMF misses it. The same issue happens to the metastasis (B) because of its small size. However, the remaining metastases are successfully detected by TSMF. The detailed validation of detection and segmentation on 11 patients were presented in table 1. Sensitivity (Sen.) and false positive per patient (FP/Patient) are used to evaluate detection results. Six metrics used in liver segmentation[5] are employed to evaluate the metastasis segmentation. They are volume overlap (VO), Dice coefficient (DC), relative absolute volume difference (RA), average symmetric absolute surface distance (AS), symmetric RMS surface distance (SR), and maximum symmetric absolute surface distance (MS).

Table 1: Validation results of metastasis detection and segmentation on 11 patients. Detection Segmentation Sen.(%) FP/Patient VO(%) DC(%) RA(%) AS(mm) SR (mm) MS (mm) 84.6 1.2 63±5.6 77 ± 4.2 27 ± 15 3.9 ± 0.95 6.5 ± 2.6 15 ± 2.7

4

Conclusion and Future Work

We have proposed a tumor sensitive matching flow (TSMF) algorithm to detect and segment ovarian cancer metastases randomly distributed in the abdomen from contrast-enhanced CT data. TSMF provides an efficient means to measure shape variance caused by metastases between patient images and atlas data while suppressing all other deformations. Therefore, metastases can be accurately located and segmented according to the TSMF field. The method can successfully

8

Jianfei liu, et al.

Fig. 5: The comparison between ground-truth (left column) and segmented metastases (right column). TSMF results are also illustrated in the center column, where TSMF vectors are color mapped to the organ surfaces (blue to red represents the increment of flow vectors). Each row corresponds to a patient. True metastases are shown in red and false positives in yellow. False positives tend to be located near the gallbladder because its intensity and shape are similar to metastases. Livers and spleens were automatically segmented. Metastasis A is missed because of incomplete liver segmentation and B is due to its small size (5.6mm).

detect 84.6% of metastases on data from 11 patients with an average surface distance of 3.88mm.

Tumor Sensitive Matching Flow

9

However, the TSMF presented 13 false positives over 11 patients. Seven of them were located on the gallbladder because it has a similar intensity distribution and shape to the metastases. Currently, we are introducing the gallbladder atlas to the metastasis detection, so as to reduce false positives. Moreover, three of four true positives were missed due to small shape changes between metastases and organs. We are developing a more accurate metastasis-likelihood function based on the information from local image structures, such as texture and shape index, to enhance the sensitivity to the metastases. In addition, segmented metastases sometimes include the surrounding tissues because the boundaries between them are undistinguished. Segmentation strategies based on other image information beyond intensities is being studied to prevent oversegmentation. Last but not least, we are collecting more datasets to test our detection algorithm. Not only are patients with metastases chosen, but also the datasets without metastases are also considered to evaluate the robustness of our detection algorithm. Acknowledgments. This research was supported by the Intramural Research Program of the NIH Clinical Center. The authors thank Dr. Elise Kohn for helpful comments.

References 1. Bilello, M., Gokturk, S.B.: Automatic detection and classification of hypodense hepatic lesions on contrast-enhanced venous-phase ct. Medical Physics 31 (2004) 2584–2593 2. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Proc. of ECCV. Volume 3024. (2004) 25–36 3. Gletsos, M., Mougiakakou, S.G., Matsopoulos, G.K., Nikita, K.S., Nikita, A.S., Kelekis, D.: A computer-aided diagnostic system to characterize ct focal liver lesions: design and optimization of a neural network classifier. IEEE Transactions on Information Technology in Biomedicine 7 (2003) 153–162 4. Janowczyk, A., Chandran, S., Singh, R., Sasaroli, D., Coukos, G., Feldman, M.D., Madabhushi, A.: Hierarchical normalized cuts: Unsupervised segmentation of vascular biomarkers from ovarian cancer tissue microarrays. In: Proc. of MICCAI. (2009) 230–238 5. Linguraru, M.G., Sandberg, J.K., Li, Z., Shah, F., Summers, R.M.: Automated segmentation and quantification of liver and spleen from ct images using normalized probabilistic atlases and enhancement estimation. Medical Physics 37 (2010) 771– 783 6. Linguraru, M., Richbourg, W., Watt, J., Pamulapati, V., Summers, R.: Liver and tumor segmentation and analysis from ct of diseased patients via a generic affine invariant shape parameterization and graph cuts. In: Proc. of MICCAI 2011 Workshop on Abdominal Imaging - Computational and Clinical Applications. Volume 7029. (2011) 198–206 7. Liu, J., Subramanian, K., Yoo, T.: Temporal volume flow: an approach to tracking failure recovery. In: Proc. of SPIE Medical Imaging. (2011) Lake Buena Vista, Florida, USA.

10

Jianfei liu, et al.

8. Maurer, J.C., Qi, R., Raghavan, V.: A linear time algorithm for computing exact euclidean of distance transform of binary images in arbitrary dimension. IEEE Transactions on Pattern Analysis and Machine Intelligence 25 (2003) 265–270 9. Memarzadeh, S., Berek, J.: Advances in the management of epithelial ovarian cancer. The Journal of Reproductive Medicine 46 (2001) 621–629 10. Rueckert, D., Sonoda, L.I., Hayes, C., Hill, D.L.G., Leach, M.O., Hawkes, D.J.: Nonrigid registration using free-form deformations: Application to breast mr images. IEEE Transactions on Medical Imaging 18 (1999) 712–721 11. Sethian, J.A.: Level set methods: evolving interfaces in computation geometry, fluid mechanics, computer vision, and materials Science. 1th edn. Cambridge University Press (1999) 12. Smeets, D., Loeckx, D., Stijnen, B., Dobbelaer, B.D., Vandermeulen, D., Suetens, P.: Semi-automatic level set segmentation of liver tumors combining a spiral scanning techniques with supervised fuzzy pixel classification. Medical Image Analysis 14 (2010) 13–20 13. Summers, R.M.: Computed tomographic virtual colonoscopy computer-aided polyp detection in a screening population. Gastroenterology 129 (2005) 1832–1844 14. Young, D.: Iterative Solution of Large Linear Systems (Computer Science and Applied Mathematics). 1th edn. Academic Press (1971)