Deep Learning for Hypokinesia Detection in Singlephoton Imaging Data

Halima Dziri*1, Hamida Romdhane1, Mohamed Ali cherni2 and Dorra Ben Sellem3

¹Université de Tunis El Manar, Laboratoire de recherche en Biophysique et Technologies Médicales (LRBTM), Tunis, Tunasia
²Université de Tunis, LR13 ES03 SIME, ENSIT, Montfleury 1008 Tunisia
³Université de Tunis El Manar, Faculté de Médecine de Tunis, 1007 Tunasia;
⁴Institut Salah Azaiez, Service de Médecine Nucléaire, 1006, Tunis, Tunasia

*Corresponding author

Halima Dziri, Assistant Professor, Department of Surgery, Niazi Medical and Dental College, Sargodha, Pakistan.

Abstract

Background: Nuclear medicine is a critical tool in diagnosing various cardiovascular diseases. Despite its diagnostic and prognostic value, interpretation and analysis of isotopic ventriculography images are still widely performed manually by clinicians.

Methods: A classification algorithm is proposed to analyze isotopic ventriculography data using image processing and the intelligence artificial techniques. These algorithms provided opportunities for developing automated analysis and interpretation systems. In this paper, we apply the intelligence artificial based deep learning for analyzing isotopic ventriculography data.  We propose a preprocessing data based on randomly transformation and we integrate the dropout in order to increase the architecture performance. Then, we compare it to a transfer learning.

Results: The accuracy and the Receiver operating characteristic (ROC) analysis were applied to compare the two classifier. A high accuracy (90%) was achieved by the proposed architecture, compared to the AlexNet architecture (85%). The area under the curve was significant for the two classifier. A high z-scors for the proposed method confirm that this architecture is the best classifier in term of performance ( (AUC=0.90; p-value=0.002; Z-scors= 0.707) vs (AUC=0.85; p-value=0.008; Zscors= -0.707)).

Conclusion: results show that the proposed method is very efficient for Hypokinesia detection in Single-photon Imaging Data.

Keywords: Hypokinesia, Normokinesia, Heart Disease, Classification, CNN.

Introduction

The assessment of heart diseases is controversial, a fact that may be interpreted by the influence of left ventricle (LV) dysfunction [1]. Many indicators may be helpful to characterize cardiac function during contraction, as the quantification of the left ventricle ejection fraction (LVEF) but their feasibility is not enough for the doctor's decision. For this reason, the efficient diagnosis consists of tracking the left ventricle kinetics and the detection of the abnormal contractility such as hypokinesia. Hypokinesia is a type of LV movement disorder. It specifically means that the LV wall motion has a slow movement.

That represents markers of much cardiac pathology such as cardiomyopathy, heart failure,  myocarditis. Thus each pathology has a group of diseases that affect the heart muscle [5]. Several cardiac imaging modalities are applied for the diagnosis of LV function, such as cardiovascular computed tomography, echocardiography, cardiac magnetic resonance imaging, nuclear medicine [2, 3].

Particularly, single-photon imaging is used to diagnose ventricle kinetics, computing clinical parameters such as the end-systolic volume (ESV), the end-diastolic volume (EDV), and the LVEF. However, interpretation and analysis of isotopic ventriculography images are still widely performed manually by clinicians. Thus the interpretation varies from doctor to another.  For recent years, searchers in the cardiac field involve the use of artificial intelligence to improve the diagnosis [6]. However, Artificial intelligence is any computer technology that solves complex problems that were thought to be reserved for human intelligence.

Machine learning is a sub-discipline of artificial intelligence [7]. It can be classified into three learning kind. The first type is the supervised learning, including Support Vector Machine (SVM), Naive Bayes (NB), K-nearest neighbors (KNN), Genetic algorithms (GA), Random Forests (RF), and Gradient Boosting (GB), neural network (NN). Where, to predict the coveted and known result, algorithms should be receives database labeled by humans. For example, if it is wished for predict whether a LV function is abnormal, analysis should be carried based on a healthy dataset containing a set of patients that presented such a function and another set in which this abnormalities was not observed. This learning type is significant for regression problems and classification, but it need a lot of data that must be labeled by humans. It follows that is timeconsuming.

The second learning type is the unsupervised learning, containing K_means, principal component analysis, Kohonen neural networks, Autoencoders... In this type the algorithms use a dataset unlabeled to predict unknown outcome. Unsupervised learning aims to identify unfamiliar disease mechanisms, phenotypes or genotypes from hidden patterns existing in the data. In unsupervised learning, the objective is to discover the hidden patterns in the dataset without label data by humans. One major limitation of this learning is the biases because of hardness identifying the initial cluster pattern and the presence of noisy data can outcome in inaccurate decisions.

The last type is reinforcement learning like Markov Decision Process (MDP) and Q learning (QL). It can be defined as a hybrid of the former of two types (supervised and unsupervised learning). The goal of reinforcement learning is to increase the algorithms accuracy employing trial and error. Thus it helps to take decisions sequentially. But this learning kind is time-consuming and computingheavy. To analyzing medical images, artificial intelligence has fast become a methodology of choice [8]. Since this technology enable decision support systems can provide a rapid diagnosis of several diseases. Thus it provides a significantly changing in patient healthcare services [9]. Particularly, the use of machine learning techniques made a major contribution to cardiovascular image analysis [10]. While it has an impact on avoiding the delay and errors in diagnosis. An overview of the literature in cardiovascular image analysis displays different studies. For example, for heart failure risk prediction Samuel O.W., et al [11] was proposed an Artificial Neural Network (ANN) and Fuzzy based integrated decision support system. Also, the ANN was used by Nakajima, K et al [12] to detect the myocardial ischemia in myocardial perfusion imaging.

Betancur et al. [13] applied the support vector machine in single-photon emission computed tomography database.

A good result obtained were compatible with the experts decision to define mitral valve plane (VP)  positioning during left ventricular segmentation in order to detect the ischemic areas and the obstructive stenosis. Ouyang, D. et al [14] proposed the EchoNet-Dynamic, which was can predict ejection fraction and identifies cardiomyopathy in echocardiography database. This tool was based on video deep learning using the convolutional neural network (CNN).

Also, Attia, Z. I. et al [15] used the neural network to screen the cardiac contractile dysfunction in electrocardiogram database. The machine learning based on decision tree algorithm is also used by Dabiri, Y et al [16] to predict left ventricular mechanics. This model allow to predect volume, LV pressure, and stress.

In the present paper, study aim is to predict the hypokinesia in 64 patients with diabetes and oncology as assessed by isotopic ventriculography (followed at the Salah AZAIEZ Institute). based on the parametric image the LV dysfunction is identifed by tracking the LV contraction while the heart cycle using covariance analysis. Then a different deep neural network architecture is utilized to classify each LV contraction for normokinetics or hypokinesia.

The paper is organized as follows: in the second section, we describe the methodology for Left ventricular contractility classifcation. The third section, shows the experimental results obtain by applying the proposed classifier on the test dataset, then compares these results with those obtained by other classifcation methods.

Finally, section, concludes our work.

Materials and Methods

Study Population: The database contains 68 sequences for patients (34 patients with hypokinesia and 34 patients’ normokinesia) followed at the Salah AZAIEZ Institute for various cancers and having undergone isotopic ventriculography, to evaluate the cardiac repercussions of chemotherapy.  Images are acquired at the Nuclear Medicine department of the same Institute. Red blood cells of patients were labeled in vivo with Tc99m after pretreatment with a stannous solution. The procedure involves two successive injections: injection of 5 mg pyrophosphate, which fixed on the red blood cells, and, 20 minutes after, 740 MBq of Tc99m. The equilibrium study begins 5-10 minutes later. The radiopharmaceutical is distributed in the blood compartment and the four heart chambers.

Proposed Method

Parametric image:  The parametric image allows monitoring of ventricular kinetics by comparing the movement of each pixel with reference region. For each pixel in the isotopic ventriculography images, the degree of covariance is calculated by the following covariance function:

With I (i, j) is the pixel value, M is the number of images in the isotopic ventriculography sequence and µI (i; j) is the average of the pixel values (i, j) in sequence. After the selection of the reference regions (ROI), the mean activity ROI(t) of the ROI is computed for each image of the sequence and the temporal average µROI also computed.

We extract the LV boundaries using segmentation based on active contour [17].  Then the detected boundaries used as a mask to identify the LV in the parametric image [18]. The process showing in figure 1 is a pre-processing of database. Then, it used in classification with convolution neural network.

Convolution neural network

CNN is a type of deep neural network  ( DNN) popularly ref erred to as CNN or ConvNet[19].

CNNs are inspired by the visual biological cortex. CNN take s an image as it moves through a sequence of convolutionary, nonlinear, pooling and dense layers and receives class score as the best definition of the image.

Transfer learning

Transfer learning is a popular machine learning approach where a pre-trained model is reused on a second task. In this paper, the transfer learning AlexNet is used to predict the hypokinesia from the parametric images. Fig 2

presents the AlexNet architecture which is developed by Alex Krizhevsky [20].  Because the input size in AlexNet architecture is 227*227*3, we change the input size by changing the image size from 64*64*3 to 227*227*3. The final fully connected layer is also changed to become 2 neurons.

Proposed architecture 

Architecture and training parameter

In this study we use cross-validation technique in which we divide the dataset in to two groups (70%)   for the training data and 30% for the test data.

Preprocessing techniques to deal with overfitting is applied to the training data.  It allows increasing the number of training images which are useful for small training sets to achieve good CNN architecture performance. The architecture of a convolutional neural network for the LV contractility classification is summarized in figure 3. The network takes in a fixed-size input of 64x64 pixelsized color images.

Each image is transmitted through 3 convolutionslayers,       2 layers of maxpools, and 2 layers of full connections. All co nvolutionary layers consist of 3x3 filters with stride 1 and all  max-pooling is applied with stride 2 over a 2x2 window. After all convolution layers, the batch normalization layers a re inserted, used to accelerate      deep network training by m aking data standardization an essential part of the network architecture[21,22].

ReLU (Rectified Linear Unit) is used for activation function. It is linear and simply threshold at zero. This activation function improves learning rate and classifier accuracy as opposed to the sigmoid activation function. The ReLU is expressed as:

𝑓(𝑥) = max (0, 𝑥)  (2)

The activation function used for the fully connected layer is the Softmax.

The convolution neural network is developed with the following hyper-parameter values: the learning rate is 0.00001, the batch size is 128, the number of epochs is 500, momentum is 0.8 and the optimizer type is “sgdm”.

Figure 1: Process of left ventricle extraction from parametric image

Figure 2: AlexNet architecture proposed by Alex Krizhevsky [20]

Figure 3: Proposed CNN architecture

Figure 4: Example of augmentation data: a) original image; b) original image+rotation+translation

Figure 5: Confusion matrix: a) confusion matrix of AlxNet classifier, b) confusion matrix of proposed architecture with HP=hypokinesia; NP= normokinesia.

Figure 6: ROC curve of the two classifier 

Reducing Overfitting

Dropout

The technique recently introduced, known as dropout is developed by Hinton et al. [23] that involves randomly deactivating hidden neuron with probability p (commonly p = 0.5) during training time. In this way, the neurons which are "dropped out" do not contribute to the forward transfer and are not active in backpropagation.

Dropout is the solution uses for prevent overfit by preventing neurons from co-adapting to one another. Because a neuron cannot rely on any other neuron to be active throughout any particular iteration of training, the neuron must learn to obtain inputs in general, rather than specifically.

Data augmentation

The present database is characterized by insufficient size for training the convolution neural network. Hence, we utilize augmentation algorithms to enlarge the size of database to reducing the overfitting.

Data augmentation [24, 25] is a popular approach, which consists in transforming the available data into new data using label-preserving transformations. This transformation is perfomed by randomly generating. Besides there are different augmentation techniques such as resizing, rotation, translation, and reflection Table 1 illustrates the several augmentation techniques that preprocesses database before training.

We generate seven new data then evaluate each one. Figure 4 present an example of augmented data correspondents to the augmented data5. Where we generate an image data augmenter. This augmenter translates the input images horizontally and vertically up to three pixels at random, and rotates image by random angles in the range [-20 20] degrees.

A total of 1024 images are generated used in training steps.

Results and discussion

Figure 5 illustrate the Confusion matrix obtain by the two classifiers, which is often applied to analyze categorical data (normokinesia or hypokinesia). The Confusion matrixes lead to calculate several metrics for each classifier. True positive (TP) indicates the number of positive abnormal contractility that are detected as hypokinesia. True negative (TN) indicates the number of normal contractility that are detected as normokinesia. False positive (FP) indicates the number of normal contractility that are detected as hypokinesia. False negative (FN) indicates the number of patient have hypokinesia that are detected as normokiesia.

Accuracy, sensitivity, specificity are the metrics used in this study. We utilize the sensitivity and the specificity to draw the roc curve. The ROC curve is a plot of the true positive rate against the false positive rate for the different possible cut points of a diagnostic test

Fig 6 show the ROC curve of the two methods. The wheat line represents a reference line, which corresponds to an AUC of 0.5. The green line represents the proposed architecture line; it corresponds to an AUC of 0.9. The blue line represents the AlexNet architecture line, which corresponds to an AUC of 0.85.

Table 1:

Table 2: Comparaison of classifiers performance

Table 3: Study the effect of image augmentation on deep learning accuracy

The AUC of the proposed architecture is close to 1, which indicates that it is the most efficient architecture. This result was confirmed statistically. Indeed, the AUC result for the two classifier was significant (table 2). We also used the z-score, to aid comparison. The Z-score is calculated as follows [26]:

The AUC̅̅̅̅̅̅ is the mean of the area under the curve and σ(AUC) is the standard deviation of the AUC. The highest z score was obtained for proposed architecture (table 2), which further confirms its superiority over the AlexNet architecture for the detection of hypokinesia the AUC. The highest z score was obtained for proposed architecture (table 2), which further confirms its superiority over the AlexNet architecture for the detection of hypokinesia.

In this analysis, we measure the accuracy for each architectur e, and the results provided in      table 3   indicate that the arc hitecture proposed is the most effective.  So the accuracy ach ieved by the proposed architecture is equal to 90 percent.

Conclusion

In this work, we presented a new architecture based on the deep learning to assess the hypokinesia in isotopic ventriculography images. We compared it to the knowing transfer learning AlexNet, the two architecture applied to the same database. First, we compared the results of the two classifiers to the doctor's decision. The proposed architecture was the most accurate for detecting cardiac wall motion abnormalities.

Second, we compared the two method based on ROC curve, better results were also obtained with the proposed classifier, thus clearly confirming its superiority.

References

  1. Boissier, F., Razazi, K., Seemann (2017). Left ventricular systolic dysfunction during septic shock: the role of loading conditions. Intensive Care Med 43, 633–642
  2. Flachskampf, F., Baron, T. (2020). Heart failure and cardiac imaging: Choosing wisely in the era of multimodality imaging. Anatolian Journal of Cardiology, 23(4), 204-208.
  3. Clarysse, P, P. Friboulet (2015). Multi-modality cardiac imaging: processing and analysis. Hoboken: ISTE Ltd and Wiley, 2015.
  4. Zamzmi, G, Hsu, L. Y., Li, W., Sachdev, V., & Antani, S et al (2020). Harnessing Machine Intelligence in Automatic Echocardiogram Analysis: Current Status, Limitations, and Future Directions. IEEE Reviews in Biomedical Engineering. 2020.
  5. KAYA, Z. Raczek, P. Rose, N. Myocarditis and dilated cardiomyopathy. In: The Autoimmune Diseases. Academic Press, 2020. p. 1269-1284.
  6. Massalha, S., Clarkin, O., Thornhill, R., Wells, G., & Chow, B. J. et al (2018). Decision support tools, systems, and artificial intelligence in cardiac imaging. Canadian Journal of Cardiology, 34(7), 827-838.
  7. Souza Filho, E. M. D., Fernandes, F. D. A., Soares, C. L. D. A., Seixas, F. L., Santos, A. A. S., Gismondi, R. A., ... & Mesquita, C. T. et al (2019). Artificial Intelligence in Cardiology: Concepts, Tools and Challenges-“The Horse is the One Who Runs, You Must Be the Jockey”. Arquivos brasileiros de cardiologia, (AHEAD).
  8. Johri, P., Saxena, V. S., Kumar, A. K., & Gaur, N. K. et al (2020). The Use of AI in Decision Support System for Disease Diagnosis: A Study. Available at SSRN 3553199.
  9. Casimir P. (2015), “Role of Clinical Decision Support Systems in improving clinical practice” , MoJ Clinical & Medical Case Report.
  10. Krittanawong, C., Zhang, H., Wang, Z., Aydar, M., & Kitai, T. et al (2017). Artificial intelligence in precision cardiovascular medicine. Journal of the American College of Cardiology, 69(21), 2657-2664.
  11. Samuel, O. W., Asogbon, G. M., Sangaiah, A. K., Fang, P., & Li, G. et al (2017). An integrated decision support system based on ANN and Fuzzy_AHP for heart failure risk prediction. Expert Systems with Applications, 68, 163–172.
  12. Nakajima, K., Matsuo, S., Wakabayashi, H., Yokoyama, K., Bunko, H., Okuda, K., ... & Edenbrandt, L. (2015). Diagnostic performance of artificial neural network for detecting ischemia in myocardial perfusion imaging. Circulation Journal, 79(7), 1549-1556.
  13. Betancur, J., Rubeaux, M., Fuchs, T. A., Otaki, Y., Arnson, Y., Slipczuk, L., ... & Berman, D. S. et al (2017). Automatic valve plane localization in myocardial perfusion SPECT/CT by machine learning: anatomic and clinical validation. Journal of Nuclear Medicine, 58(6), 961-967.
  14. Ouyang, D., He, B., Ghorbani, A., Langlotz, C., Heidenreich, P. A., Harrington, R. A, & Zou, J. Y. et al (2019). Interpretable AI for beat-to-beat cardiac function assessment. medRxiv, 19012419.
  15. Attia, Z. I., Kapa, S., Lopez-Jimenez, F., McKie, P. M., Ladewig, D. J., Satam, G, ... & Asirvatham, S. J. et al (2019). Screening for cardiac contractile dysfunction using an artificial intelligence–enabled electrocardiogram. Nature medicine, 25(1), 70-74.
  16. Dabiri, Y., Velden, A. V. D., Sack, K. L., Choy, J. S., Kassab, G. S., & Guccione, J. et al (2019). Prediction of Left Ventricular Mechanics Using Machine Learning. Frontiers in physics, 7, 117.
  17. Dziri, H., Cherni, M. A., & Ben-Sellem, D. (2021). New Hybrid Method for Left Ventricular Ejection Fraction Assessment from Radionuclide Ventriculography Images. Current Medical Imaging, 17(5), 623-633.
  18. Dziri, M. A. Cherni and D. B. Sellem (2020). Detecting Abnormal ventricular Contractility from Radionuclide Ventriculography Images. 2020 5th International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), Sousse, Tunisia, 2020, pp. 1-6.
  19. Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., ... & Sánchez, C. I. et al (2017). A survey on deep learning in medical image analysis. Medical image analysis, 42, 60-88.
  20. Krizhevsky, A., Sutskever, I. and Hinton, G.E., 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems (pp. 1097-1105).
  21. Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. ArXiv preprint arXiv: 1502.03167.
  22. Schilling, F (2016). The Effect of Batch Normalization on Deep Convolutional Neural Networks; DiVA Publisher: Uppsala, Sweden.
  23. Geoffrey E. Hinton, Simon Osindero, and Yee-Whye The (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18:1527–1554, 2006.
  24. Fawzi, A., Samulowitz, H., Turaga, D., & Frossard, P. et al (2016) . Adaptive data augmentation for image classification. In 2016 IEEE International Conference on Image Processing (ICIP) (pp. 3688-3692). Ieee.
  25. Fawzi, A., & Frossard, P. (2015). Manitest: Are classifiers really invariant?. ArXiv preprint arXiv: 1507.06535.
  26. Hajian-Tilaki, K (2013). Receiver operating characteristic (roc) curve analysis for medical diagnostic test evaluation. Caspian J Intern Med, 4, 627.
TOP