If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Currently, a new coronavirus called COVID-19 is the biggest challenge of the human at 21st century. Now, the spread of this virus is such that mortality has risen strongly in all cities of countries. Therefore, it is necessary to think of a solution to handle the disease by fast and timely diagnosis. This paper proposes a method that uses chest X-ray imagery to divide 2-4 classes into 7 different Scenarios, including Bacterial, Viral, Healthy, and COVID-19 classes. The aim of this study is to propose a method that uses chest X-ray imagery to divide 2-4 classes into 7 different Scenarios, including Bacterial, Viral, Healthy, and COVID-19 classes.
Methods
6 different databases from chest X-ray imagery that have been widely used in recent studies have been gathered for this aim. A Convolutional Neural Network-Long Short Time Memory model is designed and developed to extract features from raw data hierarchically. In order to make more realistic assumptions and use the Proposed Method in the practical field, white Gaussian noise is added to the raw chest X-ray imagery. Additionally, the proposed network is tested and investigated not only on 6 expressed databases but also on two additional databases.
Results
On the test set, the proposed network achieved an accuracy of more than 90% for all Scenarios excluding Scenario V, i.e. Healthy against the COVID-19 against the Viral, and also achieved 99% accuracy for separating the COVID-19 from the Healthy group. The results showed that the proposed network is robust to noise up to 1 dB. It is worth noting that the proposed network for two additional databases, which were only used as test databases, also achieved more than 90% accuracy. In addition, in comparison to the state-of-the-art pneumonia detection approaches, the final results obtained from the proposed network is so promising.
Conclusions
The proposed network is effective in detecting COVID-19 and other lung infectious diseases using chest X-ray imagery and can thus assist radiologists in making rapid and accurate detections.
Introduction
The root of the coronavirus word is Greek (κορώνη), i.e. crown or halo, which refers to the virus appearance, means Viral infection, under an electron microscope which is similar to a royal crown. That's why coronavirus is also referred to as the crowned virus [
]. The COVID-19 emerged as an epidemic disease in China, Wuhan City, in December 2019. Today, this has altered to a pandemic as a dangerous public health problem all around the world [
Yan, L.; Zhang, H.-T.; Xiao, Y.; et al. Prediction of criticality in patients with severe Covid-19 infection using three clinical features: a machine learning-based prognostic model with clinical data in Wuhan. MedRxiv.2020.
]. To date, seven of the 40 different species in the coronavirus family have been found to be transmitted to humans due to common diseases such as cold [
]. Previous studies have presented that some viruses such as SARS and MERS are transmitted from cats or camels to humans. It is thought that the COVID-19 has been transmitted from anteaters and bats to humans for the first time [
]. Common symptoms of this virus are dry cough, fever, and breath shortness. Also, muscle pain, sputum production, and sore throats are mild symptoms of the COVID-19 [
Sarkar, B.; Ullah, M. A.; Johora, F. T.; et al. The essential facts of Wuhan novel coronavirus outbreak in China and epitope-based vaccine designing against 2019-nCoV. BioRxiv.2020.
]. The structure of this virus consists of two outer and inner layers. The internal structure of COVID-19 includes the nucleus of the virus that contains genetic material. The outer layer of the virus is made of protein crowns [
]. The virus genome enters the cytoplasm after entering the host cell. According to studies, the incidence of coronavirus in men is higher than in women. Also, children are less likely to get the virus than adults [
]. According to the instructions of hospitalization published by the World Health Organization (WHO) recently, the essence of coronavirus must be verified by Reverse Transcription Polymerase Chain Reaction (RT-PCR) or a gene sequence as a key indicator for respiratory or blood specimen symptoms [
]. But, because of the sample collection, transportation limitations, and kit results, the overall positive RT-PCR rating for throat swab samples stated at the initial presentation is around 30-60% [
]. Furthermore, the long diagnosis time based on the RT-PCR kits causes many patients to not be quickly identified with COVID-19 as well as not receive appropriate treatment. Just because the RT-PCR kits are limited for the diagnosis of COVID-19. So, the patient may die due to a lack of necessary treatment to be hospitalized, the highly contagious nature of the virus, the low sensitivity as well as the duration for diagnosis based on RT-PCR Kits until the outcome of the diagnosis is known [
]. In addition, considering the extremely infectious nature of this virus, the patient carries a risk of infection to more people. Thus, the fast diagnosis of COVID-19 is essential for controlling and treating this disease [
]. Also, a scan of the chest compared to RT-PCR kits makes nurses less likely to be infected with the virus.
Among the chest imaging methods, Computerized Tomography (CT) and X-ray scans are prevalent. X-ray is a scan of the body for the symptoms of lung infections, fractures, tumors, and pneumonia. CT scan is a developed X-ray machine for providing clearer images from bones, tissue, and organs. The X-ray method is easier, quicker, and more economical than CT, but it is a more harmful method than that [
]. By taking X-ray images from the chest, doctors can visually diagnose Viral, Bacterial, COVID-19 infections, and so on. Visual diagnosis is usually a displeasing, time-consuming, and incorrect process. Because it needs expert human resources and can lead to low accuracy. Moreover, this type of image has certain weaknesses, such as overlapped organs, blurred barriers, and less contrast, which may not lead to a correct diagnosis of pneumonia [
]. So, according to these facts, recently the automatic detection of virus types including coronavirus on the basis of chest scanning imagery has attracted more attention. The automatic analysis of the COVID-19 causes the workload of hospital staff to decrease by a rapid diagnosis. This analysis is very effective for preventing mortality and treating timely. To date, different computational methods on the basis of chest scanning have been developed to observe, analyze and detect the COVID-19 automatically, which will be discussed below.
] detected pneumonia from X-ray images by three Deep Transfer Learning (DTL) networks – i.e. Inception v3, Inception-ResNet v2, and ResNet 50. In this research, the ResNet 50 model is the most considerable network among the others. In this network, the precision of the 2-stage classification algorithm is 98%. Loannis et al. [
] employed 1427 X-ray imagery for automatic classification of Bacterial, Viral, and COVID-19 pneumonia. They utilized five DTL networks including MobileNet v2, VGG, ResNet v2, Inception, and Xception to classify three pneumonia diseases. This study depicts that the VGG network is the best amongst others. The precision of 2-class classification – i.e. Healthy and covid19 – and 3-class classification – i.e. Viral, Bacterial, and COVID-19 – has been recorded 98.75% and 93.48%, respectively. Loannis et al. [
] also utilized deep Convolutional Neural Networks (CNNs) to separate pulmonary viruses from X-ray imagery, automatically. The precision of that for 2-class and 7-class classification on the basis of MobileNet is 99.18% and 87.66%, respectively. Pabira et al. [
Sethy, P. K.; Behera, S. K.; Ratha, P. K.; et al. Detection of coronavirus disease (COVID-19) based on deep features and support vector machine. arXiv preprint. 2020.
] have utilized a Deep Neural Network (DNN) approach to detect COVID-19, MERS, and SARS infections from X-ray imagery, automatically. Using ResNet 50 as well as Support Vector Machine (SVM), this model reached 95% accuracy in disease classification. Also, the Generative Adversarial Networks (GANs) have been used along with fine-tuned DTL infections for automatic pneumonia detection from chest X-ray images [
Khalifa, N. E. M.; Taha, M. H. N.; Hassanien, A. E.; et al. Detection of coronavirus (covid-19) associated pneumonia based on generative adversarial networks and a fine-tuned deep transfer learning model using chest x-ray dataset. arXiv preprint.2020.
] has shown that automatic pneumonia detection is practical with X-ray images, five DTL networks – i.e. DenseNET, AlexNet, Inception v3, GoogLeNet, and ResNet 18 – and Data Augmentation (DA) methods. The precision of the 2-class pneumonia classification in this model is 96%. Stephen et al. [
] have proposed an effective DL model for pneumonia classification by chest X-ray imagery. The architecture of the network includes 4 convolutional layers and 2 dense. These researchers achieved 93.7% accuracy in the 2-class classification of Healthy and pneumonia. In the following, a Transfer Learning (TL) method along with a deep residual network has been introduced for the automatic classification of 2-class pneumonia. This network includes 49-convolutional layers and 2-dense. Ultimately, they found 96.70% accuracy with 92.7% f1 score [
]. This network has 5 convolutional layers for feature extraction. The SVM, decision tree, and K-Nearest Neighbors (KNN) have also been used for the network classification section. Here, the best performance – i.e. accuracy, sensitivity, specificity and precision on the basis of SVM classifier is 98.97%, 89.39%, as well as 99.75%, and 96.72, respectively. The X-ray images for automatic COVID-19 detection have been used in many papers. Brunese et al. [
] have utilized a developed VGG16-TL network to detect two COVID-19 and Healthy classes. The precision of this report is about 98% and the detection time is about 2.49 sec. On the basis of CT scan images, Ardakani et al. [
] used DTL networks to recognize 2-class (non) COVID-19. Here, ten DTL networks including Xception, ResNet-50, VGG-16, ResNet-18, SqueezeNet, ResNet-101, AlexNet, MobileNet-V2, GoogleNet, and VGG-19 were used, and the Xception network reached hopeful results. The accuracy of this network reported about 99%. Jaiswal et al. [
] used the chest CT images using a combination of a DenseNet TL with convolutional networks for automatic diagnose of 2-class (non) COVID-19. In this study, the precision for separating reported approximately 96%. Horri et al. [
] have used three types of medical imagery– i.e. Ultrasound, CT, and X-ray – to identify 2-class Healthy and COVID-19 automatically. In this research, an enhanced VGG TL network has been used. The classification accuracy for Ultrasound, CT, and X-ray modes has been reported 100%, 84%, and 86%, respectively. Sharma et al. [
] presented a smart ontology-based IoT method for remote patient monitoring and early detection of COVID-19 using biomedical signals. This method provided the most up-to-date information on corona patients as well as identifiable data for remote monitoring. For detection, the aforementioned method achieved an accuracy of 96.33%. Le et al. [
] proposed a novel IoT enabled Depthwise separable Convolution Neural Network (DWS-CNN) with a Deep Support Vector Machine (DSVM) for COVID-19 detection. The DWS-CNN model was designed to detect both binary and multiple classes of COVID-19. This model's accuracy on binary and multiclass was reported to be 98.54% and 99.06%, respectively. Dansana et al. [
] used an X-ray and CT scan image dataset to train a CNN model-based VGG-19, Inception V2 and decision tree model for binary classification pneumonia. The validation accuracy of the fine-tuned version VGG-19, Inception V2, and decision tree model was found to be 91%, 78%, and 60%, respectively. Oh et al. [
] presented a patch-based CNN approach with a relatively small number of trainable parameters for COVID-19 diagnosis. The mentioned method was inspired by statistical analysis of the potential imaging biomarkers of the CXR radiographs. Experimental results showed that this method achieves state-of-the-art performance and provides clinically interpretable saliency maps, which are useful for COVID-19 diagnosis and patient triage. Nayak et al. [
] proposed a DL automated method for early detection of COVID-19 infection using X-ray images. They compared the performance of eight pre-trained CNN models, including AlexNet, VGG-16, GoogleNet, MobileNet-V2, SqueezeNet, ResNet-34, ResNet-50, and Inception-V3, in distinguishing COVID-19 from normal cases. The models were validated using publicly available chest X-ray images, and the best performance was obtained by ResNet-34, which had an accuracy of 98.33%. Wang et al. [
] suggested a novel DTL algorithm based on Pre-Trained Models (PTMs) to extract features. A selection algorithm of a pre-trained network for fusion to determine the best two models was designed in this method. Furthermore, deep chest CT fusion using discriminant correlation analysis was proposed to aid in the fusion of the two features from the two models. In four classes, this method achieved sensitivities of 95.61%, 96.25%, 98.30%, and 97.86%, respectively.
Review of pneumonia detection studies shows that, although many studies have been performed in this regard so far, there are limitations in these studies. In most of these studies, traditional methods or DNN approaches have been used for pneumonia detection. Traditional methods for extracting and selecting optimal features require specialized knowledge, whereas DNN approaches extract features hierarchically and can produce end-to-end learning. In most studies that are based on DNN approaches, the TL models are generally used to train them, which often have a high computational time in the training process. Furthermore, most of these studies have emphasized only on separation of 2-class Healthy and pneumonia. It means that the number of 2-class research done for pneumonia detection is higher than multi-class research conducted for that. As a result, it is essential to investigate more comprehensive Scenarios for the classification of various pneumonia types. According to these different issues, in the current study attempts have been made to overcome the related issues. From this standpoint, the current study presents a novel deep Convolutional Neural Network-Long Short Time Memory (CNN-LSTM) model with the proposed architecture that has been trained by scratch, not TL models, for extracting features hierarchically from raw data in order to detection of various pneumonia types. For this aim, X-ray imagery (based on different categories) from different databases that have been widely used in recent studies has been collected. The current research has examined 7 various Scenarios of Bacterial, Viral, COVID-19, and Healthy from the chest X-ray imagery in 4 classes to provide high accuracy to separate classes from each other. The fusion of the CNN and Long Short Time Memory (LSTM) networks can reduce feature dimensions, increase stability, improve the training process, increase the speed of convergence, and detection accuracy. The proposed model is an end-to-end classifier and it does not need any feature extraction as well as feature selection. Therefore, the optimal features of every class are learned with the deep CNN-LSTM model, automatically. Since noise is an evitable problem, in order to make more realistic assumptions and use the Proposed Method (P-M) in the practical field, white Gaussian noise is added to the raw chest X-ray imagery. In addition, the proposed network is tested and investigated based on two databases that were not used in the network training process. Finally, DTL networks commonly used for pneumonia detection, such as ResNet 50, Inception, Xception, and VGG 19, are used as comparisons. The main contributions of the present study are summarized as follows:
•
Designing a CNN-LSTM model to detect both binary and multiple classes of pneumonia.
•
Designing a network that is an end-to-end system and does not require any feature extraction or selection.
•
Fusion of the CNN and LSTM networks to reduce feature dimensions, increase stability, improve the training process, increase the speed of convergence, and detection accuracy.
•
Evaluating the proposed network in the presence of environmental noises.
•
Testing and evaluating the proposed network by using two databases that were not used in the network training process.
The rest of this paper is organized as follows. First of all, section 2 is about the database collected (on the basis of the chest X-ray imagery) and the mathematical background of CNN and LSTM networks. Section 3 presents the P-M. Section 4 exhibits the simulation results and compares the P-M with the common methods. In the end, the conclusion is presented in section 5.
Materials and methods
In this section, the dataset of chest X-ray imagery is examined first. Afterward, the mathematical backgrounds of CNN and LSTM are provided.
Datasets of chest X-ray images
This study has used chest X-ray imagery from 6 different databases that have been widely used in recent studies [
]. The dataset consists of the posterior/anterior chest image of cases with pneumonia. This dataset is divided into four categories. The following categories include 2923 images of Healthy people, 2840 images of people with Viral pneumonia, 2778 images of people with Bacterial pneumonia, and 371 images of people with COVID-19 pneumonia. Fig. 1. shows these four different categories. As you see, it does not exist any noticeable difference between COVID-19 and Viral groups. So, this disease is not recognized visually. In this study, 7 various Scenarios for chest X-ray imagery have been employed that are very useful in the medical field. These Scenarios are shown in Table 1.
Fig. 1The X-ray images of Chest for four groups (from left to right) including healthy, Bacterial, Viral, and COVID19, respectively.
] with more COVID-19 samples were used separately as test sets to evaluate the proposed algorithm. Datasets 1 and 2 are divided into three categories. Categories in Dataset 1 include 317 Healthy images, 855 Bacterial and Viral pneumonia images, and 116 COVID-19 pneumonia images, while categories in Dataset 2 include 939 Healthy images, 942 Viral pneumonia images, and 800 COVID-19 pneumonia images.
In total, data from 8 benchmark databases have been used to evaluate the proposed algorithm. Of these 8 databases, 6 were used together to assign training, validation and test sets. Also, 2 databases separately are only intended for the allocation of test sets (considering 6 datasets as training sets).
Convolutional neural network
CNN is considered as a stronger substitute for the conventional Neural Network (NN). It is very efficient to develop classification techniques based on machine vision [
]. The convolution layer output is known as the feature mapping. This work uses the max-pooling layer to choose the maximal values for feature map. Then, the dropout method is utilized to prevent overfitting; thus, every neuron is thrown out from this network at every training stage to decrease network. Then, for normalizing data in this network, the Batch Normalization (BN) layer is employed as follows [
where, in this BN layer, is an input vector. The is an output response accociated with a neuron at layer L. The , , and presents a little constant for numerical stability. The and are the scale and shift parameters, respectively, which are obtained by learning. After every layer, an activation function is used. In this study, two types of activation functions were used: Softmax and LeakyRelu. LeakyRelu (see Equation 2) is employed for the convolution layers due to being nonlinearity and sparseness.
(2)
The Softmax activation function computes the likely distribution of output classes. So, it is employed in the last FC layer as follows:
(3)
where is the input vector. The output values, , are between 0 and 1, and their summation is equal to 1 [
Deep neural networks–based damage detection using vibration signals of finite element model and real intact state: An evaluation via a lab-scale offshore jacket structure.
A Recurrent Neural Network (RNN) is widely utilized for dealing with variable-length sequence inputs. A recurrent hidden vector is responsible for storing the long-distance history and it has dependency upon on before hidden vector [
]. The duty of LSTM is to solve the RNN instability and gradient vanish problems. This network is used to compute a balanced summation of input signals. Next, by passing as an activation function , every LSTM unit utilizes memory at time . So, we have , where is the output gate for monitoring the amount of content resulted in memory. The calculation of the output gate is done by Equation 4 [
where and are the activation function, initial weight matrix and initial bias vector, respectively. Equation 5 is used to update the memory cell, , where is the content of the new memory obtained by Equation 6 as follows [
The current memory is not handled by forgetfulness gate,, and that of new memory content must be added up to the memory cell to be controlled by updating entrance gate, . This work is done by Equations 7-8 [
In this section, the P-M is presented based on DNNs to automatic detection of pneumonia (Bacterial, Viral, COVID-19, and Healthy) from the chest X-ray imagery. The block diagram of the P-M has been depicted in Fig. 2.
Fig. 2The block-diagram of the P-M for automatic detection of pneumonia (Bacterial, COVID-19, Viral and Healthy).
The preprocessing operations on chest X-ray imagery have three various steps that this section describes them. All images are first converted to RGB format. They are then resized to 224 × 224 × 3, due to the fact that databases have different colors and formats. Afterward, they are normalized between 0 and 1. Because the COVID-19 cases are limited, the number of chest X-ray images for each group is not balanced, as shown in Section 2.1. As a result, it can lead to poor classification performance as well as overfitting issues; thus, the DA method is used to dominate these issues. As a result, in the third step, the number of imagery of the COVID-19 group is artificially increased. This process improves the generalization ability of the model during training. The rotation range refers to the range of randomly rotated images during training, which is considered to be 40 degrees in the P-M. Also, both the width shift – i.e. the horizontal translation of the imagery – and height shift – i.e. the vertical translation of the images – are 0.2%. The number of COVID-19 samples before and after DA is 371 and 2842, which is nearly equal to the other groups.
Proposed network
The proposed deep model is designed by fusing CNN and LSTM networks. The benefits of both networks can be used simultaneously by fusing these networks. The fusion of LSTM networks with CNN networks has been used in many studies to reduce feature dimensions, increase stability, improve the training process, increase the speed of convergence, and detection accuracy [
]. According to this, in the proposed network's architecture, the fusion of 5-convolution 2-D layers and 3-LSTM layers has been used to detect automatically pneumonia (Bacterial, Viral, COVID-19, and Healthy) from chest X-ray imagery. To implement the proposed CNN-LSTM model, a cross-library is used by the Python programming language. Here, the selection of the proposed architecture is done by 1) a dropout layer. 2) a convolution layer with nonlinear Leaky-Relu function, next a max-pooling layer followed by a BN layer, 3) before step is rehearsed 4 times, 4) the output of the before step is connected to a 2D matrix, 5) also, the output of before step connects to 3 layers of LSTM with Leaky-Relu nonlinear functions in series, then these layers are followed by a BN layer, and 6) two FC layers are employed to access the output layer. In the following, Table 2 depicts the number of filters, the size of the strides, and the architectural details of the proposed network. As you see in Table 2, the dimensionality decrease of hidden layers continues from 224 × 224 × 3 to 50. In the end, the selected feature vector is connected to the FC layer with the nonlinear Softmax function. The architectural details of the proposed algorithm are shown in Fig. 3. Here, two main interferences have an effect on the proposed CNN-LSTM model, the wide kernel in the first convolution layer and small kernels in the remaining convolution layers. The first interference enables to removal high frequency noises in contrast to small kernels, and the second interference can present input features well; so, these interferences enhance the overall network performance.
Table 2The size of filters and steps recommended for the proposed model.
All parameters of the proposed CNN-LSTM model are carefully modified using the trial and error method in order to achieve the best rate of convergence. Different types of optimizers and different numbers and sizes of filters were used in the design of the proposed network architecture. At last, the training process is accomplished by the cost function, Mean Squared Error (MSE) [
The overall number of samples and their allocation for training, validation, and testing sets for every Scenario are shown in Table 4. As it is shown in Table 4, 70% of samples are randomly chosen for the training process, 10% of samples chosen for the validation process, and the remaining 20% are chosen for the testing process.
Table 4The number of samples and their allocation in the proposed model.
In this section, the P-M results for the pneumonia automatic detection from the chest X-ray imagery based on the CNN-LSTM model are presented and discussed. All of simulations were carried out on the Google Collaborator system with 14 GB of RAM plus Tesla K80 GPU graphics card.
To investigate the performance of the P-M, the proposed CNN-LSTM model error and accuracy (based on validation data) for Scenarios I and VII, as well as the confusion matrix, t-SNE charts, and bar chart diagram of precision, sensitivity, accuracy, and specificity (based on testing data) for all Scenarios (I-VII) are provided in Fig. 4. According to Fig. 4(a), the network error for Scenario I decreases with the number of iterations, and after 40 iterations, it reaches its steady-state value, and the network error eventually decreases from 0.1929 to 0.0024. Furthermore, in Fig. 4(a), the steady-state value of Scenario VII is on its 130th iteration, and the network error eventually decreases from 0.1186 to 0.0426. Furthermore, the network's accuracy for 2-class (Scenario I) and 4-class (Scenario VII) pneumonia classification is 99.42% and 91.70%, respectively. As you see in Fig. 4(b), the network's accuracy for all Scenarios is above 90% except Scenario V (i.e. Healthy against of Covid-19 against of Viral). This could be because the viral and Covid-19 images look so similar and are more difficult to distinguish than in other cases. In the Scenario I associated with the 2-class classification of Healthy and COVID-19, 563 COVID-19 samples from the testing set are well identified, while only 2 samples are misdiagnosed. Also, the final accuracy of the COVID-19 class separation from the Healthy class is approximately 99%, which is very promising. The visualization of the feature representations of all the testing samples extracted from the last FC layer via t-SNE method is given in Fig. 4(c). The feature representations are depicted as a scatter diagram with distinct colors, which express different classes. According to Fig. 4(c), as can be seen from the visualization of the last FC layer, almost all samples are separated from each other for Scenarios I and VII, and this indicates that the proposed model is able to extract high-level features from raw data and relatively has a good performance in discriminating of different classes. Fig. 4(d) shows that the values of all classification Scenarios are very promising. The results of precision, specificity, accuracy and sensitivity separation of the Healthy group from COVID-19 (Scenario I) are about 99%.
Fig. 4The proposed CNN-LSTM model error and classification accuracy (based on validation data) for Scenarios I and VII, as well as the confusion matrix, t-SNE charts, and bar chart diagram of precision, sensitivity, accuracy, and specificity (based on testing data) for all Scenarios (I-VII); (a) Error and accuracy, (b) Confusion matrix, (c) t-SNE charts, (d) Bar chart diagram.
Fig. 4The proposed CNN-LSTM model error and classification accuracy (based on validation data) for Scenarios I and VII, as well as the confusion matrix, t-SNE charts, and bar chart diagram of precision, sensitivity, accuracy, and specificity (based on testing data) for all Scenarios (I-VII); (a) Error and accuracy, (b) Confusion matrix, (c) t-SNE charts, (d) Bar chart diagram.
Fig. 4The proposed CNN-LSTM model error and classification accuracy (based on validation data) for Scenarios I and VII, as well as the confusion matrix, t-SNE charts, and bar chart diagram of precision, sensitivity, accuracy, and specificity (based on testing data) for all Scenarios (I-VII); (a) Error and accuracy, (b) Confusion matrix, (c) t-SNE charts, (d) Bar chart diagram.
To more investigate the performance of the P-M based on the CNN-LSTM model, Scenario VII – i.e. Healthy against of COVID-19 against of Bacterial against of Viral – is simulated using four DTL networks. ResNet 50 [
] are four comparative DTL networks that have been extensively used in recent studies. Fig. 5 shows the proposed model's accuracy in comparison to DTL networks for various iterations (See Fig. 5(a)), as well as the bar chart diagram of each DTL network's sensitivity, specificity, accuracy, and precision (See Fig. 5(b)). From Fig. 5(a), after 150 iterations, the accuracy of the proposed CNN-LSTM model, Inception, ResNet 50, Xception, and VGG 19 reaches 91%, 86%, 84%, 83%, and 80%, respectively. Comparing performance of the proposed model and DTL models shows that proposed model, provides better results than DTL models. Also, the proposed model converges to the desired value faster of DTL models (See Fig. 5(a)). These results are highly related to the proposed CNN-LSTM model's unique architecture, which can automatically extract useful features from raw images layer by layer and achieve higher accuracy at a faster rate. The running time of testing and training phases is given for the classification of all Scenarios in Table 5. As shown in Table 5, the running time of the testing and training phases for Scenario VII is longer than for the other Scenarios. In addition, the running time of the Scenario VII based on the proposed CNN-LSTM model comparing the ResNet 50, VGG 19, Xception, and Inception v3 in 150 iterations is depicted in Table 6. As can be seen from Table 6, the running time of the proposed model for both phases is less than other comparative methods. This is due to the fact that DTL networks frequently have a high computational time.
Fig. 5The performance of the proposed model in comparison to DTL networks, as well as the bar chart diagram of the sensitivity, specificity, accuracy, and precision of each DTL network; (a) Accuracy of the proposed network comparing with DTL networks, (b) Bar chart diagrams.
To make more realistic assumptions, white Gaussian noise with varying Signal-to-Noise Ratios (SNRs) is added to the raw chest X-ray imagery in testing phase which is depicted in Fig. 6. Also, the classification accuracy of each model (P-M, ResNet 50, VGG 19, Inception, and Xception) for every SNR has been reported in this Figure. As shown in Fig. 6(b), the testing accuracy of the proposed model, ResNet 50, VGG 19, Inception, and Xception is robust to observation noise up to 0 dB, 10 dB, 20 dB, 10 and 20 dB, respectively, such that their accuracy is still approximately 90%, 80%, 78%, 80%, and 80%. As a result, the proposed model, in addition to being more accurate than comparable models, is more resistant to observation noise. It is due to using the wide kernel at the first convolution layer which enables to removal high frequency noises in contrast to small kernels.
Fig. 6The chest X-ray imagery along with white Gaussian noise in the various ranges of SNR and also the classification accuracy of each network (P-M, ResNet 50, VGG 19, Inception, and Xception) for every SNR; (a) Chest X-ray imagery along with white Gaussian noise, (b) The classification accuracy of each network.
In previous literature, the training set and testing set have been selected from one or more common datasets. However, in order to use the proposed model in the practical field, the proposed model should be evaluated with the benchmark databases which are not used as the training data. To this end, by considering the 6 training databases, the performance of the proposed model has been assessed [
]. It should be noted that no information from Databases 1 and 2 have been used to train the proposed model. In fact, the chest X-ray imagery from 6 different databases is used simultaneously to train the proposed model, and Datasets 1 and 2 are used separately to evaluate this model. The t-SNE chart, confusion matrix of the last FC layer and the bar chart diagram of sensitivity, specificity, accuracy, and precision of the proposed model, for Datasets 1 and 2 (which only have been used as the testing set) are shown in Fig. 7. According to this Figure, the testing accuracy of the proposed model based on Dataset 1 and Dataset 2 is nearly 95% and 92%. From this standpoint, it can be stated that the proposed model can provide good performance for Datasets 1 and 2 even when it is trained with other 6 different databases. In fact, using 6 different databases as a training set causes the proposed model almost to be generalized. According to this, it can be said that the proposed model can perform very well when tested with different databases which are not used in the training phase.
Fig. 7The t-SNE chart, confusion matrix and and the bar chart diagram of sensitivity, specificity, accuracy, and precision of the proposed network for Datasets 1 and 2; (a) The t-SNE chart and confusion matrix for Datasets 1, (b) The t-SNE chart and confusion matrix for Datasets 1, (c) The bar chart diagram.
Lastly, the performance of the proposed model compared with the recent researches is given in Table 7. This table demonstrates that the proposed approach outperforms recent approaches in terms of the number of classes classified and the Scenarios studied. However, due to the difference between datasets, methods, categories, and various simulation environments, a one-to-one comparison usually is not achievable.
Table 7Comparing the performance of the P-M with recent studies.
Khalifa, N. E. M.; Taha, M. H. N.; Hassanien, A. E.; et al. Detection of coronavirus (covid-19) associated pneumonia based on generative adversarial networks and a fine-tuned deep transfer learning model using chest x-ray dataset. arXiv preprint.2020.
It should be highlighted that although the databases used for this study are reliable, the data related to COVID-19 is limited. However, the above-mentioned findings and results of the paper should be considered in line with the limitation of the study. This study has some limitations with respect to databases. A notable difficulty and challenge is related to the COVID-19 data that researchers should use a number of data sets in combination with together to remove this defect. However, a shortage of COVID-19 samples is still evident. Here, to overcome the shortage of COVID-19 samples and in order to balance data, DA technique has been utilized. In this study, classical DA was used to increase data of COVID-19, and it is necessary to perform DA using GANs and compare the results with classic DA results. Another limitation and challenge is the number of database categories. In this study, there were a total of four detection categories: Healthy, Viral, Bacterial, and COVID-19. In order to use the proposed model in the practical fields, it must be evaluated using larger datasets and more categories of other lung infectious diseases, such as SARS, MERS, FLU, and so on.
Conclusion
Due to the COVID-19, all humans' daily lives have recently been confronted with new challenges. The number of infectious people with this virus continues to increase, significantly. So, rapid detection of this virus, as well as timely treatment, is very crucial. In this paper, using a fusion of LSTM and CNN networks, a novel method for the automatic detection of pneumonia is presented. In this method, chest X-ray imagery is utilized to separate 2-4 classes based on 7 various Scenarios. In the proposed method, in all Scenarios except Scenario V, i.e. Healthy versus Covid-19 versus Viral, above 90% accuracy is achieved, which is very promising in contrast with the newest pneumonia detection approaches. Furthermore, the proposed CNN-LSTM model is compared with four DTL models – ResNet 50, VGG 19, Xception, and Inception – that have been widely used for pneumonia detection. The results revealed that the proposed model outperformed the comparative models in terms of speed and accuracy. To evaluate the proposed model versus observation noises, white Gaussian noise is added to the raw chest X-ray imagery. The proposed model is resistant to observation noise up to 0 dB. It is also expected that using this technique will decrease medical cost, the incidence of nurses/doctors to COVID-19 during swab sampling, and future fatality.
References
Roosa K.
Lee Y.
Luo R.
et al.
Real-time forecasts of the COVID-19 epidemic in China from February 5th to February 24th, 2020.
Yan, L.; Zhang, H.-T.; Xiao, Y.; et al. Prediction of criticality in patients with severe Covid-19 infection using three clinical features: a machine learning-based prognostic model with clinical data in Wuhan. MedRxiv.2020.
Sarkar, B.; Ullah, M. A.; Johora, F. T.; et al. The essential facts of Wuhan novel coronavirus outbreak in China and epitope-based vaccine designing against 2019-nCoV. BioRxiv.2020.
Sethy, P. K.; Behera, S. K.; Ratha, P. K.; et al. Detection of coronavirus disease (COVID-19) based on deep features and support vector machine. arXiv preprint. 2020.
Khalifa, N. E. M.; Taha, M. H. N.; Hassanien, A. E.; et al. Detection of coronavirus (covid-19) associated pneumonia based on generative adversarial networks and a fine-tuned deep transfer learning model using chest x-ray dataset. arXiv preprint.2020.
Deep neural networks–based damage detection using vibration signals of finite element model and real intact state: An evaluation via a lab-scale offshore jacket structure.
00011-X&id=gr1.jpg){kind=link}
00011-X&id=gr2.jpg){kind=link}
00011-X&id=gr3.jpg){kind=link}
00011-X&id=gr4a.jpg){kind=link}
00011-X&id=gr4b.jpg){kind=link}
00011-X&id=gr4c.jpg){kind=link}
00011-X&id=gr5.jpg){kind=link}
00011-X&id=gr6.jpg){kind=link}
00011-X&id=gr7.jpg){kind=link}