{"id":60187,"date":"2024-09-30T11:06:07","date_gmt":"2024-09-30T11:06:07","guid":{"rendered":"https:\/\/biomedpharmajournal.org\/?p=60187"},"modified":"2024-10-09T18:21:14","modified_gmt":"2024-10-09T18:21:14","slug":"enhancing-skin-disease-diagnosis-with-tffnet-a-two-stream-feature-fusion-network-integrating-cnns-and-self-attention-block","status":"publish","type":"post","link":"https:\/\/biomedpharmajournal.org\/staging\/vol17no3\/enhancing-skin-disease-diagnosis-with-tffnet-a-two-stream-feature-fusion-network-integrating-cnns-and-self-attention-block\/","title":{"rendered":"Enhancing Skin Disease Diagnosis with TFFNet: A Two-Stream Feature Fusion Network Integrating CNNs and Self Attention Block"},"content":{"rendered":"\n

Introduction<\/strong><\/p>\n\n\n\n

Melanoma, a\nhighly aggressive skin cancer, accounts for only 1% of skin cancers, yet it is\nthe leading cause of death 1<\/sup>. Computer-aided methods for skin cancer\ndetection are necessary due to a shortage of dermatologists per capita. The\nAmerican Cancer Society predicts 99,780 new melanoma cases (57,180 men and\n42,600 women) and 7,650 deaths (5,080 men and 2,570 women) in 2022. As the\nfield of computer vision and Artificial Intelligence has advanced, image\nanalysis has become increasingly useful in a wide range of scene-parsing\napplications. Computer-assisted diagnosis and detection heavily rely on medical\nimage analytics 2<\/sup>. Early disease detection and diagnosis are major\nchallenges in healthcare. Only then can appropriate therapy begin. Millions of\nindividuals around the world are affected by skin diseases today, which can be\ndetrimental to both personal health and national economies if not addressed\npromptly 3<\/sup>. In 2021, according to the American Cancer Society, 7,180\nindividuals died from melanoma. Additionally, the American Cancer Society\npredicted in their 2022 annual report that there would be roughly 99,780 new\ninstances of skin disease (melanoma), with an expected death rate of 7,650\npeople 4<\/sup>. Diseases that produce itching or pain, on the other hand,\nmight lead to substantial damage and deformation. Damage to the skin from these\ndisorders can also affect a person’s sense of well-being and confidence 5<\/sup>.\nThe common perception is that some skin diseases are rather harmless. However,\nthe vast majority of sufferers opt to treat their skin issues on their own.\nMedications for skin diseases can worsen the condition if they are not\neffective against the underlying cause. Perhaps the individual is unaware of\nthe severity of their skin issue 6<\/sup>.<\/p>\n\n\n\n

Examining\ndermoscopic images is the best standard for identifying skin diseases.\nDermatologists utilize various dermoscopic tools, including the pigment\nnetwork, dots\/globules, and color regression, to make diagnoses from dermoscopy\nimages. However, this method has several drawbacks, such as the need for\nadvanced dermoscopic equipment and the time and effort required to train\ndermatologists in using these tools 7, 8<\/sup>. Additionally, the\ninflammatory nature of skin diseases and overlapping characteristics of\ninfectious diseases result in considerable visual variation and irregularities\nin the overall appearance and feel of skin lesions. Inexperienced\ndermatologists often struggle to identify subtle variations using their eyes\nalone. Recent advancements in artificial intelligence, particularly in the\nhealthcare industry, focusing on the analysis of medical images, have made it\nan attractive tool for developing algorithms for medical image interpretation.\nThis is especially true in the context of the medical industry, where machine learning\nnetworks have proven to be very useful in image analysis due to their ability\nto independently learn image representations. Dermatologists have a critical\nneed for computer-aided design (CAD) systems based on innovative\nproblem-solving approaches 9<\/sup>. This would not only alleviate the\nstrain on the nation’s healthcare infrastructure but also reduce the waiting\ntime for medical dermoscopy. Convolutional neural networks (CNNs) and other\nforms of deep learning have demonstrated superiority over conventional methods\nin human disease diagnosis. The availability of powerful computational\nresources has led to the continuous development of more advanced deep learning\nsystems. Nevertheless, due to the extensive training time required, these\nintricate systems could occasionally be wasteful. Due to their ability to\nachieve accurate results with fewer parameters and less effort, CNN models have\ngained popularity. In this study, the MobileNetV2 and NASNetMobile backbone architectures are employed to categorize skin diseases.<\/p>\n\n\n\n

Below is the\noutline for the remainder of the paper. We reviewed the studies that have been\nconducted on the topic of skin disease classification in Section 2. The\nmethodological approach and overall structure of the model are detailed in\nSection 3. In Section 4, we examine the training and validation processes for\nthe model. The proposed work concludes with some last notes and an outline of\npotential future work in Section 5.<\/p>\n\n\n\n\n\n
\"\"<\/td>\n\n

Figure 1: Shows the proposed method\u2019s workflow.<\/strong><\/p>\n

<\/p>\n

Click here to view Figure<\/a><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n\n\n

Literature Review <\/strong><\/p>\n\n\n\n

Skin diseases are\na major health risk for humans. The diagnosis might be impacted by factors such\nas high sensitivity, the need for laborious laboratory procedures, considerable\ntime investment, and intricate physical manipulation. Furthermore, the similarity\namong many skin lesions often leads to frequent misidentifications 10<\/sup>.\nThis work aimed to construct a unified CAD model for segmenting and classifying\nskin lesions using a deep learning architecture. At the outset of this\nprocedure, source dermoscopic images are pre-processed using a variant of a\nbio-inspired multiple exposure fusion method that emphasizes contrast\nenhancement. The second step is to create a bespoke CNN architecture with 26\nlayers specifically for the task of identifying and isolating skin lesions.\nFinally, four CNN models are learned from the segmented lesion images\n(ResNet-50, Xception, VGG16, and ResNet-101). Finally, a convolutional sparse\nimage decomposition fusion method is used to combine the deep feature vectors\nthat were obtained from each CNN model. In the last stage, the ideal features\nare chosen for classification using univariate measurements and the Poisson\ndistribution feature selection method. The final step in the classification\nprocess involves employing a multi-class support vector machine with the\nselected features.<\/p>\n\n\n\n

The first steps\ninvolve using a lightweight attention module to identify feature correlations,\nfine-tuning a pre-trained model (ResNet-50) on the HAM10000 dataset to extract\nlatent high-level features, increasing the number of samples from\nunderrepresented groups using synthetic minority class oversampling, and\nfeeding those into an XGBoost model for training and prediction 11<\/sup>.\nThis combination of high-level attributes and generic statistics will be\nemployed. A hybrid network with multi-scale Gaussian difference preprocessing,\ndual-stream convolutional neural networks, and transformers 12<\/sup> is\nused to reliably separate skin lesions found with dermoscopy. To cautiously\nimprove the lesion area and edge information and eliminate noisy features like\nhair, three Gaussian difference convolution kernels were trained. By utilizing\nmulti-scale Gaussian convolution, the model can effortlessly extract and\nincorporate edge and lesion information while simultaneously reducing noise.\nSecondly, for accurate alignment, a dual-stream network is employed to extract\nfeatures from both the original image and the Gaussian difference image\nseparately. Then, these features are fused in the feature space. Combining\nmodels from vision transformers with convolutional neural networks enhances\ndata consumption on a local and global scale. Lastly, self-attention and\ncoordination techniques are utilized to make important aspects more noticeable.\nA densely connected Res2Net and feature fusion attention module-based approach\nis proposed for gesture image recognition 13<\/sup>. Using dense\nconnections and group convolution, they propose the densely connected Res2Net\nto improve upon Res2Net. By using SK-Net to choose features, densely connected\nRes2Net is made more adaptable to the receptive field. The resulting network is\nused to extract features from high- and low-level gesture images. The FFA was\ncreated to combine high-level and low-level features and eliminate superfluous\ndata from features.<\/p>\n\n\n\n

The AlexNet model\n14<\/sup> was modified by changing the activation function to detect skin\ntumors in the HAM10000 dataset. F-score, accuracy, and recall all reached a new\nhigh of 98.20%. To classify skin lesions, an ensemble model 15<\/sup> was\nintroduced, combining stacked ensemble techniques from Inceptionv3, Xception,\nDenseNet121, DenseNet201, and InceptionResNet-V2, based on fine-tuning and\ntransfer learning. Compared to state-of-the-art approaches, the proposed model\nachieved a higher accuracy of 97.93%. They initiated the image classification\nprocess by introducing their innovative ESRGAN preprocessing strategy for ISIC\n2018 images. Subsequently, they applied different deep learning models 16<\/sup>,\nachieving an overall accuracy of 83.2%. Notably, CNN, Resnet-50 (83.7%),\nInceptionV3 (85.8%), and InceptionResnet (84%) contributed to this success. The\npre-trained versions of the deep learning models MobileNetV2 and DenseNet201\nwere improved by adding more convolution layers, which allowed for more\naccurate diagnosis of skin cancer 17<\/sup>. In the most recent iteration,\nboth models have three convolutional layers stacked on top of one another. The\napproach that has been presented has the potential to distinguish between\nbenign and malignant types. With an accuracy of 95.50 percent, the modified\nversion of the DenseNet201 model that was proposed beats both the\nstate-of-the-art baselines and the state-of-the-art approaches from the most\nrecent literature review. Additionally, the enhanced sensitivity of the\nDenseNet201 model is 93.96%, while its specificity is 97.03%.<\/p>\n\n\n\n

A lightweight\nmodel capable of accurately diagnosing skin lesions was proposed 18<\/sup>.\nEven though this results in a very small number of trainable parameters, the\nemployment of dynamically scaled kernels in the layers is what allows for the\nachievement of optimal outcomes. Within the framework of the suggested\nparadigm, the activation functions ReLU and leaky ReLU are both put to use. The\nmodel correctly categorized every class included in HAM-10000, with a success\npercentage of 97.85% overall. Employing HAM-10000 19<\/sup>, researchers\ntested 11 distinct CNN models by using seven skin disease classes. They\naddressed the problem of imbalance and the striking similarities between images\nof different skin diseases by employing transfer learning, fine-tuning, and\ndata augmentation. DenseNet169 emerged as the top-performing algorithm out of\n12 different CNN architecture variants. The system achieved 92.25% accuracy,\n93.59% sensitivity, and a 93.27% F1-score. The framework developed for\nautomating the SLC process of dermoscopy is named “Dermo-Expert” 20<\/sup>.\nThe preprocessing step and the convolutional stage are both included in the\nhybrid CNN’s processing pipeline. To develop more precise lesion feature maps,\nthe hybrid CNN presented uses three distinct feature extractor modules. After\ncategorizing the various feature maps using a variety of completely linked\nlayers, the resulting maps are assembled to provide a prediction regarding the\ntype of lesion. Lesion segmentation, augmentation (based on geometry and\nintensity), and class rebalancing are all components of their proposed\npreprocessing step for their methodology. These features include imposing a\ncost on the decline of every class and combining additional graphics with the\nunderrepresented groups. After being put through its paces on the ISIC-2017,\nISIC-2018, and HAM-10000 datasets, Dermo-Expert earned an AUC of 0.96, 0.95,\nand 0.97, respectively.<\/p>\n\n\n\n

These points\nsummarize the problems with existing studies. To begin with, most of the\nresearch currently available relies on unprocessed visualizations for the\npurpose of identifying skin diseases, which is inefficient and inaccurate.\nSecond, the importance of combining multiple features for skin disease\ndetection is often under-researched. To overcome the limitations of prior\nresearch, we propose a TFFNet (Two-Stream Feature Fusion Network) for skin\ndisease diagnosis that can classify a wide range of skin diseases. The proposed\napproach takes both color and grayscale images and extracts the most relevant features\nfrom each.<\/p>\n\n\n\n\n\n
\"\"<\/td>\n\n

Figure 2: The architecture of the proposed network TFFNet<\/strong><\/p>\n

<\/p>\n

Click here to view Figure<\/a><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n\n\n

Table 1: Fused Layers with SAConv and DWSC modules.<\/strong><\/p>\n\n\n\n\n\n\n\n\n\n\n
\n

Stage<\/strong><\/p>\n<\/td>\n

\n

M1<\/strong><\/p>\n<\/td>\n

\n

M2<\/strong><\/p>\n<\/td>\n

\n

stride<\/strong><\/p>\n<\/td>\n

\n

Layers<\/strong><\/p>\n<\/td>\n<\/tr>\n

\n

0<\/p>\n<\/td>\n

\n

DWSC1 3\u00d73<\/p>\n<\/td>\n

\n

–<\/p>\n<\/td>\n

\n

1<\/p>\n<\/td>\n

\n

1<\/p>\n<\/td>\n<\/tr>\n

\n

1<\/p>\n<\/td>\n

\n

DWSC2 5\u00d75<\/p>\n<\/td>\n

\n

–<\/p>\n<\/td>\n

\n

1<\/p>\n<\/td>\n

\n

2<\/p>\n<\/td>\n<\/tr>\n

\n

2<\/p>\n<\/td>\n

\n

DWSC3 3\u00d73<\/p>\n<\/td>\n

\n

Conv1 5\u00d75<\/p>\n<\/td>\n

\n

2<\/p>\n<\/td>\n

\n

4<\/p>\n<\/td>\n<\/tr>\n

\n

3<\/p>\n<\/td>\n

\n

DWSC4 5\u00d75<\/p>\n<\/td>\n

\n

Conv2 7\u00d77<\/p>\n<\/td>\n

\n

2<\/p>\n<\/td>\n

\n

6<\/p>\n<\/td>\n<\/tr>\n

\n

4<\/p>\n<\/td>\n

\n

DWSC5 3\u00d73<\/p>\n<\/td>\n

\n

Conv3 5\u00d75<\/p>\n<\/td>\n

\n

2<\/p>\n<\/td>\n

\n

8<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n

Research gaps and motivation<\/strong><\/p>\n\n\n\n

In terms of skin disease identification, the following research gaps have been identified<\/p>\n\n\n\n

The majority of research has been conducted using small datasets 5 6<\/sup>. Therefore, there is a need for analysis using large datasets to enhance the performance of trained models.<\/p>\n\n\n\n

Using conventional image processing methods to identify and extract disease-specific traits from skin examinations is a daunting task.<\/p>\n\n\n\n

As every skin disease has its own unique characteristics, automatic feature extraction is necessary to improve classification accuracy. However, this incurs a considerable amount of computation. Although the vast majority of deep learning models offer automated feature learning, tailored deep architectures are required to resolve the trade-off between complexity and accuracy.<\/p>\n\n\n\n\n\n
\"\"<\/td>\n\n

Figure 3: Predicting skin diseases using self-attention using CNN.<\/strong><\/p>\n

<\/p>\n

Click here to view Figure<\/a><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n\n\n

Research contributions<\/strong><\/p>\n\n\n\n

The primary research contributions of the proposed effort are as follows:<\/p>\n\n\n\n

This study establishes a new architecture known as “TFFNet” by modifying the conventional CNN design and creating two parallel modules: the CNN with a Self-Attention (SA) block 21<\/sup> and the Depthwise Separable Convolution (DWSC) module. After implementing these changes, the overall number of trainable parameters dropped significantly. The proposed approach learned more than seven million characteristics to identify diseases, surpassing other deep learning approaches detailed in the literature.<\/p>\n\n\n\n

The proposed architecture achieved good classification accuracy while utilizing minimal processing resources. In contrast to CNN models such as MobileNetV2 and NASNetMobile, these networks employ appropriate customization to address the complexity versus accuracy trade-off.<\/p>\n\n\n\n

We used three datasets and 13,894 images to depict various skin diseases in this work. After developing six data pre-processing correction methods for image enhancement, we progressively merged the unique information from each modality.<\/p>\n\n\n\n

Table 2: Specifics of the data set used.<\/strong><\/p>\n\n\n\n\n\n\n\n\n
\n

Data-set <\/strong><\/p>\n<\/td>\n

\n

Classes<\/strong><\/p>\n<\/td>\n

\n

Test set<\/strong><\/p>\n<\/td>\n

\n

Training set <\/strong><\/p>\n<\/td>\n<\/tr>\n

\n

ISIC 2016<\/p>\n<\/td>\n

\n

Benign, Malignant<\/p>\n<\/td>\n

\n

379<\/p>\n<\/td>\n

\n

900<\/p>\n<\/td>\n<\/tr>\n

\n

ISIC 2017<\/p>\n<\/td>\n

\n

Benign nevi, Melanoma, Seborrheic keratosis<\/p>\n<\/td>\n

\n

600<\/p>\n<\/td>\n

\n

2000<\/p>\n<\/td>\n<\/tr>\n

\n

HAM10000<\/p>\n<\/td>\n

\n

Basal cell carcinoma, Actinic keratoses, Dermatofibroma, Benign keratosis,<\/p>\n

Melanocytic nevi, Melanoma, Vascular lesions<\/p>\n<\/td>\n

\n

3004<\/p>\n<\/td>\n

\n

7011<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n

Methodology<\/strong><\/p>\n\n\n\n

The system’s proposed workflow is illustrated in Figure 1, providing an overview of the entire process.<\/p>\n\n\n\n

Proposed network’s architecture<\/strong><\/p>\n\n\n\n

The system’s proposed workflow is illustrated in Figure 1, providing\nan overview of the entire process. Skin disease classification has recently\nutilized various cutting-edge CNN models. This investigation incorporates\nMobileNetV2 and NASNetMobile as foundational architectures. These models have\ndemonstrated superior performance with reduced computational complexity in\naddressing a range of computer vision challenges compared to alternative\nmethods. The key components include SACNNs and DWSC modules. The newly proposed\nnetwork, named ‘TFFNet,’ integrates existing CNN architectures through the\nincorporation of two-stream feature fusion modules. Figure 2 illustrates the\ncomprehensive design of the proposed model. In contrast to traditional CNN\nnetworks, it aims to provide accurate categorization while minimizing the number\nof parameters involved. Table 1 provides a breakdown of the 21 layers. Here, we\nwill discuss the importance of each module in the proposed architecture.<\/p>\n\n\n\n

DWSC Module<\/strong><\/p>\n\n\n\n

Initially, Sifre 22<\/sup> proposed DWSC, which found application in image classification. The concept entails decomposing the convolution operation, a strategy known as DWSC. This approach converts a conventional convolution operation into a blend of depthwise separable and pointwise convolution operations. In the depthwise separable convolution process, each input channel undergoes filtration independently, and the resultant linear input channels are subsequently merged. This convolutional method substitutes a solitary convolutional layer with two distinct layers \u2014 one for spatial filtering and the other for merging purposes. Consequently, depthwise separable convolution effectively reduces both the model’s size and the parameter count. By combining depthwise separable and pointwise convolution, this method turns a regular convolution into something new. Prior to merging the linear input channels, the separable convolution procedure applies a separate filter to each channel input. Rather than using a single convolution layer, this convolutional method splits the processing into two distinct layers: one to perform spatial filtering and another to combine results. Both the size of the model and the number of parameters can be successfully reduced using depthwise separable convolution. When it comes to input feature maps, however, a typical convolution kernel just requires three parameters: the height (H), width (W), and input channel (Ic<\/sub><\/em> ). With Oc<\/sub><\/em> standing for the number of output channels, the resulting convolution layer (h \u00d7 w \u00d7 Ic<\/sub><\/em> )  is depicted as K x K x Ic<\/sub> x Oc<\/sub><\/em>. Two crucial operations are involved in depthwise separable convolution: the depthwise separable convolution operation and the pointwise convolution operation. To put it mathematically, the operation of depthwise separable convolution is:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

In the context of the depthwise separable convolution operation, K denotes the kernels characterized by dimensions K x K x Ic<\/sub>. The input feature map (I) is used to regenerate the G output feature map by applying the nth <\/sup><\/em>filters of the K kernels to the nth <\/sup><\/em>set of channels. Using pointwise convolution is a part of learning new features. In mathematical terms, this can be expressed as the following:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

In pointwise convolution, the kernel’s dimension is K x K x Ic<\/sub> x Oc<\/sub><\/em> .<\/p>\n\n\n\n

One advantage of this module (DWSC) over regular convolutions is\nthe reduced number of parameters. Both ReLU and global average pooling are\nconnected to the three 3 \u00d7 3 depth-wise convolutions and the two 5 \u00d7 5 DWS\nconvolutions.  Results showed that DWSC\nimproved training speed in stages 1-3 with little overhead on parameters, while\nin stages 4\u20138, it vastly increased. An optimal trade-off between training time\nand parameters was achieved. <\/p>\n\n\n\n

SACNN Module<\/strong><\/p>\n\n\n\n

Firstly, let’s\nestablish the meanings of the terms “convocation-layer” and\n“SA”. Subsequently, we delve into an elucidation of the mechanics\nunderlying the SA-CNN.<\/p>\n\n\n\n

Convolution neural network<\/strong><\/p>\n\n\n\n

Features are extracted using the two models from both color and grayscale images. To reduce the overall cross-entropy loss between multi-label predictions, we employ a method that is co-trained by pairs of images from two different modalities. IRGB<\/sub><\/em>‘s loss function is denoted by LFrgb<\/sub><\/em> , Igray<\/sub><\/em>‘s by LFgray<\/sub><\/em>, and LFfusion<\/sub><\/em>‘s by Ifusion<\/sub><\/em> . The cross-entropy loss function is denoted by CrEnt (). We train the RGB branch, the grayscale branch, and the fusion branch jointly using a loss function that combines the below three loss functions.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

We have developed\nthe proposed method to obtain the multi-receptive field of skin disease in\norder to gain insight into a wider variety of skin diseases. The\nmulti-receptive fields are made up of nested convolution layers with different\nkernel sizes, like 5 \u00d7 5 and 7 \u00d7 7.  In\norder to obtain more diseased areas multi-receptive fields are used to cover a\nbroader area of skin disease. The lesser convolutional kernel and the larger\nconvolutional kernel work together to train layers with varying weights that\ncorrespond to their respective receptive fields. They probe a wider diseased\narea, which ultimately enhances the model’s precision. In order to execute\nchannel-integrated and non-linear processing, we combine the feature maps of\nall convolutions and the ReLU + 5 \u00d7 5 convolution layer. Before this, we used\nthe 7 \u00d7 7 and 5 \u00d7 5 convolutional layers+Max pooling layer combination. The 5 \u00d7\n5 convolution is\nsliding a filter across the image to create feature maps. ReLU is then applied to the feature maps to keep only\nthe positive values, aiding the neural network in recognizing important\npatterns in the data. The filter calculates the feature map region’s average\nusing two average and max pooling layers. Thus, max pooling returns the most\nprominent feature in a feature map patch, while average pooling returns the\naverage of all features.<\/p>\n\n\n\n

In the TFFNet method, the diseases identified by their characteristics are given equal weight through the use of a maximum and average pooling process. When we pool features, we start with a feature vector F and end up with a vector V. In the instance where one makes use of maximum pooling, this vector Vmp<\/sup><\/em> is given by:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

where C is the feature map’s channel count. For feature map n = (1, C), let  be the set. The network produces a total of C similar feature maps as its output. All the features included in make up the f, and the mp denotes the max pooling operation. And when we consider the use of average max pooling, this vector Vavg <\/sup> is given by<\/em><\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Self-attention mechanism<\/strong><\/p>\n\n\n\n

Employing the attention mechanism across the sequence of hidden states obtained from convolution allows us to compute the vector for convolved features. Here at the convolution layer, our model made use of the intra-layer convolution connection that we just described. Therefore, because of the convolution link, Hu<\/sub><\/em> hidden units will influence numerous nearby units through the employment of right context Hu<\/sub>R<\/sup><\/em> and left contexts Hu<\/sub>L<\/sup><\/em>, as defined as:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

The vectors are obtained in self-attention by taking into consideration all hidden states, which are represented by convolve features. Convolutional features will yield a context vector Cv<\/sub><\/em>. Hu<\/sub><\/em> in the following way, using a weighted sum of all convolve features:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

The attention weight (A)w<\/sub><\/em> and weight matrix ( Wm<\/sub>) are used here. With convolution training, Cv<\/sub> , which stand for vectors, learn together. Focusing on convolve characteristics that significantly impact patient disease prediction is the goal of these attention vectors.<\/p>\n\n\n\n

SACNN<\/strong><\/p>\n\n\n\n

We combined the context vector that was computed before with the convolution in the following way for the SACNN (Fig. 3):<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Where  represents a weight matrix with dimensions defined by the product of a=1, 2, \u2026. i and b= 1, 2, \u2026. j, where i and j are specific parameters. The term I(t) corresponds to the input text representation, and B1<\/sup> <\/em>is the bias term. C (t-1)\u03b8<\/sup><\/em> denotes a previously computed convolution at time (t-1). Va <\/sub><\/em> signifies an attention vector, and the result of a convolution obtained from C1<\/sup> (t)<\/em> is considered the feature-map.<\/p>\n\n\n\n

Rectified Linear Unit (ReLU) activation functions capture non-linear correlations in feature maps. ReLU is mathematically defined as:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

The attention\nmethod is incorporated into the convolution that follows\na non-linear operation in the convolution that has been developed. When\ncompared to some established approaches that are considered to be\nstate-of-the-art, the experimental results reveal that the convolution that was\ndeveloped achieves a higher level of accuracy.<\/p>\n\n\n\n

Max pooling layer<\/strong><\/p>\n\n\n\n

The DWSC and SACNN modules are concatenated before the max pooling layer. In order to carry out the pooling procedure, the feature map C(t)2<\/sup>a,b<\/sub><\/em> derived from convolution in order to identify, P(Y, X, L)<\/em>  from DWSC and select large-granular features from images of diseases. Assuming that image-level granular features are obtained, the pooling process is anticipated to yield phrase-level granular features, as shown below:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Where C(t) 3<\/sup>b<\/sub><\/em> illustrates the feature-map that was produced as a result of the max-pooling.<\/p>\n\n\n\n

Fully connected layer<\/strong><\/p>\n\n\n\n

Furthermore, at this particular layer, a fully connected operation is carried out on the feature map C3<\/sup><\/em> , which is produced from the max-pooling layer, as illustrated in Figure 2:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Here, C3<\/sup><\/em> and C4<\/sup><\/em>  denote the feature maps derived from pooling and full connection operations, respectively. B4<\/sup> <\/em> and W4<\/sup><\/em>  represent the bias and weight parameters of the full connection layer.<\/p>\n\n\n\n

Each of the feature maps that are derived from full connection and pooling procedures is denoted by C3<\/sup><\/em>and C4<\/sup><\/em> in this context. The parameters of the fully connected layer are denoted by W4<\/sup><\/em>  and B4<\/sup><\/em>, which stands for weight and bias.<\/p>\n\n\n\n

The pseudo-code of the skin disease feature detection based two stream fusion algorithms is shown in Algorithm 1.<\/p>\n\n\n\n

Algorithm 1: Skin disease feature detection based two-stream fusion <\/strong> <\/p>\n\n\n\n\n\n
\n

Input: Two-stream model two input images Irgb<\/sub><\/em>  and Igray<\/sub><\/em>., the proposed model includes MobileNetV2 and NASNetMobile models, Maximum epochs Mepoch <\/sub><\/em>= 120 with batch size of Bs <\/sub><\/em>= 16 for the Network, and LR (learning rate) = 0.001.<\/p>\n

Output: The optimized feature detection based two-stream feature fusion network<\/p>\n

      1:    While M1<\/sub>  \u2264  Mn<\/sub> \u00d7 Bs<\/sub>  do<\/em><\/p>\n

      2:         Two-stream images batch as Irgb<\/sub>  and Igray<\/sub>. .<\/p>\n

      3:         Apply Pre-processing (image resize, shift range, rescale, rotation, shear & zoom,  flip) to both RGB and Grayscale images.<\/p>\n

      4:         Apply fusion on CNN models and add DWSC by Equations (1, (2), and (3).<\/p>\n

      5:         Proposed approach jointly using a loss function by Equation (4).<\/p>\n

      6:         Extract gradient images using gradient operator by Equation (5), and (6).<\/p>\n

      7:         Employing the Self-attention-based convolution by Equation (9), and (10).<\/p>\n

      8:        Jointly optimized feature detection based dual-scale fusion network by Equation (11), and (12).<\/p>\n

      9:    End<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n\n\n

Results and Discussion<\/strong><\/p>\n\n\n\n

This\nsection offers a comprehensive explanation of the dataset, along with the\ntechniques for experimentation, model training, and validation. The final\nsub-section of the paper provides an analysis of performance utilizing several\ncutting-edge models.<\/p>\n\n\n\n

Implementation\nDetails<\/strong><\/p>\n\n\n\n

In\nthis Google Colab Keras experiment, we utilize the K80, P100, and T4 GPUs. To\nmaximize the effectiveness of segmentation networks, we use a batch size of 16\nwith the Adam optimizer. We employed a learning rate of 0.001. The optimal\nvalues for the maximum number of epochs are 80 for transfer learning and 120\nfor the fused model. Images are randomly downsized to 256 \u00d7 224 for CNN\ntraining and then horizontally flipped.<\/p>\n\n\n\n

Dataset description<\/strong><\/p>\n\n\n\n

Images\nutilized in this study were sourced from the HAM10000 23<\/sup>, ISIC2017 24<\/sup>,\nand ISIC2016 25<\/sup> databases. All three datasets contain 13,894 images,\neach depicting various skin diseases across different areas of the body. Table\n2 displays the distribution of images across the different datasets. The\neffectiveness of the proposed method in identifying skin diseases is\ndemonstrated using each dataset. The models are divided into training and\ntesting sets in a 70:30 ratio.<\/p>\n\n\n\n

Data Preprocessing<\/strong><\/p>\n\n\n\n

In\nthis section, we provide a full description of the preprocessing conducted on\nthe datasets. The preprocessing involves resizing the images and augmenting the\ndata.<\/p>\n\n\n\n

Resize<\/strong><\/p>\n\n\n\n

Images of high resolution are found in the HAM10000, ISIC2016, and ISIC2017 datasets. When used directly for training, images of skin lesions have a resolution of 600 \u00d7 450 pixels, which results in a significant increase in the amount of calculation required. Therefore, to comply with the specifications of the model, we reduced the dimensions of each image from 600 \u00d7 450 pixels to 224 \u00d7 224 pixels. We allocate 70% for training and 30% for testing. <\/p>\n\n\n\n

Data augmentation<\/strong><\/p>\n\n\n\n

Despite the fact that the three datasets contain 13894 images between them, this amount of information is insufficient to satisfy the requirements of deep learning algorithms. As a result, we apply six different data augmentation procedures to the training samples. These include randomly rotating the samples, shifting them horizontally and vertically, randomly zooming, randomly twisting, flipping, and resizing them.<\/p>\n\n\n\n

Evaluation\nMetrics<\/strong><\/p>\n\n\n\n

Accuracy, precision, recall, and F1 score are some of the evaluation measures we utilize based on recommendations from our dataset. Definitions for these quantitative measures can be found in (13)-(16). Following is the formula for their computation:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Findings and comparison<\/strong><\/p>\n\n\n\n

This\nsegment provides a thorough perspective on experimentation, comparison, and\ndiscussion. In the concluding subsection, an evaluation of performance is\nshowcased through the analysis of various cutting-edge models.<\/p>\n\n\n\n

The findings from single models<\/strong><\/p>\n\n\n\n

Table\n3 displays the classification results for multiclass skin diseases using the\nHAM10000, ISIC2016, and ISIC2017 datasets with two single-model frameworks:\nMobileNetV2 and NASNetMobile. The NASNetMobile model combined with the HAM10000\ndataset achieves the best results, with an accuracy of 81%. It also attains a\n70 F1-score, 81 recall rate, and 68 precision rates. On the ISIC2017 dataset,\nthe NASNetMobile model performs second-best, achieving an accuracy rate of 80% with\nan F1-score of 69, recall of 80, and precision of 74. The highest-performing metrics\nare bolded for emphasis. Additionally, the NASNetMobile model has approximately\n4.1 million trainable parameters, while MobileNetV2 has around 3.2 million.<\/p>\n\n\n\n

Table 3: The classification results of two single models on different datasets.<\/strong><\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
\n

Single network – ISIC -2016 Dataset<\/p>\n<\/td>\n<\/tr>\n

\n

Model<\/p>\n<\/td>\n

\n

Precision<\/p>\n<\/td>\n

\n

Recall<\/p>\n<\/td>\n

\n

F1-Score<\/p>\n<\/td>\n

\n

Testing Accuracy<\/p>\n<\/td>\n<\/tr>\n

\n

MobileNetV2<\/p>\n<\/td>\n

\n

77<\/p>\n<\/td>\n

\n

79<\/p>\n<\/td>\n

\n

78<\/p>\n<\/td>\n

\n

79<\/p>\n<\/td>\n<\/tr>\n

\n

NASNetMobile<\/p>\n<\/td>\n

\n

73<\/p>\n<\/td>\n

\n

78<\/p>\n<\/td>\n

\n

74<\/p>\n<\/td>\n

\n

78<\/p>\n<\/td>\n<\/tr>\n

\n

Single network – ISIC -2017 Dataset<\/p>\n<\/td>\n<\/tr>\n

\n

MobileNetV2<\/p>\n<\/td>\n

\n

74<\/p>\n<\/td>\n

\n

79<\/p>\n<\/td>\n

\n

65<\/p>\n<\/td>\n

\n

79<\/p>\n<\/td>\n<\/tr>\n

\n

NASNetMobile<\/p>\n<\/td>\n

\n

74<\/p>\n<\/td>\n

\n

80<\/p>\n<\/td>\n

\n

69<\/p>\n<\/td>\n

\n

80<\/p>\n<\/td>\n<\/tr>\n

\n

Single network – HAM10000 Dataset<\/p>\n<\/td>\n<\/tr>\n

\n

MobileNetV2<\/p>\n<\/td>\n

\n

72<\/p>\n<\/td>\n

\n

79<\/p>\n<\/td>\n

\n

62<\/p>\n<\/td>\n

\n

79<\/p>\n<\/td>\n<\/tr>\n

\n

NASNetMobile<\/p>\n<\/td>\n

\n

68<\/p>\n<\/td>\n

\n

81<\/p>\n<\/td>\n

\n

70<\/p>\n<\/td>\n

\n

81<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n

The\nfindings from fused model<\/strong><\/p>\n\n\n\n

Table\n4 displays the classification outcomes of a fused model (MobileNetV2 +\nNASNetMobile) for multiscale skin diseases using the HAM10000, ISIC2016, and\nISIC2017 datasets. With the fused model architecture, the best results are\nachieved by the MobileNetV2 + NASNetMobile model on the ISIC2017 dataset,\nattaining an accuracy rate of 84%. The NASNetMobile model achieves an F1-Score,\nrecall, and precision of 81, 76, and 81, respectively. On the HAM10000 dataset,\nthe MobileNetV2 + NASNetMobile fused model performs second-best, with an\naccuracy rate of 83%. For this dataset, MobileNetV2 + NASNetMobile achieve an\nF1-Score, recall, and precision of 81, 80, and 76, respectively. This fused\nmodel comprises approximately 8.6 million trainable parameters.<\/p>\n\n\n\n

Table 4: The classification results of fused models on different datasets.<\/strong><\/p>\n\n\n\n\n\n\n\n\n\n\n\n
\n

Late fusion – ISIC -2016 Dataset<\/strong><\/p>\n<\/td>\n<\/tr>\n

\n

Model<\/strong><\/p>\n<\/td>\n

\n

Precision<\/strong><\/p>\n<\/td>\n

\n

Recall<\/strong><\/p>\n<\/td>\n

\n

F1-Score<\/strong><\/p>\n<\/td>\n

\n

Testing Accuracy<\/strong><\/p>\n<\/td>\n<\/tr>\n

\n

MobileNetV2 + NASNetMobile<\/p>\n<\/td>\n

\n

77<\/p>\n<\/td>\n

\n

79<\/p>\n<\/td>\n

\n

80<\/p>\n<\/td>\n

\n

82<\/p>\n<\/td>\n<\/tr>\n

\n

Late fusion – ISIC -2017 Dataset<\/p>\n<\/td>\n<\/tr>\n

\n

MobileNetV2 + NASNetMobile<\/p>\n<\/td>\n

\n

81<\/p>\n<\/td>\n

\n

76<\/p>\n<\/td>\n

\n

81<\/p>\n<\/td>\n

\n

84<\/p>\n<\/td>\n<\/tr>\n

\n

Late fusion – HAM10000 Dataset<\/p>\n<\/td>\n<\/tr>\n

\n

MobileNetV2 + NASNetMobile<\/p>\n<\/td>\n

\n

76<\/p>\n<\/td>\n

\n

80<\/p>\n<\/td>\n

\n

81<\/p>\n<\/td>\n

\n

83<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n

The\nfindings from proposed model<\/strong><\/p>\n\n\n\n

Table\n5 presents the classification results obtained by applying the TFFNet model to\nthe HAM10000, ISIC2016, and ISIC2017 datasets for the analysis of multiclass\nskin diseases. Testing the TFFNet model architecture on the HAM10000 dataset\nyielded a remarkable accuracy rate of 90%. The proposed model achieved an\nF1-score of 81, a recall of 80, and a precision of 83, respectively. Similarly,\nthe TFFNet model demonstrated strong performance on the ISIC2017 dataset,\nachieving a 90% accuracy rate. It received scores of 81, 82, and 78 for F1\nscore, recall, and precision, respectively. This model comprises approximately\n7.5 million trainable parameters.<\/p>\n\n\n\n

Table 5: The classification results of proposed models on different datasets.<\/strong><\/p>\n\n\n\n\n\n\n\n\n\n\n\n
\n

Proposed method – ISIC -2016 Dataset<\/strong><\/p>\n<\/td>\n<\/tr>\n

\n

Model<\/strong><\/p>\n<\/td>\n

\n

Precision<\/strong><\/p>\n<\/td>\n

\n

Recall<\/strong><\/p>\n<\/td>\n

\n

F1-Score<\/strong><\/p>\n<\/td>\n

\n

Testing Accuracy<\/strong><\/p>\n<\/td>\n<\/tr>\n

\n

TFFNet<\/p>\n<\/td>\n

\n

82<\/p>\n<\/td>\n

\n

78<\/p>\n<\/td>\n

\n

80<\/p>\n<\/td>\n

\n

89<\/p>\n<\/td>\n<\/tr>\n

\n

Proposed method – ISIC -2017 Dataset<\/p>\n<\/td>\n<\/tr>\n

\n

TFFNet<\/p>\n<\/td>\n

\n

78<\/p>\n<\/td>\n

\n

82<\/p>\n<\/td>\n

\n

81<\/p>\n<\/td>\n

\n

90<\/p>\n<\/td>\n<\/tr>\n

\n

Proposed method – HAM10000 Dataset<\/p>\n<\/td>\n<\/tr>\n

\n

TFFNet<\/p>\n<\/td>\n

\n

83<\/p>\n<\/td>\n

\n

80<\/p>\n<\/td>\n

\n

81<\/p>\n<\/td>\n

\n

90<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n

Analysis\nof TFFNet architecture<\/strong><\/p>\n\n\n\n

The training accuracy of the TFFNet model proposed in\nthis study demonstrated a discernible increase, characterized by a swift\ninitial ascent followed by a gradual levelling off. In contrast, the validation\naccuracy exhibited a consistent upward trajectory, as depicted in Fig. 4c,\nalbeit with fluctuations throughout the training period. Unlike the CNN models,\nthe validation curves of TFFNet displayed comparatively less fluctuation.\nParticularly noteworthy is the fact that signs of saturation started to emerge\nduring the 80th epoch. A significant decrease in the discrepancy between\nvalidation and training accuracy was obtained compared to the CNN model. TFFNet\nconsistently displayed stable performance in terms of validation accuracy,\nsuggesting that the model’s fitting capabilities could potentially result in\nsuperior generalization on unseen test data when compared to CNN models.<\/p>\n\n\n\n

Comparison\non our single, fused, and proposed models with three datasets<\/strong><\/p>\n\n\n\n

Compared\nto the single and fused approaches, the proposed framework outperforms them\nconsistently across all metrics, including F1-score, precision, accuracy, and\nrecall, as depicted in Figure 4. In this experiment, we utilized three\ndistinct datasets: HAM10000, ISIC-2017, and ISIC-2016. Testing on the HAM10000 and\nISIC2017 datasets yielded an impressive accuracy rate of 90% for the proposed\nTFFNet model architecture. The results of the study are presented in Table 6,\nwhich indicates that TFFNet achieved the highest accuracy among all models.\nAdditionally, TFFNet had fewer parameters compared to MobileNetV2,\nNASNetMobile, and the combination of the two models.<\/p>\n\n\n\n\n\n
\"\"<\/td>\n\n

Figure 4: Precision, recall, F1score, and accuracy comparison of (a) single models; (b) fused models, and (c) proposed model<\/strong><\/p>\n

<\/p>\n

Click here to view Figure<\/a><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n\n\n

Table 6: Analysis of TFFNet with different models on trainable parameters.<\/strong><\/p>\n\n\n\n\n\n\n\n\n\n
\n

Models<\/strong><\/p>\n<\/td>\n

\n

Training parameters<\/strong><\/p>\n<\/td>\n<\/tr>\n

\n

MobileNetV2<\/p>\n<\/td>\n

\n

3,257,895<\/p>\n<\/td>\n<\/tr>\n

\n

NASNetMobile<\/p>\n<\/td>\n

\n

4,125,468<\/p>\n<\/td>\n<\/tr>\n

\n

MobileNetV2 + NASNetMobile<\/p>\n<\/td>\n

\n

8,684,985<\/p>\n<\/td>\n<\/tr>\n

\n

TFFNet<\/p>\n<\/td>\n

\n

7,572,286<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n

Comparison\nof proposed model with state of the art on HAM10000 dataset<\/strong><\/p>\n\n\n\n

Among\nstate-of-the-art methods, our proposed TFFNet method surpasses them all, with\nan average improvement of 1.4% over the second-best method. Table 7 presents\nthe results comparing the accuracy of the HAM10000 dataset to that of the\nprevious equivalent reference.<\/p>\n\n\n\n

Table 7: Accuracy comparison with a previous related reference for the HAM10000 dataset.<\/strong><\/p>\n\n\n\n\n\n\n\n\n\n
\n

Authors<\/strong><\/p>\n<\/td>\n

\n

Year<\/strong><\/p>\n<\/td>\n

\n

Accuracy (%)<\/strong><\/p>\n<\/td>\n<\/tr>\n

\n

Gouda 26<\/sup><\/p>\n<\/td>\n

\n

2022<\/p>\n<\/td>\n

\n

83.2<\/p>\n<\/td>\n<\/tr>\n

\n

Hoang 27<\/sup><\/p>\n<\/td>\n

\n

2022<\/p>\n<\/td>\n

\n

86.33<\/p>\n<\/td>\n<\/tr>\n

\n

Kim 28<\/sup><\/p>\n<\/td>\n

\n

2023<\/p>\n<\/td>\n

\n

88.6<\/p>\n<\/td>\n<\/tr>\n

\n

Ours<\/p>\n<\/td>\n

\n

2023<\/p>\n<\/td>\n

\n

90.12<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n

Comparison\nof proposed model with state of the art on ISIC 2017 dataset<\/strong><\/p>\n\n\n\n

We conducted a\ncomparative analysis to assess the efficiency of our proposed TFFNet approach\nagainst the most recent and innovative classification strategies. The results\nof these comparisons were analyzed and evaluated. Table 8 presents the\ncomparison between the ISIC2017 dataset and a relevant previous reference,\naimed at assessing the accuracy of the ISIC2017 dataset. Our proposed method\ndemonstrates superior performance compared to state-of-the-art methods, with an\naverage accuracy that is 2.5% higher than the method currently considered to be\nin second place.<\/p>\n\n\n\n

Table 8: Accuracy comparison with existing methods for the ISIC 2017dataset.<\/strong><\/p>\n\n\n\n\n\n\n\n\n\n
\n

Authors<\/strong><\/p>\n<\/td>\n

\n

Year<\/strong><\/p>\n<\/td>\n

\n

Accuracy (%)<\/strong><\/p>\n<\/td>\n<\/tr>\n

\n

Al-masni 29<\/sup><\/p>\n<\/td>\n

\n

2020<\/p>\n<\/td>\n

\n

81.57<\/p>\n<\/td>\n<\/tr>\n

\n

Yilmaz 30<\/sup><\/p>\n<\/td>\n

\n

2021<\/p>\n<\/td>\n

\n

82<\/p>\n<\/td>\n<\/tr>\n

\n

Kim 28<\/sup><\/p>\n<\/td>\n

\n

2023<\/p>\n<\/td>\n

\n

87.5<\/p>\n<\/td>\n<\/tr>\n

\n

Ours<\/p>\n<\/td>\n

\n

2023<\/p>\n<\/td>\n

\n

90.52<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n

Comparison\nof proposed model with state-of-the-art on ISIC2016 dataset<\/strong><\/p>\n\n\n\n

According to\nTable 9, our proposed TFFNet method achieves the highest average accuracy of\n89.70% among all compared approaches. This accuracy is 0.81% and 1.4% higher\nthan the previous two methods, namely Dahou 33<\/sup> and Wei 32<\/sup>,\nrespectively.<\/p>\n\n\n\n

Table 9: Accuracy compared with previous methods for the ISIC 2016dataset.<\/strong><\/p>\n\n\n\n\n\n\n\n\n\n\n
\n

ss<\/strong><\/p>\n<\/td>\n

\n

Year<\/strong><\/p>\n<\/td>\n

\n

Accuracy (%)<\/strong><\/p>\n<\/td>\n<\/tr>\n

\n

Al-masni 29<\/sup><\/p>\n<\/td>\n

\n

2020<\/p>\n<\/td>\n

\n

80<\/p>\n<\/td>\n<\/tr>\n

\n

Yu 31<\/sup><\/p>\n<\/td>\n

\n

2020<\/p>\n<\/td>\n

\n

86.8<\/p>\n<\/td>\n<\/tr>\n

\n

Wei 32<\/sup><\/p>\n<\/td>\n

\n

2020<\/p>\n<\/td>\n

\n

87.6<\/p>\n<\/td>\n<\/tr>\n

\n

Dahou 33<\/sup><\/p>\n<\/td>\n

\n

2023<\/p>\n<\/td>\n

\n

88.19<\/p>\n<\/td>\n<\/tr>\n

\n

Ours<\/p>\n<\/td>\n

\n

2023<\/p>\n<\/td>\n

\n

89.70<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n

Conclusion<\/strong><\/strong><\/p>\n\n\n\n

Skin diseases affect a significant number of individuals and rank among the most widespread categories of ailments globally. Among these, acne stands out at the top of the list alongside various other skin diseases, each posing its own set of risks, ranging from minor discomforts to life-threatening conditions. The integration of computer-aided diagnosis 34<\/sup> has greatly facilitated the medical community in the identification and categorization of skin diseases, addressing a substantial challenge. The field of skin disease classification 35<\/sup> has seen the emergence of several deep learning models. However, there remains room for improvement in areas such as computational efficiency and dataset-specific accuracy. To truly enhance the effectiveness of computer-aided diagnosis, it must be capable of accurately discerning specific skin diseases from an extensive list. However, as the number of skin classes increases, the complexity and parameter count of a model naturally escalate. This is where the TFFNet model comes into play, offering a solution with fewer parameters while maintaining satisfactory accuracy. Contrary to the assumption that augmenting parameter numbers enhances accuracy, the results of our study challenge this notion. The incorporation of two modules with the SA block demonstrates a reduction in parameters without compromising accuracy. The proposed TFFNet model exhibited total accuracies of 90.12%, 90.52%, and 89.70% on test datasets. Further evaluation using microscopic and histological images can shed light on potential challenges in disease categorization for this innovative network.<\/p>\n\n\n\n

Acknowledgment <\/strong><\/p>\n\n\n\n

We thank Graphic Era (Deemed to be University) Dehradun for the use of their research facilities.<\/p>\n\n\n\n

Conflict of Interest<\/strong> <\/p>\n\n\n\n

On behalf of all authors, the corresponding author states that there is no conflict of interest. <\/p>\n\n\n\n

Funding Sources<\/strong><\/p>\n\n\n\n

This research received no external funding. <\/p>\n\n\n\n

Data availability<\/strong> <\/p>\n\n\n\n

Not applicable.<\/p>\n\n\n\n

Reference<\/strong><\/p>\n\n\n\n

  1. Zhou, S.K.; Greenspan, H.; Davatzikos, C.; Duncan, J.S.; Van Ginneken, B.; Madabhushi, A.; Prince, J.L.; Rueckert, D.; Summers, R.M. A Review of Deep Learning in Medical Imaging: Imaging Traits, Technology Trends, Case Studies with Progress Highlights, and Future Promises. Proc. IEEE 2021, 109, 8s20\u2013838.
    CrossRef<\/a><\/li>
  2.  Bai, W.; Suzuki, H.; Huang, J.; Francis, C.; Wang, S.; Tarroni, G.; Guitton, F.; Aung, N.; Fung, K.; Petersen, S.E.; et al. A population-based phenome-wide association study of cardiac and aortic structure and function. Nat. Med. 2020, 26, 1654.
    CrossRef <\/a><\/li>
  3.  Shanthi, T.; Sabeenian, R.S.; Anand, R. Automatic Diagnosis of Skin Diseases Using Convolution Neural Network. Microprocess. Microsyst. 2020, 76, 103074.
     CrossRef <\/a><\/li>
  4. American Cancer Society. Key Statistics for Melanoma Skin Cancer. 2022. Available online: https:\/\/www.cancer.org\/cancer\/ melanoma-skin-cancer\/about\/key-statistics.html.<\/li>
  5. Wei, L.-S.; Gan, Q.; Ji, T. Skin Disease Recognition Method Based on Image Color and Texture Features. Comput. Math. Methods Med. 2018, 2018, 8145713.
     CrossRef <\/a><\/li>
  6. Amarathunga, A.A.L.C.; Ellawala, E.P.W.C.; Abeysekara, G.N.; Amalraj, C.R.J. Expert System for Diagnosis of Skin Diseases. Int. J. Sci. Technol. Res. 2015, 4, 174\u2013178. <\/li>
  7.  Bajwa, M.N.; Muta, K.; Malik, M.I.; Siddiqui, S.A.; Braun, S.A.; Homey, B.; Dengel, A.; Ahmed, S. Computer-Aided Diagnosis of Skin Diseases Using Deep Neural Networks. Appl. Sci. 2020, 10, 2488.
     CrossRef <\/a><\/li>
  8. Monisha, M.; Suresh, A.; Rashmi, M.R. Artificial Intelligence Based Skin Classification Using GMM. J. Med. Syst. 2019, 43, 3.
     CrossRef <\/a><\/li>
  9. Kassem, M.A.; Hosny, K.M.; Dama\u0161evi\u02c7cius, R.; Eltoukhy, M.M. Machine Learning and Deep Learning Methods for Skin Lesion Classification and Diagnosis: A Systematic Review. Diagnostics 2021, 11, 1390.
     CrossRef <\/a><\/li>
  10. Maqsood, S., & Dama\u0161evi\u010dius, R. (2023). Multiclass skin lesion localization and      classification using deep learning-based features fusion and selection framework for smart healthcare. Neural Networks<\/em>, 160<\/em>, 238-258.
    CrossRef<\/a> <\/li>
  11. Cai, H., Brinti Hussin, N., Lan, H., & Li, H. (2023). A Skin Cancer Detector Based on Transfer Learning and Feature Fusion. Current Bioinformatics<\/em>, 18<\/em>(6), 517-526.
     CrossRef <\/a><\/li>
  12. Zhao, X., & Ren, Z. (2023, January). Multi-scale Gaussian Difference Preprocessing and Dual Stream CNN-Transformer Hybrid Network for Skin Lesion Segmentation. In International Conference on Multimedia Modeling (pp. 671-682). Cham: Springer Nature Switzerland.
    CrossRef <\/a><\/li>
  13. Tian, Q., Sun, W., Zhang, L., Pan, H., Chen, Q., & Wu, J. (2023). Gesture image recognition method based on DC-Res2Net and a feature fusion attention module. Journal of Visual Communication and Image Representation, 95, 103891.
     CrossRef <\/a><\/li>
  14. Rajput, G.; Agrawal, S.; Raut, G.; Vishvakarma, S.K. An accurate and noninvasive skin cancer screening based on imaging technique. Int. J. Imaging Syst. Technol. 2022, 32, 354\u2013368.
     CrossRef <\/a><\/li>
  15. Raza, R.; Zulfiqar, F.; Tariq, S.; Anwar, G.B.; Sargano, A.B.; Habib, Z. Melanoma Classification from Dermoscopy Images Using Ensemble of Convolutional Neural Networks. Mathematics 2022, 10, 26.
    CrossRef <\/a><\/li>
  16. Gouda, W.; Sama, N.U.; Al-Waakid, G.; Humayun, M.; Jhanjhi, N.Z. Detection of Skin Cancer Based on Skin Lesion Images Using Deep Learning. Healthcare 2022, 10, 1183.
     CrossRef <\/a><\/li>
  17. Rehman, M.Z.U.; Ahmed, F.; Alsuhibany, S.A.; Jamal, S.S.; Ali, M.Z.; Ahmad, J. Classification of Skin Cancer Lesions Using Explainable Deep Learning. Sensors 2022, 22, 6915.
    CrossRef<\/a> <\/li>
  18. Aldhyani, T.H.H.; Verma, A.; Al-Adhaileh, M.H.; Koundal, D. Multi-Class Skin Lesion Classification Using a Lightweight Dynamic Kernel Deep-Learning-Based Convolutional Neural Network. Diagnostics 2022, 12, 2048.
    CrossRef <\/a><\/li>
  19. Kousis, I.; Perikos, I.; Hatzilygeroudis, I.; Virvou, M. Deep Learning Methods for Accurate Skin Cancer Recognition and Mobile Application. Electronics 2022, 11, 1294.
    CrossRef <\/a><\/li>
  20. Hasan, K.; Elahi, T.E.; Alam, A.; Jawad, T.; Mart\u00ed, R. DermoExpert: Skin lesion classification using a hybrid convolutional neural network through segmentation, transfer learning, and augmentation. Inform. Med. Unlocked 2022, 28, 100819.
     CrossRef <\/a><\/li>
  21. Tan, L., Wu, H., Xia, J., Liang, Y., & Zhu, J. (2024). Skin lesion recognition via global-local attention and dual-branch input network. Engineering Applications of Artificial Intelligence, 127, 107385.
     CrossRef <\/a><\/li>
  22. Muhammad, W., Aramvith, S., & Onoye, T. (2021). Multi-scale Xception based depthwise separable convolution for single image super-resolution. Plos one, 16(8), e0249278.
     CrossRef <\/a><\/li>
  23. P. Tschandl, C. Rosendahl, and H. Kittler, \u2018\u2018The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions,\u2019\u2019 Scientific Data, vol. 5, no. 1, pp. 1\u20139, Aug. 2018.
     CrossRef <\/a><\/li>
  24. Hasan MK, Elahi MTE, Alam MA, Jawad MT, Mart\u00edR. DermoExpert: skin lesion classification using a hybrid convolutional neural network through segmentation, transfer learning, and augmentation. Inform Med Unlock. (2022) 28:100819. doi: 10.1016\/j.imu.2021.100819.
     CrossRef <\/a><\/li>
  25. D. Gutman, N.C. Codella, E. Celebi, B. Helba, M. Marchetti, N. Mishra, A. Halpern, Skin lesion analysis toward melanoma detection: a challenge at the international symposium on biomedical imaging (ISBI) 2016, hosted by the international skin imaging collaboration (ISIC), arXiv preprint arXiv:1605.01397, https:\/\/doi.org\/10. 48550\/arXiv.1605.01397, 2016.<\/li>
  26. Gouda, W.; Sama, N.U.; Al-Waakid, G.; Humayun, M.; Jhanjhi, N.Z. Detection of skin cancer based on skin lesion images using deep learning. Healthcare 2022, 10, 1183.
     CrossRef <\/a><\/li>
  27. Hoang, L., Lee, S. H., Lee, E. J., & Kwon, K. R. (2022). Multiclass skin lesion classification using a novel lightweight deep learning framework for smart healthcare. Applied Sciences<\/em>, 12<\/em>(5), 2677.
     CrossRef <\/a><\/li>
  28. Kim, C., Jang, M., Han, Y., Hong, Y., & Lee, W. (2023). Skin Lesion Classification Using Hybrid Convolutional Neural Network with Edge, Color, and Texture Information. Applied Sciences<\/em>, 13<\/em>(9), 5497.
     CrossRef <\/a><\/li>
  29. Al-masni, A.M.; Kim, D.; Kim, T. Multiple skin lesions diagnostics via integrated deep convolutional networks for segmentation and classification. Comput. Methods Programs Biomed. 2020, 190, 105351.
     CrossRef <\/a><\/li>
  30. Yilmaz, A., Kalebasi, M., Samoylenko, Y., Guvenilir, M. E., & Uvet, H. (2021). Benchmarking of Lightweight Deep Learning Architectures for Skin Cancer Classification using ISIC 2017 Dataset. arXiv preprint <\/em>arXiv:2110.12270<\/em>.<\/li>
  31. Yu, Z.; Jiang, F.; Zhou, F.; He, X.; Ni, D.; Chen, S.; Wang, T.; Lei, B. Convolutional descriptors aggregation via cross-net for skin lesion recognition. Appl. Soft Comput. 2020, 92, 106281.
     CrossRef <\/a><\/li>
  32. Wei, L.; Ding, K.; Hu, H. Automatic skin cancer detection in dermoscopy images based on ensemble lightweight deep learning network. IEEE Access 2020, 8, 99633\u201399647.
     CrossRef <\/a><\/li>
  33. Dahou, A., Aseeri, A. O., Mabrouk, A., Ibrahim, R. A., Al-Betar, M. A., & Elaziz, M. A. (2023). Optimal Skin Cancer Detection Model Using Transfer Learning and Dynamic-Opposite Hunger Games Search. Diagnostics<\/em>, 13<\/em>(9), 1579.
     CrossRef <\/a><\/li>
  34. Ding, H., Huang, Q., & Alkhayyat, A. (2024). A computer aided system for skin cancer detection based on Developed version of the Archimedes Optimization algorithm. Biomedical Signal Processing and Control, 90, 105870.
    CrossRef <\/a><\/li>
  35. Akilandasowmya, G., Nirmaladevi, G., Suganthi, S. U., & Aishwariya, A. (2024). Skin cancer diagnosis: Leveraging deep hidden features and ensemble classifiers for early detection and classification. Biomedical Signal Processing and Control, 88, 105306.
    CrossRef <\/a><\/li><\/ol>\n","protected":false},"excerpt":{"rendered":"

    Introduction Melanoma, a highly aggressive skin cancer, accounts for only […]<\/p>\n","protected":false},"author":15,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[117],"tags":[],"class_list":["post-60187","post","type-post","status-publish","format-standard","hentry","category-vol17no3"],"_links":{"self":[{"href":"https:\/\/biomedpharmajournal.org\/staging\/wp-json\/wp\/v2\/posts\/60187","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/biomedpharmajournal.org\/staging\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/biomedpharmajournal.org\/staging\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/biomedpharmajournal.org\/staging\/wp-json\/wp\/v2\/users\/15"}],"replies":[{"embeddable":true,"href":"https:\/\/biomedpharmajournal.org\/staging\/wp-json\/wp\/v2\/comments?post=60187"}],"version-history":[{"count":5,"href":"https:\/\/biomedpharmajournal.org\/staging\/wp-json\/wp\/v2\/posts\/60187\/revisions"}],"predecessor-version":[{"id":61685,"href":"https:\/\/biomedpharmajournal.org\/staging\/wp-json\/wp\/v2\/posts\/60187\/revisions\/61685"}],"wp:attachment":[{"href":"https:\/\/biomedpharmajournal.org\/staging\/wp-json\/wp\/v2\/media?parent=60187"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/biomedpharmajournal.org\/staging\/wp-json\/wp\/v2\/categories?post=60187"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/biomedpharmajournal.org\/staging\/wp-json\/wp\/v2\/tags?post=60187"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}