{"id":25019,"date":"2018-12-25T11:42:35","date_gmt":"2018-12-25T11:42:35","guid":{"rendered":"http:\/\/biomedpharmajournal.org\/?p=25019"},"modified":"2020-04-24T03:43:38","modified_gmt":"2020-04-24T03:43:38","slug":"image-analysis-on-fingertip-video-to-obtain-ppg","status":"publish","type":"post","link":"https:\/\/biomedpharmajournal.org\/staging\/vol11no4\/image-analysis-on-fingertip-video-to-obtain-ppg\/","title":{"rendered":"Image Analysis on Fingertip Video to Obtain PPG"},"content":{"rendered":"

Introduction<\/strong><\/p>\n

PPG signal can be described as non-invasive, inexpensive, portable and convenient tool for calculatingOxygen saturation (SPO2), blood pressure (systole\/diastole), Heart Rate, Arterial age, vascular disease etc. Duringthe capture, red contact video of fingertips, miss touch errors can produce significant variation in real time resultsdue to noise. Similarly, high pressure on camera and low pressure on camera can produce incorrect pixel intensity and therefore, produces incorrect result. They can be treated as noise and needs to be removed up to a level to keep up the originality of a signal to give correct heart rate.<\/p>\n

PPG signal can be measured with conventional pulse oximetry devices or smart phone camera. Here in our experiment we have used smart phone camera to capture fingertip red image video for 30 sec. on average in visible wavelength (400-700 nm) under normal physical & environmental condition for obtaining brightness signal (based on intensity variation) to plot & analyze the PPG signal.Infrared is a special form of invisible electromagnetic wave that can be felt by heat. For better picture, IR radiation is blocked by film or filter (camera glass). If no IR filter, pictures will not be as good as digital camera. So, it is a challenge to determine SPO2 using mobile camera video as for this both the Red & IR light is needed.<\/p>\n

Here, in our study we have performedanalysis on captured video to obtain good quality PPG signal. Using this analysis we can tell if the captured image is a suitable choice for plotting the brightness signal or not! We have also compared the result with FDA approved oximetry devices to draw the conclusion.There are few challenges with PPG like a. different smartphones lead to different color saturation in the image b. motion artifacts due to shifting of fingers c. different fingertip pressure on the lens d. different environmental condition (dark \/ light) e. ROI (region of interest selection) in a frame f. Total frame number to be processed out of total count f. LED\/Flash is used or not!<\/p>\n

Video Capturing & PPG1,2<\/sup><\/strong><\/p>\n

The captured reflected light at camera sensor has two components a. AC or pulsatile component of artery b. DC or non-pulsatile component due to tissue background, venous blood and constant part of arterial blood flow.This AC & DC componentsof both Red & IR are used to measure oxygen saturation in the blood. Oxygen saturation is nothing but a ratio in between oxyhemoglobin to the amount of hemoglobin present in the blood. Oxygen combined with hemoglobin to form the compound to carry oxygen in blood.It\u2019s normal range is in between 95-100.<\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 1: PPG Light Component2<\/sup><\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

The amplified waveform of brightness signalis used to measure heart rate, systolic point, diastolic point etc.<\/p>\n

PPGCapturing with Mobile Camera1,7,8<\/sup><\/strong><\/p>\n

PPG is a form of cardiac pulse which is measured optically. It reflects the blood movement in the vessels that travels from heart to fingertips and toes through the vessels in the wave like motion. Here, in our experiment it is measured with average intensity variation method. There is another method namely curve-line movement based method. In intensity variation, we have measured average intensity variation of every frame in red channel. For systolic peak the corresponding image intensity is high & for diastolic peak it is low. In this experiment, we have shown why not to choose green or blue channel for measuring average pixel intensity!<\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 2.1: Captured Red Image with Intensity variation [Ayan Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 2.2: PPG Area [Ayan Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 2.3: Pulsatile Part [Ayan Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

Every smartphone camera has an optical sensor. Introduction of smartphone & tabs to healthcare segment has a potential to improve healthcare & thereby reduce healthcare costs. Latest smartphones have advanced powerful hardware, software, better memory & storage, well connectivity to cloud storage and backup option, advanced OS, embedded sensors got diverse, they are fast and do have long running batteries. The quality of good PPG signals depends on the properties of the skin area, camera characteristics, light emitting diode (LED) usage, and light conditions. A good PPG signal can help to predict accurate Heart Rate, BP, SPO2 & Respiratory rate.<\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 3: PPG Capturing2<\/sup><\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

In our experiment, left & right hand fingertips are held on smartphone camera lens with flash for 30 sec & without flash for 30 sec without creation of additional pressure and captured video is stored in the server for further analysis in MATLAB. PPG data can be considered of two types – a. Reflective Mode (Video with flash) b. Transmission mode (Video without flash). Data has been collected with two different smartphones Moto-G Turbo & Samsung J7 Next Gen. in this project.FPS of mobile camera is also an important parameter for capturing video and experiment has shown its range in between 25-30 FPS and we have used same data as sampling frequency for corresponding video.<\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 4: FPS \u2013 Moto-G Turbo [Ayan Chatterjee]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 5: FPS \u2013 Samsung J7 Next Gen [Ayan Chatterjee]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

The samples as described in X axis of above pictures, are PPG cases captured under different circumstances (with camera flash on) like normal Light, normal Light with external Sound, dark condition, dark & mobile heat, normal Walking. In all cases with respect to both the cameras, the fps lies in between 25-30 range on average.The captured image is in RGB format so every pixel consists of R (0-255), G (0-255), B(0-255) values. For Proper touch measurement, we have used a. average intensity count of frame in red channel b. histogram analysis for red, green and blue channel c. Rules and Values for different parameters d. Gray scale image conversion and analysis (binary method). Under histogram analysis, if LED is turned on, then G and B channel intensity values are lower half of the histogram means less than 128. If LED is turned off, then G and B channel intensity values are close to zero in histogram. Hence, method of use, whether the LED is used or not can be detected with above rules.<\/p>\n

If more contact pressure is produced on camera:<\/p>\n

Deforming arterial wall leads to wrong reading<\/p>\n

Block micro circulation in the capillary<\/p>\n

Sudden up & down of pixel intensity.<\/p>\n

After FFT analysis of brightness signal flat line (Y=constant) is produced<\/p>\n

The process of heart rate mesurement can be shortened as<\/strong><\/p>\n

Video Signal \u00e0Calibration Check \u00e0Average Pixel Intensity of red channel for first 660 frame \u00e0 Analysis of image in the video \u00e0 Brightness Signal Obtaining \u00e0 Set Filter stabilization time = 2& initial signal cutoff = 0 \u00e0 Band Pass Butterworth Filter \u00e0 Select WINDOW_START = 6, \u00a0WINDOW_END = 12 out of 128 frames (FFT data point) from total frame count &select FFT window as 128 \u00e0For each set perform FFT \u00e0 Leakage reduction \u00e0 FFT Analysis \u00e0 For each set calculate local maxima \u00e0 For each set calculate HR \u00e0 Calculate average HR<\/p>\n

Assumption<\/strong><\/p>\n

Here, Experimental purpose: 40 \u2013 220 bpm = .667 to 3.66 Hz<\/p>\n

Sampling Frequency = 2* 3.66 Hz = 7.32 Hz (Nyquist)< 8 Hz<\/p>\n

N.B:\u00a0DC component operates at 0Hz and low frequency noise up to 0.5 Hz so filtering at above range will help PPG to get rid of DC & Low frequency noise. Higher the BP, shorter will be the duration of every heartbeat.<\/p>\n

Choice of ChannelAnd Histogram analysis [6, 17,18]]<\/strong><\/p>\n

We have picked up one fingertip video when body is at rest and selected its first 90 frames for analysis. We have obtained there brightness signal of their corresponding average pixel intensity calculation \/ frame on red, green & blue channel. Here we have retained original frame size 1920 * 1080.<\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 6.1: Red Channel Brightness Signal (Average Pixel Intensity vs Length) [Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 6.2: Blue Channel Brightness Signal (Average Pixel Intensity vs Length) [Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\"Figure<\/td>\nFigure 6.3: Green Channel Brightness Signal (Average Pixel Intensity vs Length) [Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

The result is showing that only red channel brightness signal is matching to the heart pulsatile signal properties due to uniform distribution of pixel intensity across red channel. Basic convention is that for average pixel intensity >= 200 is a good touch and we have done histogram analysis for R, G & B of the frames for further image analysis. Blue channel pixel intensity can be used for measuring AC & DC component for IR wave due to certain similarity so we can use it to measure oxygen saturation along with Red AC & DC component.<\/p>\n

Normal smartphone has few problems based on the categories<\/strong><\/p>\n

While camera is on for long time, heat is produced and that can be felt by touching for few headsets.<\/p>\n

Camera stabilization takes some time, thereby sometimes for initial 1-3 seconds,bad images of contact fingertips result in and for red channel those images have average pixel intensity of >250-255 as depicted below:<\/p>\n

Observation on Moto-G Turbo & Samsung J7 Next Gen and noisy image<\/strong><\/p>\n

Bad Frame with average pixel intensity > 254-255 on red channel<\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 7.1: Bad Capture-1 [Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

Frame with average pixel intensity > 250 \u2013 254 on red channel:<\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 7.2: Bad Capture-2 [Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

Good Frames with average pixel intensity > 200 \u2013 254 on red channel:<\/p>\n\n\n\n
\"Figure<\/td>\nFigure 7.3: Good Capture-1 [Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

So, it shows that images of captured video depends on the quality of the camera and circumstances. In case of Moto-G Turbo, systole and diastole can be captured correctly for those frames where average pixel intensity > 200 \u2013 250 on red channel. Here frames with average pixel intensity > 250 \u2013 255 on red channel, shows incursion of noise. So, we have to remove initial few of captured frames from full video (28-32 sec) that lies in the range of > 250-255. They are basically non-proper red color picture. We can’t remove a frame with mistouch in the middle of frame sequence as it might cause missing of some important point so may lead to incorrect reading. It\u2019s better to take reading again if any mistouch observed. For Samsung J7 Next Gen, systole and diastole can be captured correctly for those frames where average pixel intensity > 200 \u2013 254.6 on red channel. Sometimes, noise can be observed in the range of >254.6 to 255.<\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 8: Initial Bad Frame Capture (Average Pixel Intensity vs Length) [Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

The above diagram (left diagram: captured with Moto-G Turbo & right diagram: captured with Samsung J7 Next Gen.) shows that instability in the initial few frames (1-3 seconds). So, in the final algorithm we have removed initial few of the captured frames from full video (28-32 sec) and restricted the number to first 660 frames for all the videos (around > 20 sec. of capture).<\/p>\n

Average \/ mean intensity in red channel of image captured by Samsung J7 Next Gen is higher than image captured by Moto-G Turbo. So, creation of a threshold seems to be very difficult as it depends on video capture. More detail study has been described in the calibration section of this paper.<\/p>\n

Histogram analysis is used represent frequency of occurring of pixel intensity in the whole data set. It is very useful for image analysis (correct region identification, outlier detection, calibration detection, identifying PPG image pattern & noisy pattern) as depicted below:<\/p>\n

In the below diagrams, the green line is showing standard deviation & red line is showing the mean. The left side diagram is showing histogram analysis of red channel average intensity for the complete captured raw video before preprocessing. The right side diagram is showing corresponding brightness signal obtained.The standard deviation and mean can be used for outlier detection.<\/p>\n

Captured by Moto-G for a 31 years of healthy man with flash:<\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 9.1: Histogram analysis & raw brightness signal for signal-1[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 9.2: Histogram analysis of green & blue intensity for signal-1[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\"Figure<\/td>\nFigure 10.1: Histogram analysis & raw brightness signal for signal-2[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 10.2: Histogram analysis of green & blue intensity for signal-2[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 11.1: Histogram analysis & raw brightness signal for signal-3[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 11.2: Histogram analysis of green & blue intensity for signal-3[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 12.1: Histogram analysis & raw brightness signal for signal-4[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 12.2: Histogram analysis of green & blue intensity for signal-4[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 13.1: Histogram analysis & raw brightness signal for signal-5[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\"Figure<\/td>\nFigure 13.2: Histogram analysis of green & blue intensity for signal-5[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

Table 1: Signal Analysis Report [Chatterjee et al.]<\/strong><\/p>\n\n\n\n\n\n\n\n\n
Mean<\/strong><\/td>\nStandard dev.<\/strong><\/td>\nMin<\/strong><\/td>\nMax<\/strong><\/td>\n<\/tr>\n
237.5845<\/td>\n3.3690<\/td>\n229.2114<\/td>\n255<\/td>\n<\/tr>\n
243.4559<\/td>\n4.8610<\/td>\n236.4386<\/td>\n255<\/td>\n<\/tr>\n
246.6277<\/td>\n1.9523<\/td>\n241.8376<\/td>\n255<\/td>\n<\/tr>\n
246.4072<\/td>\n1.9284<\/td>\n237.2134<\/td>\n254.8942<\/td>\n<\/tr>\n
246.2110<\/td>\n1.7492<\/td>\n241.6756<\/td>\n255<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

For captured fingertip video flash in Moto-G Turbo set, intensity variation is close to 1.5 – 5. It is observed that, continuous flash produced some heat and felt while touching the camera during multiple capture.In the next paper we will be discussing if any noise incorporated in the image for this heat generation and sweating.<\/p>\n

Captured by Samsung J7 Next Gen for a 31 years of healthy man with flash:<\/em><\/p>\n

 <\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 14.1: Histogram analysis & raw brightness signal for signal-1[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 14.2: Histogram analysis of green & blue intensity for signal-1[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 15.1: Histogram analysis & raw brightness signal for signal-2[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\"Figure<\/td>\nFigure 15.2: Histogram analysis of green & blue intensity for signal-2[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\"Figure<\/td>\nFigure 16.1: Histogram analysis & raw brightness signal for signal-3[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 16.2: Histogram analysis of green & blue intensity for signal-3[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 17.1: Histogram analysis & raw brightness signal for signal-4[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 17.2: Histogram analysis of green & blue intensity for signal-4[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\"Figure<\/td>\nFigure 18.1: Histogram analysis & raw brightness signal for signal-5[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 18.2: Histogram analysis of green & blue intensity for signal-5[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

Table 2: Signal Analysis Report [Chatterjee et al.]<\/strong><\/p>\n\n\n\n\n\n\n\n\n
Mean<\/strong><\/td>\nStandard dev.<\/strong><\/td>\nMin<\/strong><\/td>\nMax<\/strong><\/td>\n<\/tr>\n
252.8043<\/td>\n0.8014<\/td>\n250.7790<\/td>\n255<\/td>\n<\/tr>\n
253.0314<\/td>\n0.7809<\/td>\n250.8493<\/td>\n255<\/td>\n<\/tr>\n
254.5383<\/td>\n0.3534<\/td>\n252.9994<\/td>\n255<\/td>\n<\/tr>\n
237.2475<\/td>\n3.6375<\/td>\n231.8594<\/td>\n255<\/td>\n<\/tr>\n
254.8057<\/td>\n0.1369<\/td>\n254.3104<\/td>\n255<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

For captured fingertip video flash in Samsung J7 Next Gen set, intensity variation is< 5.<\/p>\n

Captured by Samsung J7 Next Gen for a 31 years of healthy man without flash:<\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 19.1: Histogram analysis & raw brightness signal for signal-1[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\"Figure<\/td>\nFigure 19.2: Histogram analysis of green & blue intensity for signal-1[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\"Figure<\/td>\nFigure 20.1: Histogram analysis & raw brightness signal for signal-1[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\"Figure<\/td>\nFigure 20.2: Histogram analysis of green & blue intensity for signal-1[Chatterjee et al.]<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

Table 3: Signal Analysis Report [Chatterjee et al.]<\/strong><\/p>\n\n\n\n\n\n
Mean<\/strong><\/td>\nStandard dev.<\/strong><\/td>\nMin<\/strong><\/td>\nMax<\/strong><\/td>\n<\/tr>\n
30.8882<\/td>\n1.8652<\/td>\n25.5499<\/td>\n33.7479<\/td>\n<\/tr>\n
37.9662<\/td>\n9.1182<\/td>\n23.4509<\/td>\n52.8612<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

For captured fingertip video without flash, intensity variation is abrupt (more than 5) and not fit for calculating heart rate.For contact fingertip videos without flash, histogram analysis shows blue and green channel intensity per frame shifts left and for blue channel the range becomes > 0 &< 10 and for blue channel it ranges in between > 0 &< 1. From this pattern and comparing same to the frames captured with flash on, we can predict if an image is captured with flash or without flash.When flash is on, from the above histogram analysis, blue and green intensity values for good frames are less than 128 or values are lower half of the histogram.<\/p>\n

We have applied below rules to the captured videos and received true outcome in Boolean:<\/p>\n\n\n\n
\"Graph<\/td>\nGraph 1<\/strong><\/p>\n

 <\/p>\n

Click here to View\u00a0graph<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

Where, GLED<\/sub>min= 13 instead of 10 (used in other research paper)& other fixed values are as depicted above. [3,4,5]\n

Calibration<\/strong><\/p>\n

This is the step for measuring threshold for each video frame. Here we have considered 80%, 85% &90% of first 200 red channel frames pixel intensity and thereby its mean to measure the threshold. This methodology can be used for all type of smartphone videos where H*W is not fixed.<\/p>\n

T = mean(Ti<\/sub>) where Ti<\/sub>is the measured threshold for frame i.<\/p>\n

\u2211(every pixel intensity in red channel * n%) = Total average Intensity in red channel * n%, Where n = 80 \/ 85 \/90.<\/p>\n

Table 4: % of Intensity Analysis Report [Chatterjee et al.]<\/strong><\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Signal<\/strong><\/td>\n80%<\/strong><\/td>\n85%<\/strong><\/td>\n90%<\/strong><\/td>\n<\/tr>\n
Moto-1<\/td>\n190.1648<\/td>\n202.0501<\/td>\n213.9354<\/td>\n<\/tr>\n
Moto-2<\/td>\n200.0661<\/td>\n212.5702<\/td>\n225.0744<\/td>\n<\/tr>\n
Moto-3<\/td>\n197.3136<\/td>\n209.6458<\/td>\n221.9779<\/td>\n<\/tr>\n
Moto-4<\/td>\n197.3930<\/td>\n209.7300<\/td>\n222.0671<\/td>\n<\/tr>\n
Moto-5<\/td>\n197.4792<\/td>\n209.8216<\/td>\n222.1640<\/td>\n<\/tr>\n
Samsung-1<\/td>\n202.5047<\/td>\n215.1613<\/td>\n227.8178<\/td>\n<\/tr>\n
Samsung-2<\/td>\n202.7144<\/td>\n215.3840<\/td>\n228.0537<\/td>\n<\/tr>\n
Samsung-3<\/td>\n203.4036<\/td>\n216.1163<\/td>\n228.8290<\/td>\n<\/tr>\n
Samsung-4<\/td>\n191.9096<\/td>\n203.9039<\/td>\n215.8983<\/td>\n<\/tr>\n
Samsung-5<\/td>\n203.8867<\/td>\n216.6296<\/td>\n229.3725<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

From the above table, result shows that for thresh-holding point (Tp) selection, we can choose first 200 frames & obtain 90% of the average intensity variation in the red channel for every frame and sum them up or obtain 90% of the average intensity variation in the red channel for first 200 frames. The post condition is that it must be more than 200& less than minimum average intensity in red channel. It is independent of camera \/ selection.<\/p>\n

A rule can be established as below to improve the algorithm:<\/em><\/p>\n

90% of the average intensity variation in the red channel > 200 &&< (mean(red Channel intensity) \u2013 5 * standard deviation).<\/em><\/p>\n

From histogram analysis, it can be concluded that good touch frames are generally in between (mean(red Channel intensity) \u00b1 5 * standard deviation in red channel intensity).<\/em><\/p>\n

Testing6,7,11<\/sup><\/strong><\/p>\n

For maintaining originality, we have manually captured red image video of fingertips using mobile camera then used that raw *.mp4 file for analysis in MATLAB software. We have captured all videos in average voltage of light and at normal room temperature and at rest physical condition. Then the result is compared with two devices like Dr. Trust\u2019s Pulse Oximetry (FDA Approved) as shown in Figure 4 [Ayan Chatterjee et al.]&portable wearable BP Machine by MEDISANA as shown in Figure 5 [Ayan Chatterjee et al.].<\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 21.1: Pulse Oximetry Dr. Trust<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

We have used below devices, software& hardware tools<\/p>\n

Smart Phone: Moto G Turbo Edition<\/em><\/p>\n

Android: 6.0.1<\/em><\/p>\n

Camera: Rear<\/em>\u00a0<\/em><\/p>\n

Software: MATLAB Signal & Image Processing Toolbox<\/em><\/p>\n

Version:R2018b<\/em>\u00a0<\/em><\/p>\n

Hardware: Acer Laptop, Intel Celeron Processor<\/em><\/p>\n

Platform: Windows 10<\/em><\/p>\n

We have taken below HR readings from Dr. Trust<\/em>for 2 min in normal rest condition:<\/p>\n\n\n\n
\u00a0\"Figure<\/td>\nFigure 21.2: Heart Rate Reading with Dr. Trust<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n
\"Figure<\/td>\nFigure 22: Heart Rate Calculation with Video Pre-Processing (660 frames & skipping initial 2 fps)<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

Frame selection (ROI) <\/strong>[3,4,5,6]\n

Here, in our study we have used frame size of 1920*1080 and used the total area to find average pixel intensity in red channel for calculating the brightness signal. As depicted below, a close look can find a dark red and less red region of a good touch fingertip image. The boundary is oval in shape and we can assume it is nothing but top part of our finger. So, dark part seems to be unnecessary and noise can be there. We are working on less red region to find a “ROI” of fingertip image which is actual and without potential noise. If we are able to find out proper “ROI” with area < 1920*1080 then it will rather improve time & space complexity. In the second picture, we have put a black line to distinguish dark and less dark region of a fingertip image with good touch..<\/p>\n\n\n\n
\"Figure<\/td>\nFigure 23: Fingertip Image & Region Selection<\/strong><\/p>\n

 <\/p>\n

Click here to View figure<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

We are trying to use different clustering algorithms, edge detection algorithms with different operators like Prewitt, Sobel, Robinson Compass Masks, Kirsch Compass Masks, Laplacian Operator etc. with sharpening strategy to obtain “ROI”. So, it can be concluded as a part of future research paper.<\/p>\n

Conclusions<\/h5>\n

The paper has given a detail analysis on fingertip red image and popped up new questions and area where research progress can be made like ROI selection from fingertip image and finding effect of external circumstances on PPG video. A good capture will result in average intensity variation for every frame more than 200 in red channel. Average pixel intensity of 254-255 in red channel happens at the start of the video & lasts for 1-4 sec based on the situation and different set. So, it is better to avoid them. Average pixel intensity for red channel is used to consider for brightness signal due to uniform intensity distribution. Heart Rate Calculation with Video Pre-Processing (processing on initial 660 frames by skipping initial 2 fps) is a new addition to the algorithm.If any average pixel intensity for red channel is below 200 in the range of considered 660 frames then it\u2019s better to recalculate after taking video again.<\/p>\n

References<\/h5>\n
    \n
  1. Kanva K. A., \u00a0Sharma J. C.,Deb S. Determination of SpO2 and Heart-rate using Smartphone Camera, 2014 International Conference on Control, Instrumentation, Energy & Communication, 978-1-4799-2044-0\/14\/$31.00\u00a92014 IEEE.<\/li>\n
  2. \u00a0Chatterjee A and Roy U. K. PPG Based Heart Rate Algorithm Improvement with Butterworth IIR Filter and Savitzky-Golay FIR Filter, 2018 2nd International Conference on Electronics, Materials Engineering & Nano-Technology (IEMENTech), Kolkata. 2018;1-6. doi: 10.1109\/IEMENTECH. 2018;8465225.<\/li>\n
  3. \u00a0Zaman R., \u00a0Cho H. C.,Hartmann-Vaccarezza K., Phan N. T., Yoon G and\u00a0\u00a0Chong W. J. Novel Fingertip Image-Based Heart Rate Detection Methods for a Smartphone. Sensors.<\/em> 2017;17:358. doi:10.3390\/s17020358.<\/li>\n
  4. Chen L.,\u00a0 \u00a0Reisner T. A and Reifman J. Automated Beat Onset and Peak Detection Algorithm for Field-Collected Photo plethysmograms\u201d 31st Annual International Conference of the IEEE EMBS Minneapolis, Minnesota, USA. 2009;2-6.<\/li>\n
  5. \u00a0Fu T. H., Liu S. H and\u00a0 Tang K. T. Heart rate extraction from photoplethysmogram waveform using wavelet multi-resolution analysis. Journal of medical and biological engineering.\u00a0<\/em>2008;28(4):229-232.<\/li>\n
  6. Chatterjee A., Dr. Roy K. U. Non-Invasive PPG Signal based Cardiovascular Monitoring Techniques. JECET.<\/em> 2018;7(1):033-047. DOI: 10.24214\/jecet. B.7.1.03347.<\/li>\n
  7. Chatterjee A., Dr. Roy K. U. Non-Invasive CardioVascular Monitoring A Review Article on Latest PPG Signal based on Computer Science Researches. Sec. B<\/em>. 2018;3(10):55-061. Jan DOI : 0.18231\/2454-9150.2018.0010.<\/li>\n
  8. Chatterjee A., Dr.\u00a0Roy K. U. Algorithm To Calculate Heart Rate & Comparison Of Butterworth IIR and Savitzky-Golay FIR Filter. 2018;11(2). DOI : 10.4172\/jcsb.1000268.<\/li>\n
  9. Shamir M.,\u00a0 Eidelman L. A., Floman Y., Kaplan L and Pizov R. Pulse oximetry plethysmographic waveform during changes in blood volume. Br J Anaesth.<\/em> 1999;82(2):178-81.<\/li>\n
  10. Golparvar M.,\u00a0 Naddafnia H and Saghaei M. Evaluating the relationship between arterial blood pressure changes and indices of pulse oximetric plethysmography. Anesth Analg.<\/em> 2002;95(6):1686-90.<\/li>\n
  11. Chan M. Filtering and Signal-Averaging Algorithms for Raw ECG Signals\u201d, Ese 482 Digital Signal Processing – Washington University In Saint Louis – Final Project. 2010. December 14.<\/li>\n
  12. \u00a0Jo J.,\u00a0 Lee Y and\u00a0Shin S. H. Real-time Analysis of Heart Rate Variability for a Mobile Human Emotion Recognition System. 2009;305-700.<\/li>\n
  13. \u00a0Lee Y., Shin H. S and Jo J. Development of a PPG array sensor module, Proc. Institute of Electronics Engineers of Korea (IEEK) Summer Conference, Seoul, Korea. 2010;1368-1370.<\/li>\n
  14. \u00a0Scully C. G.,\u00a0 Lee J., Meyer J.,Gorbach A. M.,\u00a0 Granquist-Fraser D., Mendelson Y and\u00a0 Chon K. H. Physiological parameter monitoring from optical recordings with a mobile phone. IEEE Transactions on Biomedical Engineering.<\/em> 2012;59(2):303-306.<\/li>\n
  15. Lee Y. K., Kwon O. W.,\u00a0 Shin H. S., Jo J and\u00a0 Lee Y. Noise reduction of PPG signals using a particle filter for robust emotion recognition, Proc. IEEE ICCE-Berlin. 2011;202-205.<\/li>\n
  16. Chan M. Filtering and Signal-Averaging Algorithms for Raw ECG Signals\u201d, Ese 482 Digital Signal Processing – Washington University In Saint Louis – Final Project – 2010. December 14.<\/li>\n
  17. Grimaldi D., Kurylyak Y., Lamonaca F and\u00a0 Nastro A. Photoplethysmography de-tection by smartphone’s videocamera,” Proceedings of the 6th IEEE International Con-ference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications.\u00a0IDAACS.\u00a0<\/em>2011;1:488-491.<\/li>\n
  18. Lamonaca F., Kurylyak Y., Grimaldi D and Spagnuolo V.\u00a0 Reliable pulse rate evalua-tion by smartphone,MeMeA 2012-2012 IEEE Symposium on Medical Measurements and Applications.\u00a0Proceedings.<\/em> 2012;1:234-237.<\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"

    Introduction PPG signal can be described as non-invasive, inexpensive, portable […]<\/p>\n","protected":false},"author":9,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[61],"tags":[],"class_list":["post-25019","post","type-post","status-publish","format-standard","hentry","category-vol11no4"],"_links":{"self":[{"href":"https:\/\/biomedpharmajournal.org\/staging\/wp-json\/wp\/v2\/posts\/25019","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/biomedpharmajournal.org\/staging\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/biomedpharmajournal.org\/staging\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/biomedpharmajournal.org\/staging\/wp-json\/wp\/v2\/users\/9"}],"replies":[{"embeddable":true,"href":"https:\/\/biomedpharmajournal.org\/staging\/wp-json\/wp\/v2\/comments?post=25019"}],"version-history":[{"count":7,"href":"https:\/\/biomedpharmajournal.org\/staging\/wp-json\/wp\/v2\/posts\/25019\/revisions"}],"predecessor-version":[{"id":32554,"href":"https:\/\/biomedpharmajournal.org\/staging\/wp-json\/wp\/v2\/posts\/25019\/revisions\/32554"}],"wp:attachment":[{"href":"https:\/\/biomedpharmajournal.org\/staging\/wp-json\/wp\/v2\/media?parent=25019"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/biomedpharmajournal.org\/staging\/wp-json\/wp\/v2\/categories?post=25019"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/biomedpharmajournal.org\/staging\/wp-json\/wp\/v2\/tags?post=25019"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}