Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Materials and Methods
Training Dataset
Validation Dataset
Chi-Square Analysis
Optical Flow Using Lucas-Kanade Method
Analysis of Feature Matching
Results
Generalisation to PD Data
Discussion and Conclusions
Data
2 Statistics of the Training Dataset
References
All Topics
Computer Science
Computer Vision

Skin feature point tracking using deep feature encodings

Profile image of Jose Ramon Chang IriasJose Ramon Chang Irias

2021, arXiv (Cornell University)

https://doi.org/10.48550/ARXIV.2112.14159
visibility

…

description

38 pages

link

1 file

Abstract

Facial feature tracking is a key component of imaging ballistocardiography (BCG) where accurate quantification of the displacement of facial keypoints is needed for good heart rate estimation. Skin feature tracking enables video-based quantification of motor degradation in Parkinson's disease. Traditional computer vision algorithms include Scale Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), and Lucas-Kanade method (LK). These have long represented the state-of-the-art in efficiency and accuracy but fail when common deformations, like affine local transformations or illumination changes, are present. Over the past five years, deep convolutional neural networks have outperformed traditional methods for most computer vision tasks. We propose a pipeline for feature tracking, that applies a convolutional stacked autoencoder to identify the most similar crop in an image to a reference crop containing the feature of interest. The autoencoder learns to represent image crops into deep feature encodings specific to the object category it is trained on. We train the autoencoder on facial images and validate its ability to track skin features in general using manually labeled face and hand videos. The tracking errors of distinctive skin features (moles) are so small that we cannot exclude that they stem from the manual labelling based on a χ 2 -test. With a mean error of 0.6-4.2 pixels, our method outperformed the other methods in all but one scenario. More importantly, our method was the only one to not diverge. We conclude that our method creates better feature descriptors for feature tracking, feature matching, and image registration than the traditional algorithms.

Related papers

Deep learning approach for Parkinson's screening with geometric features from spiral and wave drawings
Edeh Michael Onyema, Ph.D

Springer (Multimedia Tools and Application), 2025

Parkinson's disease (PD) is the second most common neurological ailment and affects a substantial percentage of the elderly population. A deterioration in handwriting is a common indication of Parkinson's disease. Researchers have developed an algorithmbased tool to assist in distinguishing between Parkinson's disease patients and healthy persons because the disease can be difficult to identify. Unfortunately, previous studies have not tackled this problem or adopted a preventive approach. Examining handwriting is one way to detect PD. After analyzing multiple publications, we examined the 2020 and 2022 articles from Convolutional Neural Network utilizing deep learning techniques. Spiral and wave photos of those in good health and those with Parkinson's illness were employed in our collection. This investigation used the Random Forest, VGG-16, and ResNet-50 approaches. We applied ResNet-50 for the drawings and got 97% accuracy for spiral pictures over 10 epochs and 98% accuracy for wave images over eight epochs by using feature extraction and preprocessing techniques. This exceptional performance demonstrates ResNet-50's potential as a potent tool for medical diagnostics, utilizing its deep learning skills to generate precise and reliable results compared to existing studies in which ResNet-50 has 83% accuracy.

View PDFchevron_right
Monitoring Neurological Disorder Patients via Deep Learning Based Facial Expressions Analysis
Muhammad Munsif

Springer Nature Switzerland, 2022

Facial expression (FE) is the most natural and convincing source to communicate human emotions, providing valuable insides to the observer while assessing the emotional incongruities. In health care, the FE of the patient (specifically of neurological disorders (NDs) such as Parkinson's, Stroke, and Alzheimer's) can assist the medical doctor in evaluating the physical condition of a patient, such as fatigue, pain, and sadness. ND patients are usually going through proper observation and clinical tests, which are invasive, expensive and time-consuming. In this paper, an automatic lightweight deep learning (DL) based FEs recognition framework is developed that can classify the facial expression of ND patients with 93% accuracy. Initially, raw images of FEs are acquired from publicly available datasets according to the patient's most common expressions, such as normal, happy, sad, and anger. The framework cropped images through a face detector, extract high-level facial features through the convolutional layers and fed them to the dense layers for classification. The trained model is exported to an android based environment over a smart device and evaluated for real-time performance. The qualitative and quantitative results are evaluated on a standard dataset named Karolinska directed emotional faces (KDEF). Promising results are obtained of various NDs patients with Parkinson, Stroke, and Alzheimer that show the effectiveness of the proposed model.

View PDFchevron_right
Wearable-Based Parkinson’s Disease Severity Monitoring Using Deep Learning
Urban M Fietzek

Lecture Notes in Computer Science, 2020

One major challenge in the medication of Parkinson's disease is that the severity of the disease, reflected in the patients' motor state, cannot be measured using accessible biomarkers. Therefore, we develop and examine a variety of statistical models to detect the motor state of such patients based on sensor data from a wearable device. We find that deep learning models consistently outperform a classical machine learning model applied on hand-crafted features in this time series classification task. Furthermore, our results suggest that treating this problem as a regression instead of an ordinal regression or a classification task is most appropriate. For consistent model evaluation and training, we adopt the leave-one-subject-out validation scheme to the training of deep learning models. We also employ a class-weighting scheme to successfully mitigate the problem of high multi-class imbalances in this domain. In addition, we propose a customized performance measure that reflects the requirements of the involved medical staff on the model. To solve the problem of limited availability of high quality training data, we propose a transfer learning technique which helps to improve model performance substantially. Our results suggest that deep learning techniques offer a high potential to autonomously detect motor states of patients with Parkinson's disease.

View PDFchevron_right
Granular Motor State Monitoring of Free Living Parkinson's Disease Patients via Deep Learning
Urban M Fietzek

ArXiv, 2019

Parkinson's disease (PD) is the second most common neurodegenerative disease worldwide and affects around 1% of the (60+ years old) elderly population in industrial nations. More than 80% of PD patients suffer from motor symptoms, which could be well addressed if a personalized medication schedule and dosage could be administered to them. However, such personalized medication schedule requires a continuous, objective and precise measurement of motor symptoms experienced by the patients during their regular daily activities. In this work, we propose the use of a wrist-worn smart-watch, which is equipped with 3D motion sensors, for estimating the motor fluctuation severity of PD patients in a free-living environment. We introduce a novel network architecture, a post-training scheme and a custom loss function that accounts for label noise to improve the results of our previous work in this domain and to establish a novel benchmark for nine-level PD motor state estimation.

View PDFchevron_right
Matching Medical Images with Deep Learning Networks: A Survey
rama prasad

International Journal of Engineering Research & Advanced Technology (IJERAT), 2025

Many patients' lives are being saved by image-guided interventions, and the image registration issue must be regarded as the most difficult and complex problem to solve. However, the latest enormous advancements in machine learning (ML), which include the potential to deploy deep neural networks (DNNs) on modern many-core GPUs, have created a promising opportunity for tackling a variety of medical applications, including registration. The most recent research on medical image registration with DNNs is reviewed in detail in the presented work. All of the relevant papers that have already been published in the subject are included in the systematic review. This thorough overview includes a detailed discussion as well as survey of important ideas, statistical analysis from many perspectives, novelties and key contributions, confiding challenges, future directions, key-enabling approaches, and prospective trends. For readers who are actively involved in the subject, researching state-of-the-art and hoping to present a contribution to future publications, the presented study offers a deep grasp and insight.

View PDFchevron_right
A Spectrogram-Based Deep Feature Assisted Computer-Aided Diagnostic System for Parkinson’s Disease
Laiba Zahid

IEEE Access

Parkinson's disease is a neural degenerative disease. It slowly progresses from mild to severe stage, resulting in the degeneration of dopamine cells of neurons. Due to the deficiency of dopamine cells in the brain, it leads to a motor (tremor, slowness, impaired posture) and non-motor (speech, olfactory) defects in the body. Early detection of Parkinson's disease is a difficult chore as the symptoms of disease appear overtime. However, different diagnostic systems have contributed towards disease detection by considering gait, tremor and speech characteristics. Recent work has shown that speech impairments can be considered as a possible predictor for Parkinson's disease classification and remains an open research area. The speech signals show major differences and variations for Parkinson patients as compared to normal human beings. Therefore, variation in speech should be modeled using acoustic features to identify these variations. In this research, we propose three methods-the first method employs a transfer learning-based approach using spectrograms of speech recordings, the second method evaluates deep features extracted from speech spectrograms using machine learning classifiers and the third method evaluates simple acoustic feature of recordings using machine learning classifiers. The proposed frameworks are evaluated on a Spanish dataset pc-Gita. The results show that the second framework shows promising results with deep features. The highest 99.7% accuracy on vowel \o\ and read text is observed using a multilayer perceptron. Whereas 99.1% accuracy observed on vowel \i\ deep features using random forest. The deep feature-based method performs better as compared to simple acoustic features and transfer learning approaches. The proposed methodology outperforms the existing techniques on the pc-Gita dataset for Parkinson's disease detection.

View PDFchevron_right
Deep Learning for Daily Monitoring of Parkinson’s Disease Outside the Clinic Using Wearable Sensors
Lee Lancashire

Sensors

Now that wearable sensors have become more commonplace, it is possible to monitor individual healthcare-related activity outside the clinic, unleashing potential for early detection of events in diseases such as Parkinson’s disease (PD). However, the unsupervised and “open world” nature of this type of data collection make such applications difficult to develop. In this proof-of-concept study, we used inertial sensor data from Verily Study Watches worn by individuals for up to 23 h per day over several months to distinguish between seven subjects with PD and four without. Since motor-related PD symptoms such as bradykinesia and gait abnormalities typically present when a PD subject is walking, we initially used human activity recognition (HAR) techniques to identify walk-like activity in the unconstrained, unlabeled data. We then used these “walk-like” events to train one-dimensional convolutional neural networks (1D-CNNs) to determine the presence of PD. We report classification ac...

View PDFchevron_right
Deep-HR: Fast heart rate estimation from face video under realistic conditions
Mohammad Sabokrou

Expert Systems with Applications, 2021

This paper presents a novel method for remote heart rate (HR) estimation. Recent studies have proved that blood pumping by the heart is highly correlated to the intense color of face pixels, and surprisingly can be utilized for remote HR estimation. Researchers successfully proposed several methods for this task, but making it work in realistic situations is still a challenging problem in computer vision community. Furthermore, learning to solve such a complex task on a dataset with very limited annotated samples is not reasonable. Consequently, researchers do not prefer to use the deep learning approaches for this problem. In this paper, we propose a simple yet efficient approach to benefit the advantages of the Deep Neural Network (DNN) by simplifying HR estimation from a complex task to learning from very correlated representation to HR. Inspired by previous work, we learn a component called Front-End (FE) to provide a discriminative representation of face videos, afterward a light deep regression auto-encoder as Back-End (BE) is learned to map the FE representation to HR. Regression task on the informative representation is simple and could be learned efficiently on limited training samples. Beside of this, to be more accurate and work well on low-quality videos, two deep encoder-decoder networks are trained to refine the output of FE. We also introduce a challenging dataset (HR-D) to show that our method can efficiently work in realistic conditions. Experimental results on HR-D and MAHNOB datasets confirm that our method could run as a real-time method and estimate the average HR better than state-of-the-art ones.

View PDFchevron_right
Using deep autoencoders to learn robust domain-invariant representations for still-to-video face recognition
saif sayed

2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2017

Video-based face recognition (FR) is a challenging task in real-world applications. In still-to-video FR, probe facial regions of interest (ROIs) are typically captured with lower-quality video cameras under unconstrained conditions, where facial appearances vary according to pose, illumination, scale, expression, etc. These video ROIs are typically compared against facial models designed with high-quality reference still ROI of each target individual enrolled to the system. In this paper, an efficient Canonical Face Representation CNN (CFR-CNN) is proposed for accurate still-to-video FR from a single sample per person, where still and video ROIs are captured in different conditions. Given a facial ROI captured under unconstrained video conditions, the CRF-CNN reconstructs it as a highquality canonical ROI for matching that corresponds to the conditons of reference still ROIs (e.g., well-illuminated, sharp, frontal views with neutral expression). A deep autoencoder network is trained using a novel weighted loss function that can robustly generate similar face embeddings for the same subjects. Then, during operations, those face embeddings belonging to pairs of still and video ROIs from a target individual are accurately matched using a fullyconnected classification network. Experimental results obtained with the COX Face and Chokepoint datasets indicate that the proposed CFR-CNN can achieve convincing level of accuracy. The computational complexity (number of operations, network parameters and layers) is significantly lower than state-of-the-art CNNs for video FR, and suggests that the CFR-CNN represents a cost-effective solution for realtime applications.

View PDFchevron_right
An Approach to Differential Evaluation and Early Detection of Parkinson's Disease using Image Processors, and Machine Learning Authors: Yashvardhan Mayur Verma (1) ; Reetu Jain (2
iJournals Academic Publications IJSHRE | IJSRC

Parkinson's disease is a disorder affecting the motor capabilities of the individual. The typical symptoms include freezing of gait, bradykinesia, deterioration of handwriting, and impaired voice and speech abilities. The early detection of Parkinson's Disease allows patients to begin therapy and management strategies at stages of the disease that are not too severe and irreversible. This is to prevent the effects of advanced stages of Parkinson's that can severely degrade the quality of life of the patient. Several measures can be taken to prevent this and come to a conclusive diagnosis. After this, therapy to reverse the effects, or decrease further progression of Parkinson's can be expedited. There can be several factors that can support the diagnosis and these factors can be tested using several methods, such as speech processing, handwriting image processing, and body acceleration during day-today movements. For this research our dataset includes handwriting samples from 36 subjects. In this work we have researched on the application of Deep learning architectures using handwritten images of 0 and 1. In this paper, an attempt has been made to diagnose Parkinson's disease by analyzing these hand-drawn images of 0 and 1 and training a Convolutional Neural Network (CNN) to do a binary classification for different drawing patterns giving an accuracy score close to 60%. Our methods use machine learning techniques and can give conclusive results and aid clinicians in the diagnostic process.

View PDFchevron_right

References (96)

  1. Milton Abramowitz and Irene A Stegun. Handbook of mathe- matical functions with formulas, graphs, and mathematical tables, 1988.
  2. Yassine Ahmine, Guillaume Caron, El Mustapha Mouaddib, and Fatima Chouireb. Adaptive lucas-kanade tracking. Im- age and Vis. Comput., 88:1-8, 2019.
  3. Haldun Akoglu. User's guide to correlation coefficients. Turk- ish Journal of Emergency Medicine, 18(3):91-93, 2018.
  4. Hesham A Alberry, Abdelfatah A Hegazy, and Gouda I Salama. A fast sift based method for copy move forgery detection. Future Comput. and Inform. J., 3(2):159-165, 2018.
  5. Matthew Anderson, Ricardo Motta, Srinivasan Chandrasekar, and Michael Stokes. Proposal for a standard default color space for the internet-srgb. In Color and imaging confer- ence, volume 1996, pages 238-245. Society for Imaging Sci- ence and Technology, 1996.
  6. Sadaf Ansari. A review on sift and surf for underwater image feature detection and matching. In 2019 IEEE International Conference on Electrical, Computer and Communication Technologies (ICECCT), pages 1-4. IEEE, 2019.
  7. Akram Ashyani, Chi-Lun Lin, Esteban Roman, Ted Yeh, Tachyon Kuo, Wei-Fang Tsai, Yushan Lin, Ric Tu, Austin Su, Chien-Chih Wang, Chun-Hsiang Tan, and Torbjörn E M Nordling. Digitization of updrs upper limb motor exami- nations towards automated quantification of symptoms of parkinson's disease. Manuscript in preparation, 2022.
  8. Simon Baker, Daniel Scharstein, JP Lewis, Stefan Roth, Michael J Black, and Richard Szeliski. A database and evaluation methodology for optical flow. Int. J. of Com- put. Vis., 92(1):1-31, 2011.
  9. Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. Surf: Speeded up robust features. In European conference on computer vision, pages 404-417. Springer, 2006.
  10. Jeffrey S Beis and David G Lowe. Shape indexing using approx- imate nearest-neighbour search in high-dimensional spaces. In Proceedings of IEEE computer society conference on computer vision and pattern recognition, pages 1000-1006. IEEE, 1997.
  11. Fangming Bi, Xin Ma, Wei Chen, Weidong Fang, Huayi Chen, Jingru Li, and Biruk Assefa. Review on video object track- ing based on deep learning. J. of New Media, 1(2):63, 2019.
  12. Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Liao. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint, 2020.
  13. Jean-Yves Bouguet. Pyramidal implementation of the affine lucas kanade feature tracker description of the algorithm. Intel Corporation, 5(1-10):4, 2001.
  14. Daniel J Butler, Jonas Wulff, Garrett B Stanley, and Michael J Black. A naturalistic open source movie for optical flow eval- uation. In European conference on computer vision, pages 611-625. Springer, 2012.
  15. Che-Han Chang, Chun-Nan Chou, and Edward Y Chang. Clkn: Cascaded lucas-kanade networks for image alignment. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2213-2221, 2017.
  16. Chien-Chang Chen, Wei-Yu Lu, and Chung-Hsuan Chou. Ro- tational copy-move forgery detection using sift and region growing strategies. Multimed. Tools and Appl., 78(13): 18293-18308, 2019.
  17. Chun-Hong Cheng, Kwan-Long Wong, Jing-Wei Chin, Tsz-Tai Chan, and Richard HY So. Deep learning methods for re- mote heart rate measurement: a review and future research agenda. Sensors, 21(18):6296, 2021a.
  18. Yihua Cheng, Haofei Wang, Yiwei Bao, and Feng Lu. Appearance-based gaze estimation with deep learning: A review and benchmark. arXiv preprint arXiv:2104.12668, 2021b.
  19. Hsiang-Jen Chien, Chen-Chi Chuang, Chia-Yen Chen, and Reinhard Klette. When to use what feature? sift, surf, orb, or a-kaze features for monocular visual odometry. In 2016 International Conference on Image and Vision Computing New Zealand (IVCNZ), pages 1-6. IEEE, 2016.
  20. Lovish Chum, Anbumani Subramanian, Vineeth N Balasubra- manian, and CV Jawahar. Beyond supervised learning: a computer vision perspective. J. of the Indian Inst. of Sci., 99(2):177-199, 2019.
  21. Joon Son Chung, Andrew Senior, Oriol Vinyals, and Andrew Zisserman. Lip reading sentences in the wild. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3444-3453. IEEE, 2017.
  22. Gioele Ciaparrone, Francisco Luque Sánchez, Siham Tabik, Luigi Troiano, Roberto Tagliaferri, and Francisco Herrera. Deep learning in video multi-object tracking: A survey. Neurocomputing, 381:61-88, 2020.
  23. Philippe Colantoni, Jean-Baptiste Thomas, and Alain Trémeau. Sampling cielab color space with perceptual met- rics. International Journal of Imaging and Robotics, 16(3): 1-22, 2016.
  24. Jifeng Dai, Yi Li, Kaiming He, and Jian Sun. R-fcn: Object detection via region-based fully convolutional networks. In Advances in neural information processing systems, pages 379-387, 2016.
  25. Alexey Dosovitskiy, Philipp Fischer, Eddy Ilg, Philip Hausser, Caner Hazirbas, Vladimir Golkov, Patrick Van Der Smagt, Daniel Cremers, and Thomas Brox. Flownet: Learning opti- cal flow with convolutional networks. In Proceedings of the IEEE international conference on computer vision, pages 2758-2766, 2015.
  26. Youssef Douini, Jamal Riffi, Adnane Mohamed Mahraz, and Hamid Tairi. An image registration algorithm based on phase correlation and the classical lucas-kanade technique. Signal, Image and Video Process., 11(7):1321-1328, 2017a.
  27. Youssef Douini, Jamal Riffi, Mohamed Adnane Mahraz, and Hamid Tairi. Solving sub-pixel image registration prob- lems using phase correlation and lucas-kanade optical flow method. In 2017 Intelligent Systems and Computer Vision (ISCV), pages 1-5. IEEE, 2017b.
  28. Iyad Abu Doush and AL-Btoush Sahar. Currency recognition using a smartphone: Comparison between color sift and gray scale sift algorithms. J. of King Saud University-Computer and Inf. Sci., 29(4):484-492, 2017.
  29. Vincent Dumoulin and Francesco Visin. A guide to convolution arithmetic for deep learning. arXiv preprint, 2018.
  30. Guangyu Gao, Liling Liu, Li Wang, and Yihang Zhang. Fash- ion clothes matching scheme based on siamese network and autoencoder. Multimed. Syst., 25(6):593-602, 2019.
  31. I Garcia, Sebastian Bronte, Luis Miguel Bergasa, Javier Al- mazán, and J Yebes. Vision-based drowsiness detector for real driving conditions. In 2012 IEEE Intelligent Vehicles Symposium, pages 618-623. IEEE, 2012.
  32. Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset. The International Journal of Robotics Research, 32(11):1231- 1237, 2013.
  33. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT Press, Cambridge, MA, U.S.A., 2016. ISBN 978-0262035613. URL http://www.deeplearningbook.org.
  34. Albert Gordo, Jon Almazán, Jerome Revaud, and Diane Lar- lus. Deep image retrieval: Learning global representations for image search. In European conference on computer vi- sion, pages 241-257. Springer, 2016.
  35. Xifeng Guo, Xinwang Liu, En Zhu, and Jianping Yin. Deep clustering with convolutional autoencoders. In Derong Liu, Shengli Xie, Yuanqing Li, Dongbin Zhao, and El-Sayed M. El-Alfy, editors, Neural Information Processing, pages 373- 382, Cham, 2017. Springer International Publishing. ISBN 978-3-319-70096-0.
  36. Amir HajiRassouliha, Andrew J Taberner, Martyn P Nash, and Poul MF Nielsen. Subpixel phase-based image reg- istration using savitzky-golay differentiators in gradient- correlation. Comput. Vis. and Image Underst., 170:28-39, 2018.
  37. M. A. Hassan, A. S. Malik, D. Fofi, N. Saad, B. Karasfi, Y. S. Ali, and F. Meriaudeau. Heart rate estimation using facial video: A review. Biomed. Signal Proces. and Control., 38: 346-360, 2017. ISSN 17468108.
  38. K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770-778, June 2016. doi: 10.1109/CVPR.2016.90.
  39. Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Gir- shick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961-2969, 2017a.
  40. Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Gir- shick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961-2969, 2017b.
  41. G.E. Hinton and R. R. Salakhutdinov. Reducing the dimen- sionality of data with neural networks. Sci., 313(July):504- 507, 2006.
  42. Eric B Holmgren. The pp plot as a method for comparing treatment effects. J. of the Am. Stat. Assoc., 90(429):360- 365, 1995.
  43. Berthold KP Horn and Brian G Schunck. Determining optical flow. In Techniques and Applications of Image Understand- ing, volume 281, pages 319-331. International Society for Optics and Photonics, 1981.
  44. Borui Hou and Ruqiang Yan. Convolutional autoencoder model for finger-vein verification. IEEE Transactions on Instrumentation and Measurement, 69(5):2067-2074, 2019.
  45. Abdullah Ayub Khan, Asif Ali Laghari, and Shafique Ahmed Awan. Machine learning in computer vision: A review. EAI Trans. on Scalable Inf. Syst., page e4, 2021.
  46. Nabeel Khan, Brendan McCane, and Steven Mills. Better than sift? Mach. Vis. and Appl., 26(6):819-836, 2015.
  47. Diederik P Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. arXiv preprint, pages 1-15, 2014. ISSN 09252312.
  48. Vladimir A Knyaz, Oleg Vygolov, Vladimir V Kniaz, Yury Vizilter, Vladimir Gorbatsevich, Thomas Luhmann, and Niklas Conen. Deep learning of convolutional auto-encoder for image matching and 3d object reconstruction in the in- frared range. In Proceedings of the IEEE International Con- ference on Computer Vision Workshops, pages 2155-2164, 2017.
  49. H Law and J Deng. Detecting objects as paired keypoints. Lect. Notes in Comput. Sci., pages 765-781, 2018.
  50. Yann LeCun, Koray Kavukcuoglu, and Clément Farabet. Con- volutional networks and applications in vision. In Proceed- ings of 2010 IEEE international symposium on circuits and systems, pages 253-256. IEEE, 2010.
  51. Wookey Lee, Jessica Jiwon Seong, Busra Ozlu, Bong Sup Shim, Azizbek Marakhimov, and Suan Lee. Biosignal sensors and deep learning-based speech recognition: A review. Sensors, 21(4):1399, 2021.
  52. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pages 740-755. Springer, 2014.
  53. Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyra- mid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 2117-2125, 2017a.
  54. Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Pro- ceedings of the IEEE international conference on computer vision, pages 2980-2988, 2017b.
  55. Li Liu, Wanli Ouyang, Xiaogang Wang, Paul Fieguth, Jie Chen, Xinwang Liu, and Matti Pietikäinen. Deep learn- ing for generic object detection: A survey. International Journal of Computer Vision, 128(2):261-318, 2020.
  56. Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In European con- ference on computer vision, pages 21-37. Springer, 2016.
  57. David G. Lowe. Distinctive image features from scale-invariant keypoints. Int. J. of Comp. Vis., 60(2):91-110, 2004. ISSN 09205691.
  58. Bruce Lucas and Takeo Kanade. An iterative image regis- tration technique with an application to stereo vision. In IJCAI, volume 81, 04 1981.
  59. Ilja Manakov, Markus Rohm, and Volker Tresp. Walking the tightrope: an investigation of the convolutional autoencoder bottleneck. arXiv preprint, 2019.
  60. Francesca Manni, Fons van der Sommen, Svitlana Zinger, Caifeng Shan, Ronald Holthuizen, Marco Lai, Gustav Buström, Richelle JM Hoveling, Erik Edström, Adrian Elmi-Terander, et al. Hyperspectral imaging for skin fea- ture detection: Advances in markerless tracking for spine surgery. Applied Sciences, 10(12):4078, 2020.
  61. K McLaren. Xiii-the development of the cie 1976 (l* a* b*) uniform colour space and colour-difference formula. J. of the Soc. of Dyers and Colour., 92(9):338-341, 1976.
  62. Krystian Mikolajczyk and Cordelia Schmid. A performance evaluation of local descriptors. IEEE transactions on pat- tern analysis and machine intelligence, 27(10):1615-1630, 2005.
  63. Hartmut Neven, Geordie Rose, and William G Macready. Im- age recognition with an adiabatic quantum computer i. map- ping to quadratic unconstrained binary optimization. arXiv preprint arXiv:0804.4457, 2008.
  64. Aoxin Ni, Arian Azarang, and Nasser Kehtarnavaz. A review of deep learning-based contactless heart rate measurement methods. Sensors, 21(11):3719, 2021.
  65. Hyeonwoo Noh, Andre Araujo, Jack Sim, Tobias Weyand, and Bohyung Han. Large-scale image retrieval with atten- tive deep local features. In Proceedings of the IEEE inter- national conference on computer vision, pages 3456-3465, 2017.
  66. Xiaorong Pu, Ke Fan, Xiong Chen, Luping Ji, and Zhihu Zhou. Facial expression recognition from image sequences using twofold random forest classifier. Neurocomputing, 168:1173- 1180, 2015.
  67. Joseph Redmon and Ali Farhadi. Yolo9000: better, faster, stronger. In Proceedings of the IEEE conference on com- puter vision and pattern recognition, pages 7263-7271, 2017.
  68. Joseph Redmon and Ali Farhadi. Yolov3: An incremental im- provement. arXiv preprint, 2018.
  69. Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object de- tection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 779-788, 2016.
  70. Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with re- gion proposal networks. In Advances in neural information processing systems, pages 91-99, 2015.
  71. Rasmus Rothe, Radu Timofte, and Luc Van Gool. Deep expec- tation of real and apparent age from a single image without facial landmarks. Int. J. of Comp. Vis., 126(2-4):144-157, 2018.
  72. Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superglue: Learning feature match- ing with graph neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4938-4947, 2020.
  73. Gulbadan Sikander and Shahzad Anwar. Driver fatigue detec- tion systems: A review. IEEE Transactions on Intelligent Transportation Systems, 20(6):2339-2352, 2018.
  74. Edgar Simo-Serra, Eduard Trulls, Luis Ferraz, Iasonas Kokki- nos, Pascal Fua, and Francesc Moreno-Noguer. Discrimi- native learning of deep convolutional feature point descrip- tors. In Proceedings of the IEEE international conference on computer vision, pages 118-126, 2015.
  75. Marzuraikah Mohd Stofa, Mohd Asyraf Zulkifley, and Muham- mad Ammirrul Atiqi Mohd Zainuri. Skin lesions classifica- tion and segmentation: A review. International Journal of Advanced Computer Science and Applications, 12(10), 2021.
  76. Peifeng Su, Daizhi Liu, Xihai Li, and Zhigang Liu. A saliency- based band selection approach for hyperspectral imagery inspired by scale selection. IEEE Geoscience and Remote Sensing Letters, 15(4):572-576, 2018.
  77. Jiaming Sun, Zehong Shen, Yuang Wang, Hujun Bao, and Xi- aowei Zhou. Loftr: Detector-free local feature matching with transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8922- 8931, 2021.
  78. Christian Szegedy, Liu Wei, Jia Yangqing, Sermanet Pierre, Reed Scott, Anguelov Dragomir, Erham Dumitru, Van- houcke Vincent, and Rabinovich Andrew. Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. Popul. Health Manag., 18(3):186-191, 2015. ISSN 1942-7891. doi: 10.1089/pop.2014.0089. URL http: //online.liebertpub.com/doi/10.1089/pop.2014.0089.
  79. Shaharyar Ahmed Khan Tareen and Zahra Saleem. A compar- ative analysis of sift, surf, kaze, akaze, orb, and brisk. In 2018 International conference on computing, mathematics and engineering technologies (iCoMET), pages 1-10. IEEE, 2018.
  80. Jose Miguel Tarongi and Adriano Camps. Normality analysis for rfi detection in microwave radiometry. Remote Sensing, 2(1):191-210, 2010.
  81. George Brinton Thomas and Ross L Finney. Calculus And Analytic Geometry. Addison-Wesley Publishing Company, 1900 E Lake Ave Glenview, IL 60025 United States, 1961.
  82. Quoc-Viet Tran, Shun-Feng Su, and Van-Truong Nguyen. Pyramidal lucas-kanade-based noncontact breath motion detection. IEEE Transactions on Systems, Man, and Cy- bernetics: Systems, 50(7):2659-2670, 2018.
  83. Marco C. Uchida, Renato Carvalho, Vitor Daniel Tessutti, Reury Frank Pereira Bacurau, Hélio José Coelho-Júnior, Luciane Portas Capelo, Heloiza Prando Ramos, Marcia Cal- ixto dos Santos, Luís Felipe Milano Teixeira, and Paulo Hen- rique Marchetti. Identification of muscle fatigue by tracking facial expressions. PLoS ONE, 13(12):1-11, 2018. ISSN 19326203.
  84. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkor- eit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages 5998-6008, 2017.
  85. Chien-Chih Wang. Non-contact heart rate measurement based on facial videos. Master's thesis, National Cheng Kung Uni- versity, No. 1, Dasyue Rd, East District, Tainan City, 701, 2020.
  86. Nannan Wang, Xinbo Gao, Dacheng Tao, Heng Yang, and Xuelong Li. Facial feature point detection: A comprehensive survey. Neurocomputing, 275:50-65, 2018. ISSN 18728286.
  87. Philippe Weinzaepfel, Jerome Revaud, Zaid Harchaoui, and Cordelia Schmid. Deepflow: Large displacement optical flow with deep matching. In Proceedings of the IEEE inter- national conference on computer vision, pages 1385-1392, 2013.
  88. Xue Iuan Wong and Manoranjan Majji. Uncertainty quantifi- cation of lucas kanade feature track and application to visual odometry. In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 950- 958. IEEE, 2017.
  89. Jure Zbontar and Yann LeCun. Stereo matching by training a convolutional neural network to compare image patches. J. of Mach. Learn. Res., 17(1):2287-2318, 2016.
  90. Matthew D Zeiler, Dilip Krishnan, Graham W Taylor, and Rob Fergus. Deconvolutional networks. In 2010 IEEE Computer Society Conference on computer vision and pattern recog- nition, pages 2528-2535. IEEE, 2010.
  91. Zhifei Zhang, Yang Song, and Hairong Qi. Age progres- sion/regression by conditional adversarial autoencoder. In IEEE Conference on Computer Vision and Pattern Recog- nition (CVPR). IEEE, 2017.
  92. Amy Zhao, Frédo Durand, and John Guttag. Estimating a small signal in the presence of large noise. In Proceed- ings of the IEEE International Conference on Computer Vision, volume 2016-Febru, pages 671-676, 2016. ISBN 9781467383905.
  93. Jianqing Zhao, Xiaohu Zhang, Chenxi Gao, Xiaolei Qiu, Yongchao Tian, Yan Zhu, and Weixing Cao. Rapid mo- saicking of unmanned aerial vehicle (uav) images for crop growth monitoring using the sift algorithm. Remote. Sens., 11(10):1226, 2019.
  94. Liang Zheng, Yi Yang, and Qi Tian. Sift meets cnn: A decade survey of instance retrieval. IEEE transactions on pattern analysis and machine intelligence, 40(5):1224-1244, 2017.
  95. Yan-Tao Zheng, Ming Zhao, Yang Song, Hartwig Adam, Ulrich Buddemeier, Alessandro Bissacco, Fernando Brucher, Tat- Seng Chua, and Hartmut Neven. Tour the world: building a web-scale landmark recognition engine. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 1085-1092. IEEE, 2009.
  96. Xingyi Zhou, Dequan Wang, and Philipp Krähenbühl. Objects as points. arXiv preprint arXiv:1904.07850, 2019.

Related papers

Handwritten dynamics assessment through convolutional neural networks: An application to Parkinson's disease identification
clayton pereira

Artificial intelligence in medicine, 2018

Parkinson's disease (PD) is considered a degenerative disorder that affects the motor system, which may cause tremors, micrography, and the freezing of gait. Although PD is related to the lack of dopamine, the triggering process of its development is not fully understood yet. In this work, we introduce convolutional neural networks to learn features from images produced by handwritten dynamics, which capture different information during the individual's assessment. Additionally, we make available a dataset composed of images and signal-based data to foster the research related to computer-aided PD diagnosis. The proposed approach was compared against raw data and texture-based descriptors, showing suitable results, mainly in the context of early stage detection, with results nearly to 95%. The analysis of handwritten dynamics using deep learning techniques showed to be useful for automatic Parkinson's disease identification, as well as it can outperform handcrafted featu...

View PDFchevron_right
Deep Feature Tracker: A Novel Application for Deep Convolutional Neural Networks
saif sayed

arXiv (Cornell University), 2021

Feature tracking is the building block of many applications such as visual odometry, augmented reality, and target tracking. Unfortunately, the state-of-the-art vision based tracking algorithms fail in surgical images due to the challenges imposed by the nature of such environments. In this paper, we proposed a novel and unified deep learning based approach that can learn how to track features reliably as well as learn how to detect such reliable features for the tracking purpose. The proposed network dubbed as Deep-PT, consists of a tracker network which is a convolutional neural network simulating cross correlation in terms of deep learning and two fully connected networks that operate on the output of intermediate layers of the tracker to detect features and predict track-ability of the detected points. The ability to detect features based on the capabilities of the tracker distinguishes the proposed method from previous algorithms used in this area and improves the robustness of the algorithms against dynamics of the scene. The network is trained using multiple datasets due to the lack of specialized dataset for feature tracking datasets and extensive comparisons are conducted to compare the accuracy of Deep-PT against recent pixel tracking algorithms. As the experiments suggest, the proposed deep architecture deliberately learns what to track and how to track and outperforms the state-of-the-art methods.

View PDFchevron_right
FastEval Parkinsonism: an instant deep learning–assisted video-based online system for Parkinsonian motor symptom evaluation
Yu-Yuan Yang

npj digital medicine, 2024

The Motor Disorder Society's Unified Parkinson's Disease Rating Scale (MDS-UPDRS) is designed to assess bradykinesia, the cardinal symptoms of Parkinson's disease (PD). However, it cannot capture the all-day variability of bradykinesia outside the clinical environment. Here, we introduce FastEval Parkinsonism (), a deep learning-driven video-based system, providing users to capture keypoints, estimate the severity, and summarize in a report. Leveraging 840 finger-tapping videos from 186 individuals (103 patients with Parkinson's disease (PD), 24 participants with atypical parkinsonism (APD), 12 elderly with mild parkinsonism signs (MPS), and 47 healthy controls (HCs)), we employ a dilated convolution neural network with two data augmentation techniques. Our model achieves acceptable accuracies (AAC) of 88.0% and 81.5%. The frequency-intensity (FI) value of thumb-index finger distance was indicated as a pivotal hand parameter to quantify the performance. Our model also shows the usability for multi-angle videos, tested in an external database enrolling over 300 PD patients.

View PDFchevron_right
Deep Learning-Aided Parkinson's Disease Diagnosis from Handwritten Dynamics
clayton pereira

2016 29th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), 2016

Parkinson's Disease (PD) automatic identification in early stages is one of the most challenging medicine-related tasks to date, since a patient may have a similar behaviour to that of a healthy individual at the very early stage of the disease. In this work, we cope with PD automatic identification by means of a Convolutional Neural Network (CNN), which aims at learning features from a signal extracted during the individual's exam by means of a smart pen composed of a series of sensors that can extract information from handwritten dynamics. We have shown CNNs are able to learn relevant information, thus outperforming results obtained from raw data. Also, this work aimed at building a public dataset to be used by researchers worldwide in order to foster PD-related research.

View PDFchevron_right
Spatio-Temporal Attention and Magnification for Classification of Parkinson’s Disease from Videos Collected via the Internet
Rafayet Ali

2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), 2020

We present an automated framework for detecting Parkinson’s disease (PD) from videos collected through a scalable online platform. We analyzed 1380 videos of age-matched participants performing four standard motor tasks from the MDS-UPDRS. Our proposed framework leverages multiple deep neural networks to temporally and spatially segment the videos as well as magnify relevant motions. Frequency domain representations of the resulting data are then classified using supervised learning. Overall, the proposed framework achieves an accuracy of 82.5% when discriminating between those with PD and those without, and 61.8% when discriminating between those with PD with treatment, with PD without treatment, and those without PD. These results increased up to 91.8% and 73.5%, respectively, when combining the predictions of multiple models. To understand the contributions of each part of our framework we perform systematic ablation studies. We also compare between motion features based on pixel...

View PDFchevron_right

Related topics

  • Computer Science
  • Artificial Intelligence
  • Computer Vision
  • Scale Invariant Feature Transform
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved