Real-time hand gesture recognition using fine-tuned convolutional neural network
Authors:
- Jaya Prakash Sahoo,
- Allam Jaya Prakash,
- Paweł Pławiak,
- Saunak Samantray
Abstract
Hand gesture recognition is one of the most effective modes of interaction between humans and computers due to being highly flexible and user-friendly. A real-time hand gesture recognition system should aim to develop a user-independent interface with high recognition performance. Nowadays, convolutional neural networks (CNNs) show high recognition rates in image classification problems. Due to the unavailability of large labeled image samples in static hand gesture images, it is a challenging task to train deep CNN networks such as AlexNet, VGG-16 and ResNet from scratch. Therefore, inspired by CNN performance, an end-to-end fine-tuning method of a pre-trained CNN model with score-level fusion technique is proposed here to recognize hand gestures in a dataset with a low number of gesture images. The effectiveness of the proposed technique is evaluated using leave-one-subject-out cross-validation (LOO CV) and regular CV tests on two benchmark datasets. A real-time American sign language (ASL) recognition system is developed and tested using the proposed technique.
- Record ID
- CUT74f72307ba8141aba7c938098fefb926
- Publication categories
- ;
- Author
- Journal series
- Sensors, ISSN , e-ISSN 1424-8220, Biweekly
- Issue year
- 2022
- Vol
- 22
- No
- 3
- Pages
- [1-14]
- Article number
- 706
- Other elements of collation
- fot.; rys.; tab.; wykr.; Bibliografia (na s.) - 12-14; Bibliografia (liczba pozycji) - 37; Oznaczenie streszczenia - Abstr.; Numeracja w czasopiśmie - Vol. 22, Iss. 3
- Substantive notes
- Section: Intelligent Sensors
- Keywords in English
- ASL, fine-tunning, hand gesture recognition, pre-trained CNN, real-time gesture recognition, score fusion
- ASJC Classification
- ; ; ;
- DOI
- DOI:10.3390/s22030706 Opening in a new tab
- URL
- https://www.mdpi.com/1424-8220/22/3/706 Opening in a new tab
- Language
- eng (en) English
- License
- Score (nominal)
- 100
- Score source
- journalList
- Score
- Publication indicators
- Citation count
- 80
- Additional fields
- Indeksowana w: Web of Science, Scopus
- Uniform Resource Identifier
- https://cris.pk.edu.pl/info/article/CUT74f72307ba8141aba7c938098fefb926/
- URN
urn:pkr-prod:CUT74f72307ba8141aba7c938098fefb926
* presented citation count is obtained through Internet information analysis, and it is close to the number calculated by the Publish or PerishOpening in a new tab system.