Thumb Inclination-Based Manipulation and Exploration, a Machine Learning Based Interaction Technique for Virtual Environments

  • Muhammad Raees Department of Computer Science and IT, University of Malakand, Chakdara, KPK, Pakistan.
  • Sehat Ullah Department of Computer Science and IT, University of Malakand, Chakdara, KPK, Pakistan.
  • Inam Ur Rehman Department of Computer Science and IT, University of Malakand, Chakdara, KPK, Pakistan.
  • Muhammad Azhar Department of Computer Science and IT, University of Malakand, Chakdara, KPK, Pakistan.

Abstract

In the context of Virtual Reality (VR), interactions refer to the plausible actions in a Virtual Environment (VE). To have an engrossing interface, interactions by the gestures of hand are becoming prominent. With this research work, a novel interaction technique is proposed where interactions are performed on the basis of the position of thumb in dynamic image stream. The technique needs no expensive tracker but an ordinary camera to trace hand movements and position of thumb. The interaction tasks are enacted in distinct interaction states, where the Angle of Inclination (AOI) of thumb is used for state-to-state transition. The angle is computed dynamically between the tip-of-thumb and the base of the Region of Interest (ROI) of an input image. The technique works in two phases: learning phase and application phase. In the learning phase, user-defined fistpostures with distinct AOI are learnt. The Support Vector Machine (SVM) classifier is trained by the AOI of the postures. In the application phase, interactions are performed in distinct interaction states whereas a particular state is activated by posing the known posture. To follow the trajectory of thumb, dynamic mapping is performed to control a virtual hand by the position of thumb in the input image. The technique is implemented in a Visual Studio project called Thumb-Based Interaction for Virtual Environments (TIVE). The project was evaluated by a group of 15 users in a moderate lighting condition. The 89.7% average accuracy rate of the evaluation proves suitability of the technique in the wide range VR applications.

Published
Apr 1, 2021
How to Cite
RAEES, Muhammad et al. Thumb Inclination-Based Manipulation and Exploration, a Machine Learning Based Interaction Technique for Virtual Environments. Mehran University Research Journal of Engineering and Technology, [S.l.], v. 40, n. 2, p. 358 - 370, apr. 2021. ISSN 2413-7219. Available at: <https://publications.muet.edu.pk/index.php/muetrj/article/view/2091>. Date accessed: 15 apr. 2021. doi: http://dx.doi.org/10.22581/muet1982.2102.10.
Section
Articles
This is an open Access Article published by Mehran University of Engineering and Technolgy, Jamshoro under CCBY 4.0 International License