Gesture Control with an IMU Glove

The focus of this project was to build a wearable IMU glove that can be used to capture the movements of the user's hand and then translate that into movements for the robot. We also wanted to include touch sensors in the end of finger tips to give the wearer additional ability. There was three main parts to this project. The first being the glove with the IMU sensor as well as a Raspberry Pi 4 which gather the movement data of the hand and run it through the control algorithm. The second part of the project is the connection between the glove and the TurtleBot, this was completed by using a web socket. The data output of the control algorithm would be packaged using JSON and then its sent over WIFI to the TurtleBot. The final part of this project is the TurtleBot which is interfaced with using ROS which sends velocity commands to the motors allowing the movement of the users hand to be translated to movement of the TurtleBot.

Starting with the make up of the glove, We decided to go with an armored mechanics glove that allowed for durability and stability to mount the IMU to the glove while also remaining flexible enough that the user still had full dexterity. The raspberry pi was mounted on an arm band and powered off of a power bank in the users pocket giving the user the option once connected to WIFI to have almost no cords constraining them to a single spot. The IMU was mounted on a flat plane located near the center of of the users hand which allowed for the most accurate measurement of the pitch and yaw of the hand. The IMU was interfaced with the raspberry pi controller via I2C and would send back the yaw and pitch angles of the user's hand. The glove was also fitted with home made touch sensors on three of the finger tips. The pointer finger sensor would stop the program incase of an emergency, the middle finger would increase the speed of the robot and the the ring finger would decrease the speed of the robot. The thumb had a connection to ground allowing for the the pi to know when the user touched any of the other finger tips. These sensors were fabricated out of aluminum foil and jumper wires.

On start up of the code the user has to move their hand in the three different orientations (Roll, Pitch, Yaw) and this will calibrate the IMU and give a zero point to base all the movements off of. There is a built in LED that will turn on indicating the board is calibrated successfully and this will begin the transmission of data from the Raspberry pi to the TurtleBot. The control algorithm for the glove was built around the pitch and yaw of the user's hand. If the pitch was positive then the TurtleBot would move forward and if the the pitch was negative then the robot would go in reverse. Simultaneously if the user yaws left or to the right then the robot will begin to turn in that direction. This allows for 360° of movement and if the user wants the robot to increase its movement speed then they can use the finger tip sensors to increase or decrease the speed in small increments.

Wireless connection between the user and the TurtleBot is import so in this project the connection is formed using a web socket. The TurtleBot is the server side of the connection and the raspberry pi is the client side. The control data is packaged in JSON and then encoded using UTF-8 before being sent over WIFI where the TurtleBot receives and decodes the data. This data is then sent to the motor controllers of the TurtleBot using a ROS message allowing the robot to move with the user's hand motions.