Hand Gesture Recognition

Five hand gestures recognised live from a webcam and mapped to robot commands. MediaPipe gives 21 hand landmarks per frame, I recorded my own dataset, and a Random Forest and an SVM both land around 94% cross-validated accuracy. The final model runs at webcam speed.

The idea

Point a webcam at your hand and control a robot with it: open palm to stop, fist to grab, thumbs up to move forward, point to turn, peace to speed up. The interesting part is not the camera, it is everything between the camera and the command.

How it works

• Landmarks, not pixels. MediaPipe returns 21 points per frame. Training on those 63 coordinates instead of raw images makes the problem small, fast and robust to lighting.
• Normalised. A gesture means the same thing wherever the hand is in frame, so every sample is translated and scaled before the model sees it.
• My own dataset. A small recording tool captures labelled samples straight from the webcam, a few thousand frames across the five gestures.
• Two models, compared honestly. Random Forest and SVM, evaluated with 5-fold cross-validation rather than one lucky split. Both land near 94%, and both make the same mistake: fist versus thumbs up, which differ only by the thumb.

The live demo

The model runs against the live feed with a short rolling vote over recent frames, so the command does not flicker while a gesture is half formed. A video of it running is coming.