Hand tracking is software that finds a human hand in a camera image and follows its position, pose and finger positions in real time — using nothing but the camera. No gloves, no sensors, no controllers. It is how a computer can tell that you are pointing, pinching, or waving, and respond.
Runs on this device. Nothing is uploaded or recorded.
Now draw with it →Where you have already met it
- VR headsets — Quest and Vision Pro let you press buttons and grab objects with bare hands; cameras on the headset track them.
- Video-call effects — reactions that fire when you give a thumbs-up are hand tracking watching your webcam.
- Short-video filters — TikTok and Instagram effects that attach sparkles to your fingertips.
- Smart TVs and kiosks — wave-to-scroll interfaces in showrooms and some cars.
- Air drawing — webcam tools where a pinch of your fingers draws a line in mid-air.
What the software actually perceives
Modern hand tracking does not just find a hand-shaped blob. It estimates the position of the hand's individual joints — knuckles, fingertips, the wrist — many times per second, building a live skeleton of your hand. From that skeleton, gestures become readable: if the thumb tip and index tip are close together, that is a pinch; if all fingers are extended and spread, that is an open palm.
Google's open-source MediaPipe hand model, the one the demo above runs, shows what that skeleton looks like in practice. For every hand it finds, it reports 21 points, which its documentation calls “hand-knuckle coordinates”: the wrist, then four points along the thumb and along each finger, from the base out to the tip. Each point comes with a position in the image and a rough depth, measured from the wrist.
- 0Wrist
- 1–4Thumb: base, knuckle, joint, tip
- 5–8Index finger: knuckle, two joints, tip
- 9–12Middle finger
- 13–16Ring finger
- 17–20Little finger
The important idea is that gestures become a language. One pose can mean 'draw', another 'stop', another 'open a menu'. A well-designed gesture system feels less like operating a computer and more like the computer finally understanding something you already knew how to do.
Why it suddenly works in a browser
Ten years ago this needed depth cameras and desktop-class software. Two things changed: the models that locate hands became small and fast enough to run in real time on ordinary laptops and phones, and browsers became capable of running them directly on your device. The practical consequence: hand tracking now works on hardware you already own, inside a web page, with the camera feed processed locally rather than uploaded — which is also the right answer to the privacy question.
What it is genuinely good and bad at
| Strength | Limitation |
|---|---|
| Natural, zero-hardware input — everyone owns a hand | Needs the hand visible and reasonably lit |
| Expressive: position, pose and motion all carry meaning | A single camera senses depth only roughly — fine 3D precision needs a headset |
| Touchless — hygienic for shared screens, kiosks, classrooms | Sustained precision work (retouching photos) still favours a pen |
| Runs in-browser on ordinary devices | Occlusion: a hand hiding its own fingers can confuse any tracker |
Feel it, don't read about it
The gap between reading about hand tracking and feeling a line appear under your fingertip is enormous. The demo at the top of this page shows the tracking itself; AirVA's air drawing canvas shows what it is for: a free, no-install canvas where a pinch of your fingers draws in mid-air, ten seconds after you allow the camera.
The fastest way to understand hand tracking is to feel it work.
Try AirVA free →No install · no account · works in seconds