Back to everything

2023 ios computer vision swift

HGCam.

It watches your hand and reads which joints are touching. One gesture takes a photo, another starts recording, another stops.

The HGCam app icon

What it is

An iOS camera driven by hand pose. Apple's Vision framework finds the joints in your hand, and the app reads combinations of those joints touching as commands. There is a guide screen that shows you the gestures and a gallery for whatever you shot.

Pause and resume are still on the list.

Why I made it

Every group photo I take is a group photo I am not in. A timer solves that badly, and your hands are already in the frame.

How I made it

Swift and UIKit, AVFoundation for the capture, Vision for the hand pose. The file worth reading is the gesture processor, which has to decide that two joints are touching while the coordinates coming back are noisy and never quite still.

What I learned

Recognising a gesture is the easy half. Deciding when one has ended, without firing the shutter twice, is the half that takes the work.