TAKTO, Open Source Hand-Tracking Built Like LEGO

What happened

Hand tracking in VR/AR can add a substantial amount of immersion and added interactivity. Unfortunately, most camera based tracking suffers when fingers get covered, intertwined, or simply out of view of the headset’s camera.

What’s even less common are methods of giving feedback to the user’s hands. [Sebastian Molano] saw these issues and decided if anyone was going to make an open-source solution, it was going to be him with his thesis on TAKTO .

TAKTO aims to remove the prior issues mentioned with visual tracking by using magnetic encoders to tell where the device is and the finger’s position inside it. Actuators are able to pull on the fingers themselves to add even further information to the user.