LogInA paper from Facebook showcases a new approach to tracking of hands with severe self-occlusion and self-contact.
The demo above shows hand movement captured at 30fps using 124 cameras, a 6-core Xeon CPU, and Tesla V100 GPU.
Many actions with hands involve self-contact and occlusion. We're shaking hands, making a fist, interlacing our fingers while thinking, and the thing is that these actions are quite challenging to track as existing methods for tracking hands and faces are not meant for extreme amounts of self-contact and self-occlusion.
The Facebook Research team decided to extend recent advances in vision-based tracking and physically-based animation, building the first algorithm for tracking high-fidelity hand deformations through highly self-contacting and self-occluding hand gestures. The new system was built for both single hands and two hands.
"By constraining a vision-based tracking algorithm with a physically based deformable model, we obtain an algorithm that is robust to the ubiquitous self-interactions and massive self-occlusions exhibited by common hand gestures, allowing us to track two hand interactions and some of the most difficult possible configurations of a human hand," wrote the team.
You can learn more and find the full paper here. Don't forget to join our new Telegram channel, our Discord, follow us on Instagram and Twitter, where we are sharing breakdowns, latest news, awesome artworks, and more.