Multi-subject tracking in 3 dimensions

3DTracker is an open-source, versatile, and low-cost video-based behavioral analysis system developed for neuroscience. The system works as follows:

1. A 3D video is reconstructed from the images of multiple (~4) depth cameras.
2. Positions of body parts are estimated by fitting skeletal models to the 3D videos.
3. Various behaviors are automatically detected based on the trajectories of body parts.

Because 3DTracker works without any markers attached to the body, it is able to better capture animals' naturalistic behavior expressions. It also enables robust tracking of overlapped animals (e.g., during social interactions), compared to conventional 2D video based systems.

The system has been applied for analyzing social interactions, object recognition, and neural activity during social behavior. The system can be used for both rats and mice. It uses a fast pose estimation algorithm potentially capable of real-time behavioral event detection.

We hope that this open-source project helps people to explore various possible applications of 3D video analysis!