Unlocking the potential of motion capture data has never been more accessible. Kinect v2.0, with its advanced depth-sensing technology, offers a cost-effective solution for capturing intricate movements. But what if you need that data in a more versatile format like Biovision Hierarchy (BVH)? This post explores the process of converting Kinect v2.0 motion data into BVH files, opening doors to animation, game development, and various other applications. Learn how this conversion empowers you to leverage Kinect data in professional pipelines and expand your creative possibilities.
Understanding Kinect v2.0 and BVH
Kinect v2.0 captures motion data as a stream of skeletal joint positions and orientations. This data, while rich, isn’t directly compatible with many 3D animation or analysis tools. BVH, on the other hand, is a widely supported standard for representing hierarchical skeletal animation data. It’s a text-based format describing the relationships between joints and their movements over time, making it ideal for importing into various software.
Converting Kinect v2.0 data to BVH bridges the gap, allowing you to leverage affordable motion capture hardware with industry-standard software. This process involves mapping the Kinect skeletal structure to a BVH skeleton and transforming the raw Kinect data into the hierarchical structure and format required by BVH.
Methods for Converting Kinect v2.0 Motion to BVH
Several methods exist for converting Kinect v2.0 data to BVH. Software solutions like iPi Motion Capture and Brekel Kinect Pro Body offer user-friendly interfaces and automated processes for this conversion. These tools often include features like noise reduction, retargeting, and other enhancements to improve the quality of the BVH output.
Alternatively, for those comfortable with programming, various libraries and SDKs, such as the Microsoft Kinect SDK and open-source projects, provide the necessary tools to develop custom conversion solutions. This allows for greater control over the process and customization for specific needs. For example, you can adjust the skeletal mapping or implement custom filtering algorithms.
Choosing the right method depends on your technical skills, budget, and desired level of control over the conversion process.
Benefits of Using BVH Files in Animation and Game Development
BVH files play a crucial role in animation and game development pipelines. Their standardized format enables seamless integration with a wide range of software, including Autodesk Maya, Blender, and Unity. This interoperability simplifies the workflow and reduces the need for complex data conversions.
BVH files facilitate motion retargeting, allowing animations captured from one character to be applied to another, saving time and effort. This is particularly useful for game developers who can quickly create realistic character animations based on motion capture data.
- Wide software compatibility
- Efficient motion retargeting
Practical Applications and Case Studies
The conversion of Kinect v2.0 data to BVH has found applications in various fields. Researchers use it to analyze human movement in biomechanics and sports science. Animators leverage it for creating realistic character animations in films and games. Virtual reality developers utilize it to translate real-world movements into virtual environments.
For instance, in a study published in the Journal of Biomechanics (hypothetical example), researchers used Kinect v2.0 and BVH conversion to analyze the gait of athletes, identifying key performance indicators and potential areas for improvement. In the gaming industry, independent developers have used this technique to create realistic character animations for their projects, achieving professional-quality results with a limited budget.
[Infographic Placeholder]
Troubleshooting and Optimization
While the process is generally straightforward, challenges can arise. Occlusion, where the Kinect’s view of a joint is blocked, can lead to missing or inaccurate data. Calibration issues can also affect the accuracy of the captured motion. Addressing these challenges often involves optimizing the capture environment, ensuring proper lighting, and minimizing obstructions. Software solutions frequently offer tools for data cleaning and smoothing to mitigate these issues.
To ensure optimal results, consider the following:
- Proper Kinect Calibration
- Controlled Capture Environment
- Post-Processing and Data Cleaning
By carefully managing the capture process and utilizing available tools, you can achieve high-quality BVH output from your Kinect v2.0 data.
FAQ
Q: What software can I use to open BVH files?
A: Many 3D animation and modeling software packages, including Blender, Autodesk Maya, and MotionBuilder, support BVH files.
Converting Kinect v2.0 motion data to BVH unlocks a wealth of possibilities for animators, game developers, and researchers. By understanding the process and utilizing the right tools, you can leverage affordable motion capture technology in professional pipelines. Whether youโre creating realistic character animations, analyzing human movement, or developing interactive virtual reality experiences, this conversion empowers you to bring your ideas to life. Explore the tools and techniques discussed, and start capturing and converting your motion data today. Learn more about Kinect development on Microsoft’s Kinect Developer site. For deeper insights into animation principles, check out Animation Mentor. Also, explore the potential of open-source solutions like Azure Kinect Sensor SDK. Dive in and discover the potential of motion capture with Kinect and BVH. Visit our blog post on Kinect motion capture tips for additional guidance.
Question & Answer :
I would like to store the motion capture data from Kinect 2 as a BVH file. I found code which does so for Kinect 1 which can be found here. I went through the code and found several things that I was not able to understand. For example, in the mentioned code I’ve tried to understand what exactly the Skeleton skel object, found in several places in the code, actually is. If not, are there any known application available to accomplish the intended?
EDIT: I tried to change Skeleton skel to Body skel which I think is the correspondant object for kinect SDK 2.0. However I’ve got an error when I try to get the position of the body:
tempMotionVektor[0] = -Math.Round( skel.Position.X * 100,2); tempMotionVektor[1] = Math.Round( skel.Position.Y * 100,2) + 120; tempMotionVektor[2] = 300 - Math.Round( skel.Position.Z * 100,2);
I’ve gotten errors when calling the function Position for the Body skel. How can I retrieve the X, Y, Z of the skeleton in sdk 2.0?? I tried to change the above three lines to:
tempMotionVektor[0] = -Math.Round(skel.Joints[0].Position.X * 100, 2); tempMotionVektor[1] = Math.Round(skel.Joints[0].Position.Y * 100, 2) + 120; tempMotionVektor[2] = 300 - Math.Round(skel.Joints[0].Position.Z * 100, 2);
EDIT: Basically I managed to store the a bvh file after combining bodyBasicsWPF and kinect2bvh. However, it seems that the skeleton I am storing is not efficient. There are strange movements in the elbows. I am trying to understand if I have to change something in the file kinectSkeletonBVH.cp. More specifically, what are the changes in the joint axis orientation for the kinect 2 version. How can I change the following line: skel.BoneOrientations[JointType.ShoulderCenter].AbsoluteRotation.Quaternion; I tried to change that line with skel.JointOrientations[JointType.ShoulderCenter].Orientation. Am I right? I am using the following code to add the joint to BVHBone objects:
BVHBone hipCenter = new BVHBone(null, JointType.SpineBase.ToString(), 6, TransAxis.None, true); BVHBone hipCenter2 = new BVHBone(hipCenter, "HipCenter2", 3, TransAxis.Y, false); BVHBone spine = new BVHBone(hipCenter2, JointType.SpineMid.ToString(), 3, TransAxis.Y, true); BVHBone shoulderCenter = new BVHBone(spine, JointType.SpineShoulder.ToString(), 3, TransAxis.Y, true); BVHBone collarLeft = new BVHBone(shoulderCenter, "CollarLeft", 3, TransAxis.X, false); BVHBone shoulderLeft = new BVHBone(collarLeft, JointType.ShoulderLeft.ToString(), 3, TransAxis.X, true); BVHBone elbowLeft = new BVHBone(shoulderLeft, JointType.ElbowLeft.ToString(), 3, TransAxis.X, true); BVHBone wristLeft = new BVHBone(elbowLeft, JointType.WristLeft.ToString(), 3, TransAxis.X, true); BVHBone handLeft = new BVHBone(wristLeft, JointType.HandLeft.ToString(), 0, TransAxis.X, true); BVHBone neck = new BVHBone(shoulderCenter, "Neck", 3, TransAxis.Y, false); BVHBone head = new BVHBone(neck, JointType.Head.ToString(), 3, TransAxis.Y, true); BVHBone headtop = new BVHBone(head, "Headtop", 0, TransAxis.None, false);
I can’t understand where inside the code the axis for every Joint is calculated.
The code you used for Kinect 1.0 to obtain a BVH file use the joints information to build bone vectors by reading the Skeleton.
public static double[] getBoneVectorOutofJointPosition(BVHBone bvhBone, Skeleton skel) { double[] boneVector = new double[3] { 0, 0, 0 }; double[] boneVectorParent = new double[3] { 0, 0, 0 }; string boneName = bvhBone.Name; JointType Joint; if (bvhBone.Root == true) { boneVector = new double[3] { 0, 0, 0 }; } else { if (bvhBone.IsKinectJoint == true) { Joint = KinectSkeletonBVH.String2JointType(boneName); boneVector[0] = skel.Joints[Joint].Position.X; boneVector[1] = skel.Joints[Joint].Position.Y; boneVector[2] = skel.Joints[Joint].Position.Z; ..
Source: Nguyรชn Lรช ฤแบทng - Kinect2BVH.V2
Except in Kinect 2.0, Skeleton class has been replaced by the Body class, so you need to change it to deal with a Body instead, and obtain the joints by following the steps quoted below.
// Kinect namespace using Microsoft.Kinect; // ... // Kinect sensor and Kinect stream reader objects KinectSensor _sensor; MultiSourceFrameReader _reader; IList<Body> _bodies; // Kinect sensor initialization _sensor = KinectSensor.GetDefault(); if (_sensor != null) { _sensor.Open(); }We also added a list of bodies, where all of the body/skeleton related data will be saved. If you have developed for Kinect version 1, you notice that the Skeleton class has been replaced by the Body class. Remember the MultiSourceFrameReader? This class gives us access on every stream, including the body stream! We simply need to let the sensor know that we need body tracking functionality by adding an additional parameter when initializing the reader:
_reader = _sensor.OpenMultiSourceFrameReader(FrameSourceTypes.Color | FrameSourceTypes.Depth | FrameSourceTypes.Infrared | FrameSourceTypes.Body); _reader.MultiSourceFrameArrived += Reader_MultiSourceFrameArrived;The Reader_MultiSourceFrameArrived method will be called whenever a new frame is available. Letโs specify what will happen in terms of the body data:
- Get a reference to the body frame
- Check whether the body frame is null โ this is crucial
- Initialize the _bodies list
- Call the GetAndRefreshBodyData method, so as to copy the body data into the list
- Loop through the list of bodies and do awesome stuff!
Always remember to check for null values. Kinect provides you with approximately 30 frames per second โ anything could be null or missing! Here is the code so far:
void Reader_MultiSourceFrameArrived(object sender, MultiSourceFrameArrivedEventArgs e) { var reference = e.FrameReference.AcquireFrame(); // Color // ... // Depth // ... // Infrared // ... // Body using (var frame = reference.BodyFrameReference.AcquireFrame()) { if (frame != null) { _bodies = new Body[frame.BodyFrameSource.BodyCount]; frame.GetAndRefreshBodyData(_bodies); foreach (var body in _bodies) { if (body != null) { // Do something with the body... } } } } }This is it! We now have access to the bodies Kinect identifies. Next step is to display the skeleton information on-screen. Each body consists of 25 joints. The sensor provides us with the position (X, Y, Z) and the rotation information for each one of them. Moreover, Kinect lets us know whether the joints are tracked, hypothsized or not tracked. Itโs a good practice to check whether a body is tracked before performing any critical functions.
The following code illustrates how we can access the different body joints:
if (body != null) { if (body.IsTracked) { Joint head = body.Joints[JointType.Head]; float x = head.Position.X; float y = head.Position.Y; float z = head.Position.Z; // Draw the joints... } }
Source: Vangos Pterneas Blog - KINECT FOR WINDOWS VERSION 2: BODY TRACKING
