logo标签列表

Kinect 侧面骨架追踪

c#kinect
42

目前我正在使用Microsoft Kinect来测量关节之间的角度。大多数测量结果都是正确的，但当一个人侧身坐在椅子上时，Kinect无法准确追踪骨架。为了说明问题，我已经添加了三张Kinect深度视图图片。

侧身坐姿下骨架追踪结果

侧身坐姿下无骨架追踪结果

侧身坐姿下有骨架追踪结果

如您所见，其中两个测量结果是“正确的”。每当我抬起一条腿，Kinect就无法正确地追踪骨架。有没有人能解决这个问题，或者这只是Kinect的限制？

谢谢。

更新1： 在第2个屏幕截图中，这些被跟踪的关节的JointTrackingState-Enumeration被标记为Inferred，但深度视图正在跟踪我的全身。

更新2： 在第2个屏幕截图中，我正在尝试跟踪我的前腿，用绿色标出。我知道另一条腿没有被跟踪，但这并不重要，我想。

更新3： 下面的代码选择一个骨架：

private Skeleton StickySkeleton(Skeleton[] skeletons)
{
    if (skeletons.Count<Skeleton>(skeleton => skeleton.TrackingId == _trackedSkeletonId) <= 0)
    {
        _trackedSkeletonId = -1;
        _skeleton = null;
    }

    if (_trackedSkeletonId == -1)
    {
        Skeleton foundSkeleton = skeletons.FirstOrDefault<Skeleton>(skeleton => skeleton.TrackingState == SkeletonTrackingState.Tracked);

        if (foundSkeleton != null)
        {
            _trackedSkeletonId = foundSkeleton.TrackingId;
            return foundSkeleton;
        }
    }

    return _skeleton;
}

每当追踪骨架时，数据将用于绘制关节点并计算关节之间的角度。

更新4：我已经测试了坐在一个比椅子简单得多的'块'上。不幸的是，Kinect仍然表现相同。

以下是2个屏幕截图：

Sitting on a block 1

Sitting on a block 2

3
在我看来，Kinect的问题在于它无法评估地面。在屏幕1和3上，你正接触地面，Kinect利用这一点来测量一切。但在屏幕2上，你“漂浮在空中”，Kinect不知道该怎么做。这是我的看法。 - Fixus
2
那么像Kinect Sports这样的游戏呢？它们跳跃，因此“在空中”，但Kinect仍然知道该怎么做。有什么建议吗？ - Velth
2
我已经尝试了Microsoft Kinect SDK中包含的KinectExplorer示例。以下屏幕截图显示了使用KinectExplorer示例时Kinect的行为：屏幕截图1和屏幕截图2。正如您所看到的，结果相同。 请参阅我的主要帖子以获取代码，谢谢！ - Velth
1
这是Kinect SDK（骨骼跟踪）的一个限制，但您可以自由地从深度图像中放置自己的骨骼跟踪算法。我不会将其作为答案放置，但希望这能激发您对该问题的思考。这是我会做的方式： - davidbates
注意：您必须确保使用Kinect for Windows以启用坐姿模式并实际使用坐姿模式。Xbox的坐姿模式没有任何作用。 - FlavorScape
显示剩余9条评论
3个回答
13

如Renaud Dumont所说，我需要对JointTrackingState进行处理。由于您使用的是膝盖，我使用变量leftknee和rightknee来进行操作，它们都是Joints。这是代码，您可以使用JointType.FootRight和JointType.FootLeft以及Hip类型进行操作，但我将其留给您自己决定。

static Skeleton first = new Skeleton();

Joint leftknee = first.Joints[JointType.KneeLeft];
Joint rightknee = first.Joints[JointType.KneeRight];

if ((leftknee.TrackingState == JointTrackingState.Inferred ||
                leftknee.TrackingState == JointTrackingState.Tracked) && 
                (rightknee.TrackingState == JointTrackingState.Tracked ||
                rightknee.TrackingState == JointTrackingState.Inferred))
            {

            }

或者，如果您只想追踪一只膝盖，或者两只膝盖，您可以这样做：

if ((leftknee.TrackingState == JointTrackingState.Inferred ||
                leftknee.TrackingState == JointTrackingState.Tracked) && 
                (rightknee.TrackingState == JointTrackingState.Tracked ||
                rightknee.TrackingState == JointTrackingState.Inferred))
            {

            }

            else if (leftknee.TrackingState == JointTrackingState.Inferred ||
                    leftknee.TrackingState == JointTrackingState.Tracked)
            {

            }

            else if (rightknee.TrackingState == JointTrackingState.Inferred ||
                    rightknee.TrackingState == JointTrackingState.Tracked)
            {

            }

提醒一下，Skeleton被设置为static是因为它可以用于制作关节。

 static Skeleton first;

与......相反

 Skeleton first;

编辑1


我得出结论这是非常难做的，我认为上述方法会起作用，但我只是想把我正在努力工作的内容包含在内，以防您能找到一些使其起作用的方法。无论如何，这是我正在努力的代码，它是另一个class，也是另一个SkeletalTrackingState，我试图在其中制作一个Inferred enum。但不幸的是，enum是不可能继承的。如果你发现有类似作用的东西可以运行，我将永远尊重您作为比我优秀的程序员；）话不多说，这就是我试图制作的.dll：

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.Kinect;

namespace IsInferred
{
public abstract class SkeletonInferred : Skeleton
{
    public bool inferred;
    static Skeleton first1 = new Skeleton();
    Joint handright;
    Joint handleft;
    Joint footright;
    Joint footleft;
    Joint ankleleft;
    Joint ankleright;
    Joint elbowleft;
    Joint elbowright;
    Joint head;
    Joint hipcenter;
    Joint hipleft;
    Joint hipright;
    Joint shouldercenter;
    Joint shoulderleft;
    Joint shoulderright;
    Joint kneeleft;
    Joint kneeright;
    Joint spine;
    Joint wristleft;
    Joint wristright;

    public SkeletonInferred(bool inferred)
    {

    }

    public enum Inferred
    {
        NotTracked = SkeletonTrackingState.NotTracked,
        PositionOnly = SkeletonTrackingState.PositionOnly,
        Tracked = SkeletonTrackingState.Tracked,
        Inferred = 3,
    }

    private void IsInferred(object sender, AllFramesReadyEventArgs e)
    {
        handright = first1.Joints[JointType.HandRight];
        handleft = first1.Joints[JointType.HandLeft];
        footright = first1.Joints[JointType.FootRight];
        footleft = first1.Joints[JointType.FootLeft];
        ankleleft = first1.Joints[JointType.AnkleLeft];
        ankleright = first1.Joints[JointType.AnkleRight];
        elbowleft = first1.Joints[JointType.ElbowLeft];
        elbowright = first1.Joints[JointType.ElbowRight];
        head = first1.Joints[JointType.Head];
        hipcenter = first1.Joints[JointType.HipCenter];
        hipleft = first1.Joints[JointType.HipLeft];
        hipright = first1.Joints[JointType.HipRight];
        shouldercenter = first1.Joints[JointType.ShoulderCenter];
        shoulderleft = first1.Joints[JointType.ShoulderLeft];
        shoulderright = first1.Joints[JointType.ShoulderRight];
        kneeleft = first1.Joints[JointType.KneeLeft];
        kneeright = first1.Joints[JointType.KneeRight];
        spine = first1.Joints[JointType.Spine];
        wristleft = first1.Joints[JointType.WristLeft];
        wristright = first1.Joints[JointType.WristRight];

        if (handleft.TrackingState == JointTrackingState.Inferred &
            handright.TrackingState == JointTrackingState.Inferred &
            head.TrackingState == JointTrackingState.Inferred &
            footleft.TrackingState == JointTrackingState.Inferred &
            footright.TrackingState == JointTrackingState.Inferred &
            ankleleft.TrackingState == JointTrackingState.Inferred &
            ankleright.TrackingState == JointTrackingState.Inferred &
            elbowleft.TrackingState == JointTrackingState.Inferred &
            elbowright.TrackingState == JointTrackingState.Inferred &
            hipcenter.TrackingState == JointTrackingState.Inferred &
            hipleft.TrackingState == JointTrackingState.Inferred &
            hipright.TrackingState == JointTrackingState.Inferred &
            shouldercenter.TrackingState == JointTrackingState.Inferred &
            shoulderleft.TrackingState == JointTrackingState.Inferred &
            shoulderright.TrackingState == JointTrackingState.Inferred &
            kneeleft.TrackingState == JointTrackingState.Inferred &
            kneeright.TrackingState == JointTrackingState.Inferred &
            spine.TrackingState == JointTrackingState.Inferred &
            wristleft.TrackingState == JointTrackingState.Inferred &
            wristright.TrackingState == JointTrackingState.Inferred)
        {
            inferred = true;
        }
      }
    }
  }

您的项目中的代码（编译器错误）

    using IsInferred;

    static bool Inferred = false;
    SkeletonInferred inferred = new SkeletonInferred(Inferred);
    static Skeleton first1 = new Skeleton();

    Skeleton foundSkeleton = skeletons.FirstOrDefault<Skeleton>(skeleton =>  skeleton.TrackingState == SkeletonTrackingState.Inferred);

祝您好运，希望这能帮助您朝着正确的方向前进或对您有所帮助！


以下是您要求的代码。 是的，它来自Skeletal Tracking Fundamentals，但这段代码已经存在了，我不想再新启动一个项目并大部分使用相同的内容。祝愉快！

代码

     // (c) Copyright Microsoft Corporation.
     // This source is subject to the Microsoft Public License (Ms-PL).
    // Please see http://go.microsoft.com/fwlink/?LinkID=131993 for details.
    // All other rights reserved.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Data;
using System.Windows.Documents;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Navigation;
using System.Windows.Shapes;
using Microsoft.Kinect;
using Coding4Fun.Kinect.Wpf; 

namespace SkeletalTracking
{
/// <summary>
/// Interaction logic for MainWindow.xaml
/// </summary>
public partial class MainWindow : Window
{
    public MainWindow()
    {
        InitializeComponent();
    }

    bool closing = false;
    const int skeletonCount = 6; 
    Skeleton[] allSkeletons = new Skeleton[skeletonCount];

    private void Window_Loaded(object sender, RoutedEventArgs e)
    {
        kinectSensorChooser1.KinectSensorChanged += new  DependencyPropertyChangedEventHandler(kinectSensorChooser1_KinectSensorChanged);

    }

    void kinectSensorChooser1_KinectSensorChanged(object sender, DependencyPropertyChangedEventArgs e)
    {
        KinectSensor old = (KinectSensor)e.OldValue;

        StopKinect(old);

        KinectSensor sensor = (KinectSensor)e.NewValue;

        if (sensor == null)
        {
            return;
        }




        var parameters = new TransformSmoothParameters
        {
            Smoothing = 0.3f,
            Correction = 0.0f,
            Prediction = 0.0f,
            JitterRadius = 1.0f,
            MaxDeviationRadius = 0.5f
        };
        sensor.SkeletonStream.Enable(parameters);

        //sensor.SkeletonStream.Enable();

        sensor.AllFramesReady += new EventHandler<AllFramesReadyEventArgs>(sensor_AllFramesReady);
        sensor.DepthStream.Enable(DepthImageFormat.Resolution640x480Fps30); 
        sensor.ColorStream.Enable(ColorImageFormat.RgbResolution640x480Fps30);

        try
        {
            sensor.Start();
        }
        catch (System.IO.IOException)
        {
            kinectSensorChooser1.AppConflictOccurred();
        }
    }

    void sensor_AllFramesReady(object sender, AllFramesReadyEventArgs e)
    {
        if (closing)
        {
            return;
        }

        //Get a skeleton
        Skeleton first =  GetFirstSkeleton(e);

        if (first == null)
        {
            return; 
        }



        //set scaled position
        //ScalePosition(headImage, first.Joints[JointType.Head]);
        ScalePosition(leftEllipse, first.Joints[JointType.HandLeft]);
        ScalePosition(rightEllipse, first.Joints[JointType.HandRight]);
        ScalePosition(leftknee, first.Joints[JointType.KneeLeft]);
        ScalePosition(rightknee, first.Joints[JointType.KneeRight]);

        GetCameraPoint(first, e); 

    }

    void GetCameraPoint(Skeleton first, AllFramesReadyEventArgs e)
    {

        using (DepthImageFrame depth = e.OpenDepthImageFrame())
        {
            if (depth == null ||
                kinectSensorChooser1.Kinect == null)
            {
                return;
            }


            //Map a joint location to a point on the depth map
            //head
            DepthImagePoint headDepthPoint =
                depth.MapFromSkeletonPoint(first.Joints[JointType.Head].Position);
            //left hand
            DepthImagePoint leftDepthPoint =
                depth.MapFromSkeletonPoint(first.Joints[JointType.HandLeft].Position);
            //right hand
            DepthImagePoint rightDepthPoint =
                depth.MapFromSkeletonPoint(first.Joints[JointType.HandRight].Position);

            DepthImagePoint rightKnee =
                depth.MapFromSkeletonPoint(first.Joints[JointType.KneeRight].Position);

            DepthImagePoint leftKnee =
                depth.MapFromSkeletonPoint(first.Joints[JointType.KneeLeft].Position);


            //Map a depth point to a point on the color image
            //head
            ColorImagePoint headColorPoint =
                depth.MapToColorImagePoint(headDepthPoint.X, headDepthPoint.Y,
                ColorImageFormat.RgbResolution640x480Fps30);
            //left hand
            ColorImagePoint leftColorPoint =
                depth.MapToColorImagePoint(leftDepthPoint.X, leftDepthPoint.Y,
                ColorImageFormat.RgbResolution640x480Fps30);
            //right hand
            ColorImagePoint rightColorPoint =
                depth.MapToColorImagePoint(rightDepthPoint.X, rightDepthPoint.Y,
                ColorImageFormat.RgbResolution640x480Fps30);

            ColorImagePoint leftKneeColorPoint =
                depth.MapToColorImagePoint(leftKnee.X, leftKnee.Y,
                ColorImageFormat.RgbResolution640x480Fps30);

            ColorImagePoint rightKneeColorPoint =
                depth.MapToColorImagePoint(rightKnee.X, rightKnee.Y,
                ColorImageFormat.RgbResolution640x480Fps30);



            //Set location
            CameraPosition(headImage, headColorPoint);
            CameraPosition(leftEllipse, leftColorPoint);
            CameraPosition(rightEllipse, rightColorPoint);


            Joint LEFTKNEE = first.Joints[JointType.KneeLeft];
            Joint RIGHTKNEE = first.Joints[JointType.KneeRight];

            if ((LEFTKNEE.TrackingState == JointTrackingState.Inferred ||
            LEFTKNEE.TrackingState == JointTrackingState.Tracked) &&
            (RIGHTKNEE.TrackingState == JointTrackingState.Tracked ||
            RIGHTKNEE.TrackingState == JointTrackingState.Inferred))
            {
                CameraPosition(rightknee, rightKneeColorPoint);
                CameraPosition(leftknee, leftKneeColorPoint);
            }

            else if (LEFTKNEE.TrackingState == JointTrackingState.Inferred ||
                    LEFTKNEE.TrackingState == JointTrackingState.Tracked)
            {
                CameraPosition(leftknee, leftKneeColorPoint);
            }

            else if (RIGHTKNEE.TrackingState == JointTrackingState.Inferred ||
                    RIGHTKNEE.TrackingState == JointTrackingState.Tracked)
            {
                CameraPosition(rightknee, rightKneeColorPoint);
            }
        }        
    }


    Skeleton GetFirstSkeleton(AllFramesReadyEventArgs e)
    {
        using (SkeletonFrame skeletonFrameData = e.OpenSkeletonFrame())
        {
            if (skeletonFrameData == null)
            {
                return null; 
            }


            skeletonFrameData.CopySkeletonDataTo(allSkeletons);

            //get the first tracked skeleton
            Skeleton first = (from s in allSkeletons
                                     where s.TrackingState == SkeletonTrackingState.Tracked
                                     select s).FirstOrDefault();

            return first;

        }
    }

    private void StopKinect(KinectSensor sensor)
    {
        if (sensor != null)
        {
            if (sensor.IsRunning)
            {
                //stop sensor 
                sensor.Stop();

                //stop audio if not null
                if (sensor.AudioSource != null)
                {
                    sensor.AudioSource.Stop();
                }


            }
        }
    }

    private void CameraPosition(FrameworkElement element, ColorImagePoint point)
    {
        //Divide by 2 for width and height so point is right in the middle 
        // instead of in top/left corner
        Canvas.SetLeft(element, point.X - element.Width / 2);
        Canvas.SetTop(element, point.Y - element.Height / 2);

    }

    private void ScalePosition(FrameworkElement element, Joint joint)
    {
        //convert the value to X/Y
        //Joint scaledJoint = joint.ScaleTo(1280, 720); 

        //convert & scale (.3 = means 1/3 of joint distance)
        Joint scaledJoint = joint.ScaleTo(1280, 720, .3f, .3f);

        Canvas.SetLeft(element, scaledJoint.Position.X);
        Canvas.SetTop(element, scaledJoint.Position.Y); 

    }


    private void Window_Closing(object sender, System.ComponentModel.CancelEventArgs e)
    {
        closing = true; 
        StopKinect(kinectSensorChooser1.Kinect); 
    }

    private void kinectDepthViewer1_Loaded(object sender, RoutedEventArgs e)
    {

    }

   }
}

XAML

<Window x:Class="SkeletalTracking.MainWindow"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    Title="MainWindow" Height="600" Width="800" Loaded="Window_Loaded" 
    xmlns:my="clr-namespace:Microsoft.Samples.Kinect.WpfViewers;assembly=Microsoft.Samples.Kinect.WpfViewers" 
    Closing="Window_Closing" WindowState="Maximized">       
<Canvas Name="MainCanvas">
    <my:KinectColorViewer Canvas.Left="0" Canvas.Top="0" Width="640" Height="480" Name="kinectColorViewer1" 
                          Kinect="{Binding ElementName=kinectSensorChooser1, Path=Kinect}" />
    <Ellipse Canvas.Left="0" Canvas.Top="0" Height="50" Name="leftEllipse"  Width="50" Fill="#FF4D298D" Opacity="1" Stroke="White" />
    <Ellipse Canvas.Left="100" Canvas.Top="0" Fill="#FF2CACE3" Height="50" Name="rightEllipse" Width="50" Opacity="1" Stroke="White" />
    <my:KinectSensorChooser Canvas.Left="250" Canvas.Top="380" Name="kinectSensorChooser1" Width="328" />
    <Image Canvas.Left="66" Canvas.Top="90" Height="87" Name="headImage" Stretch="Fill" Width="84" Source="/SkeletalTracking;component/c4f-color.png" />
    <Ellipse Canvas.Left="283" Canvas.Top="233" Height="23" Name="leftknee" Stroke="Black" Width="29" />
    <Ellipse Canvas.Left="232" Canvas.Top="233" Height="23" Name="rightknee" Stroke="Black" Width="30" />
</Canvas>

这里有张图片，展示了Kinect有时候的不准确：

提示：注意只有我的手臂被检测到了，背景的部分没有。

2
谢谢您的回答，但您能否看一下以下视频(http://www.youtube.com/watch?v=T_QLguHvACs&t=0m45s)。Kinect能够侧向跟踪身体，为了澄清，请查看此截图(http://i42.tinypic.com/95myd4.png)。谢谢！这是微软官方视频，也称为“Kinect效应”。 - Velth
2
@MarkvanVelthoven 我认为他在考虑每个关节分配的JointTrackingState状态。PositionOnly状态值与Skeleton相关，具有该状态的Skeleton不提供Joint集合，因为它实际上没有被跟踪。因此，您应确保绘制所有关节，而不仅仅是具有JointTrackingState.Tracked状态的关节。 - Renaud Dumont
@Outlaw Lemur：非常感谢您的努力和回答。不幸的是，追踪腿部仍然不准确。越来越多的情况下，我开始认为这是Kinect的少数限制之一。附言：您尝试过代码并/或追踪腿部吗？ - Velth
1
@Outlaw Lemur，您能否分享一下您的Visual Studio解决方案，以便我测试上面编写的代码。也许我会有不同的实现方式。谢谢！ - Velth
1
更新了Kinect Effect视频的Youtube链接：https://www.youtube.com/watch?v=RN_zu3xVQ5M - Steven Magana-Zook
显示剩余6条评论
0
Kinect由于其自身SDK的限制而无法准确跟踪骨架。实际上，每次设备尝试捕捉骨架时，它总是假定骨架面向前方。如果您保存关节，从俯视图中查看并绘制x和z坐标，则可以很容易地看到这一点。
0
我也在使用Kinect，你可以获取膝盖关节角度的值，但你似乎面临的问题是由于Kinect本身推断的值没有显示出来。 最好使用整个骨架来跟踪你的膝盖并希望你能得到每个关节运动的所有值。
网页内容由stack overflow 提供, 点击上面的
可以查看英文原文，
原文链接