基于3D体感摄像机kinect的人体动作识别软件设计
摘 要
随着计算机在人们生活中的普及和深入，传统的鼠标和键盘难以满足日益多样化的控制需求。基于视觉的体感控制是实现新一代人机交互所不可缺少的一项关键技术。Kinect 作为一款革命性的产品，正在提供一项全新的人机交互的方式，使人机互动的理念更加彻底的展现出来。它能够捕捉、跟踪以及识别人体的动作，手势以及声音。
本文内容涵盖对人机交互技术应用的介绍，Kinect硬件设备构造与原理的分析，Kinect开发技术的介绍，并利用Kinect在Windows 7系统下，使用C#语言设计了一套人体动作识别系统。本系统通过设置合适的阈值，判断用左手与头部在Kinect空间Y轴的距离是否超过阈值控制鼠标左键的按下与弹起。通过右手在空气中的移动的坐标映射到屏幕坐标控制鼠标移动。本设计与其他同类设计相比，主要创新之处有以下几点：
1) 本系统在使用前无需对系统进行训练，提高了人体动作识别系统的便捷性。
2) 本系统使用关节点相对位置判断操作意图，不需要进行手势识别，提高了控制识别的成功率。
3) 本系统在人体动作识别的过程中加入了滤波算法和平滑参数，提高了系统的流畅性和稳定性。
关键词：Kinect；人机交互；人体动作识别
Abstract
As computers spread in people's lives and deepening, the traditional mouse and keyboard control is difficult to meet the increasingly diverse needs. Vision-based motion control is a key technology to achieve a new generation of human-computer interaction. As a revolutionary product, Kinect is providing a new way of human-computer interaction, makes the idea of human-computer interaction more thoroughly unfolded. It can capture and track and recognize the movement of the human body, gestures and voice.
This article covers an introduction to human-computer interaction technology, an analysis to Kinect hardware structure and theory, Kinect development technology introduction and use Kinect on Windows systems, using the C# language design a motion control system. By setting an appropriate threshold to determine whether the distance between his left hand and head in the Y axis of Kinect space exceeds the threshold, the system control the left mouse button press and bounce. By mapping the coordinates of the air to the screen, the system can control of the mouse movement. Finally, we use the PC game "Fruit Ninja" to test the system under different lighting conditions. The test results showed that: the system in the changing light conditions has good robustness.
Compared with other similar design, the main innovations of this design are the following:
1) Before using the system, users don’t need any system training, which makes the motion control system more easily to use.
2) In this system, we use the relative position of joints to determine operation intention, instead of gesture recognition, which improves the success rate of control identification
3) In skeletal tracking process，we add filtering algorithms and smoothing parameters to improve the fluency and stability of the system.
Key Words：Kinect; somatosensory control; human-computer interaction; skeletal tracking
目 录
第1章 绪论 1
1.1 课题的研究背景与意义 1
1.2 国内外研究现状 3
1.3 课题研究内容 4
第2章 Kinect人体动作识别系统开发介绍 5
2.1 Kinect简介 5
2.2 Kinect驱动平台 7
2.3 Kinect工作原理 8
第3章 设计实现 9
3.1 总体设计方案 9
3.2 实验平台搭建 10
3.2.1 软硬件环境 10
3.2.2 Kinect的安装 10
3.3 Kinect数据采集 10
3.3.1 Kinect彩色图像数据采集 10
3.3.2 Kinect深度图像数据采集 11
3.3.3 Kinect骨骼数据采集 13
3.4 数据处理 14
3.4.1 深度图像二值化处理 14
3.4.2 骨路关节点的平滑处理 15
3.5 Kinect数据合并 17
3.6 深度图像空间坐标 18
3.7 骨骼空间坐标 18
3.8 NUI坐标转换 19
3.9 系统功能核心代码 20
第4章 系统测试分析 28
第5章 总结与展望 30
5.1 总结 30
5.2 展望 30
参考文献 32
致谢 34
参考文献
[1] 余涛. Kinect应用开发实践[M]. 北京:机械工业出版社,2012:74-89.
[2] 朱涛,金国栋,芦利斌. Kinect应用概述及发展前景[J]. 现代计算机,2013,10:1007-1423[3].
[3] 余涛,叶金永,邵菲杰等. Kinect核心技术之骨架追踪技术[J]. 数字技术与应用,2012,(10):70-1.
[4] Antonio Sgorbissa, Damiano Verda. Structure-based object representation and classification in mobile roboticsthrough a Microsoft Kinect[J]. Robotics and Autonomous
[5] Andrea Sanna, Fabrizio Lamberti, Gianluca Paravati. A Kinect-based natural interface for quadrotor control[J]. Entertainment Computing, Volume 4, Issue 3, August 2013, Pages 179-186.
Systems, In Press, Corrected Proof，Available online 9 July 2013.
[6] Vitor Filipe, Filipe Fernandes, Hugo Fernandes. Blind Navigation Support System based on Microsoft Kinect[J]. Procedia Computer Science, Volume 14, 2012, Pages 94-101.
[7] 肖坚. 基于虚拟尸检技术推断死亡时间的实验研究[D]. 华中科技大学,2006.
[8] J.L. Raheja, A. Chaudhary, K. Singal. Tracking of fingertips and centers of palm using kinect[C]. Proc of 2011 IEEE Third International Conference on Computational Intelligence, Modelling and Simulation, 2011:248-252.
[9] T.T. Thanh, F. Chen, K. Kotani, et al. Extraction of Discriminative Patterns from Skeleton Sequences for Human Action Recognition[C]. Proc of IEEE International Conference on Computing and Communication Technologies, Research, Innovation, and Vision for the Future, 2012: 1-6.
[10] 李瑞峰,曹雏清,王丽. 基于深度图像和表观特征的手势识别[J]. 华中科技大学学报(自然科学版),2011,S2:88-91.
[11] S Sempena, N.U. Maulidevi, P.R. Aryan. Human action recognition using dynamic time warping[C]. Proc of 2011 International Conference on Electrical Engineering and Informatics (ICEEI), 2011: 1-5.
[12] R. Girshick, J. Shotton, P. Kohli, et al. Efficient regression of general-activity human poses from depth images[C]. Proc of IEEE International Conferenceon Computer Vision, 2011:415-422.
[13] M. Raptis,D. Kirovski, H. Hoppe. Real-time classification of dance gestures from skeleton animation[C]. Proc of the 2011 ACM S1GGRAPH, Eurographics Symposium on Computer Animation, 2011:147-156.
[14] L. Xia, C.C. Chen, J. Aggarwal. Human detection using depth information by Kinect[C].Proc of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops,2011: 15-22.
[15] 况鹰. 基于Kinect运动捕获的三维虚拟试衣[D]. 华南理工大学,2012.
[16] 陈静,陈泽宇. 基于Kinect的手势识别技术及其在教学中的应用[D]. 上海:上海交通大学,2013.
[17] 钱鹤庆,申瑞名. 应用Kinect与手势识别的增强现实教育辅助系统[D]. 上海:上海交通大学,2011.
[18] 金连文,徐睿,杨端端,镇立新,黄建成. 手指书写:一种虚拟文字识别人机交互新方法[J]. 电子学报,2007,03:396-401.
[19] 倪晨,邱鹏,王锋,曹慧. Kinect体感技术在人机交互中的应用研究[J]. 信息技术与信息化,2013,01:87-90.
[20] 荣竹,罗晓晞. 基于Kinect的体感控制技术在科普展项创新设计中的应用研究[J]. 现代机械,2013,04:93-96.
[21] 顾容,何益明,金丽燕. 基于Kinect骨骼追踪技术的智能教学控制系统的设计与实现[J]. 计算机应用与软件,2013,06:276-280.
[22] 马建荣,章苏静,李凤. 基于体感技术的亲子互动游戏设计与实现[J]. 中国电化教育,2012,09:85-88.
[23] 李兆堃. 基于Kinect体感技术的人机交互环境[J]. 数字技术与应用,2013,09:65-66.
[24] 邬大鹏. 基于视觉的手势识别及人机交互研究[D].南京航空航天大学,2010.
[25] 王康. 基于Kinect的体感交互式电子白板初探[J]. 中国现代教育装备,2012,06:29-30.
[26] 马荟. 连接未来的人机交互[J]. 互联网周刊,2010,07:64-66.
[27] 蔡睿妍. 基于Kinect的多点触控系统的设计与实现[J]. 电脑知识与技术,2012,8(16):3987-3989
[28] 张毅,张烁,罗元等. 基于Kinect深度图像信息的手势轨迹识别及应用[J]. 计算机应用研究,2012,09:3547-3550.
[29] Microsoft Corporation. Programming Guide of Kinect for Windows SDK[EB/OL].
http://msdn.microsoft.com/en-us/library/hh855348.aspx.2013-12-29
[30] 李中伟. 基于数字光栅投影的结构光三维测量技术与系统研究[D]. 武汉:华中科技大学,2009:41-64.
[31] Jarrett Webb, James Ashley. Beginning Kinect Programming with the Microsoft Kinect SDK[M]. Apress,2012:45-52.
http://www.bysj360.com/html/5839.html
http://www.bysj360.com/html/6302.html
