Buenas,
in the previous post I mentioned some of the main features to begin work with the kit for the recognition of gestures that included with the Kinect SDK .
Today we will see how to use this functionality in a WPF project. To do this follow these steps:
1 Create a new WPF project. In my case I called [ElBruno.FaceTracking02] and also I added the reference to [Microsoft.Kinect].
2. From the installation folder of the Kinect SDK Developer Toolkit, add references to the assemblies in Face Tracking. The location is similar to:
C:\Program Files\Microsoft SDKs\Kinect\Developer Toolkit v1.5.2\Samples\bin
and the references should be added to
- Microsoft.Kinect.Tookit
- Microsoft.Kinect.Tookit.FaceTracking
3 Change the project settings so that it is compiled into x 86 mode. In the project properties, select the “Build” section in Platform Target select “x 86”, and compile the project.
4. This step is a little “curious”, but in the next post I will put a more elegant way of doing this.From the folder
C:\Program Files\Microsoft SDKs\Kinect\Developer Toolkit v1.5.2\Redist\x86
copy the assemblies
- FaceTrackData.dll
- FaceTrackLib.dll
towards the compilation of our project directory.
5. Add a new UserControl in our project called [FaceTrackingViewer] and a class called [SkeletonFaceTracker]. As they have lot code inside, it is best download example zip and copy it complete.
6. The [SkeletonFaceTracker] class, the charge of using assemblies of the Toolkit, of recognizing and then paint the face points. For this, you subscribe to the [OnFrameReady] event of the Kinect sensor and it uses the Toolkit to get the points of the face. Better see some code that is fairly self-explanatory:
1: internal void OnFrameReady(KinectSensor kinectSensor, ColorImageFormat colorImageFormat,
2: byte[] colorImage, DepthImageFormat depthImageFormat,
3: short[] depthImage, Skeleton skeletonOfInterest)
4: {
5: _skeletonTrackingState = skeletonOfInterest.TrackingState;
6: if (_skeletonTrackingState != SkeletonTrackingState.Tracked)
7: {
8: return;
9: }
10:
11: if (_faceTracker == null)
12: {
13: try
14: {
15: _faceTracker = new FaceTracker(kinectSensor);
16: }
17: catch (InvalidOperationException)
18: {
19: Debug.WriteLine("AllFramesReady - creating a new FaceTracker threw an InvalidOperationException");
20: _faceTracker = null;
21: }
22: }
23:
24: if (_faceTracker == null) return;
25: var frame = _faceTracker.Track(colorImageFormat, colorImage, depthImageFormat, depthImage, skeletonOfInterest);
26: LastFaceTrackSucceeded = frame.TrackSuccessful;
27: if (!LastFaceTrackSucceeded) return;
28: if (_faceTriangles == null)
29: {
30: _faceTriangles = frame.GetTriangles();
31: }
32: _facePoints = frame.GetProjected3DShape();
33: }
7. This class has another function which is which is responsible for drawing the mask on a canvas, where are the points of the face. Apart from the collection of points, creates an array of triangles and the paints on a work context.
1: public void DrawFaceModel(DrawingContext drawingContext)
2: {
3: if (!LastFaceTrackSucceeded || _skeletonTrackingState != SkeletonTrackingState.Tracked)
4: {
5: _startFaceRecognition = DateTime.MinValue;
6: return;
7: }
8:
9: var faceModelPts = _facePoints.Select(t => new Point(t.X + 0.5f, t.Y + 0.5f)).ToList();
10:
11: var faceModel = _faceTriangles.Select(t => new FaceModelTriangle
12: {
13: P1 = faceModelPts[t.First],
14: P2 = faceModelPts[t.Second],
15: P3 = faceModelPts[t.Third]
16: }).ToList();
17:
18: var faceModelGroup = new GeometryGroup();
19: for (var i = 0; i < faceModel.Count; i++)
20: {
21: var faceTriangle = new GeometryGroup();
22: faceTriangle.Children.Add(new LineGeometry(faceModel[i].P1, faceModel[i].P2));
23: faceTriangle.Children.Add(new LineGeometry(faceModel[i].P2, faceModel[i].P3));
24: faceTriangle.Children.Add(new LineGeometry(faceModel[i].P3, faceModel[i].P1));
25: faceModelGroup.Children.Add(faceTriangle);
26: }
27:
28: drawingContext.DrawGeometry(Brushes.BlueViolet, new Pen(Brushes.Red, 1.0), faceModelGroup);
29: if (_startFaceRecognition == DateTime.MinValue)
30: {
31: _startFaceRecognition = DateTime.Now;
32: }
33: else
34: {
35: if (DateTime.Now.Subtract(_startFaceRecognition).TotalSeconds > 5)
36: {
37: Application.Current.Dispatcher.BeginInvoke(new Action(() =>
38: {
39: System.Threading.Thread.Sleep(300);
40: _startFaceRecognition = DateTime.MinValue;
41: }));
42: }
43: }
44: }
8. Now go to the viewfinder, which is that “paints” this surface [FaceTrackingViewer]. Inside the function [OnAllFramesReady] where processed data from the Kinect sensor is identified each of the skeletons in a State [SkeletonTrackingState.Tracked]. Once identified, creates a new instance of the SkeletonFaceTracker class and run the above steps to identify a face face. Each SkeletonFaceTracker instance is stored in a collection.
1: foreach (Skeleton skeleton in this.skeletonData)
2: {
3: if (skeleton.TrackingState == SkeletonTrackingState.Tracked
4: || skeleton.TrackingState == SkeletonTrackingState.PositionOnly)
5: {
6: if (!this.trackedSkeletons.ContainsKey(skeleton.TrackingId))
7: {
8: this.trackedSkeletons.Add(skeleton.TrackingId, new SkeletonFaceTracker() { });
9: }
10:
11: SkeletonFaceTracker skeletonFaceTracker;
12: if (this.trackedSkeletons.TryGetValue(skeleton.TrackingId, out skeletonFaceTracker))
13: {
14: if (this.Kinect != null)
15: {
16: skeletonFaceTracker.OnFrameReady(this.Kinect, colorImageFormat,
17: colorImage, depthImageFormat, depthImage, skeleton);
18: skeletonFaceTracker.LastTrackedFrame = skeletonFrame.FrameNumber;
19: }
20: }
21: }
22:
23: }
9. Finally, in the OnRender() of the control function is “painted” each one of the faces.
1: protected override void OnRender(DrawingContext drawingContext)
2: {
3: base.OnRender(drawingContext);
4: foreach (var faceInformation in this.trackedSkeletons.Values)
5: {
6: faceInformation.DrawFaceModel(drawingContext);
7: }
8: }
10. The complete example includes a bit of code in the main to detect the kinect sensor and work with the same window. And another couple of details which I will comment on in a next post.
The example can be downloaded from http://sdrv.Ms/MZY0Eq
Saludos @ La Finca
El Bruno
 |
 |
 |
El Bruno 
Leave a comment