Windows 10 and YOLOV2 for Object Detection Series
- Introduction to YoloV2 for object detection
- Create a basic Windows10 App and use YoloV2 in the camera for object detection
- Transform YoloV2 output analysis to C# classes and display them in frames
- Resize YoloV2 output to support multiple formats and process and display frames per second
Hi!
Let me start from my previous post where I already coded a a real-time video camera feed process using with Tiny-YoloV2. The model returns results in a 416×416 size image, and that’s why the detected frame with the person looks kind of weird.
It’s not complicated to work with the output so it can be adapted to the size of the Webcam control, only a couple of lines of code.
| private uint _overlayCanvasActualWidth; | |
| private uint _overlayCanvasActualHeight; | |
| private void DrawYoloBoundingBox(YoloBoundingBox box, Canvas overlayCanvas) | |
| { | |
| // get current webcam and canvas size | |
| _overlayCanvasActualWidth = (uint)YoloCanvas.ActualWidth; | |
| _overlayCanvasActualHeight = (uint)YoloCanvas.ActualHeight; | |
| // process output boxes | |
| var x = (uint)Math.Max(box.X, 0); | |
| var y = (uint)Math.Max(box.Y, 0); | |
| var w = (uint)Math.Min(overlayCanvas.ActualWidth – x, box.Width); | |
| var h = (uint)Math.Min(overlayCanvas.ActualHeight – y, box.Height); | |
| // fit to current canvas and webcam size | |
| x = _overlayCanvasActualWidth * x / 416; | |
| y = _overlayCanvasActualHeight * y / 416; | |
| w = _overlayCanvasActualWidth * w / 416; | |
| h = _overlayCanvasActualHeight * h / 416; | |
| … |
And in this way, we can draw the frames with the right format. In the following image it is possible to see how a person is detected, with a low precision score and as well as one of the pictures on the wall it is detected as a monitor.
From here, it’s all about optimization, for example, take the value of the webcam control only when the app is resized.
| protected override async void OnNavigatedTo(NavigationEventArgs e) | |
| { | |
| LoadYoloOnnxModel(); | |
| Window.Current.SizeChanged += Current_SizeChanged; | |
| await CameraPreview.StartAsync(); | |
| CameraPreview.CameraHelper.FrameArrived += CameraHelper_FrameArrived; | |
| } | |
| private void Current_SizeChanged(object sender, WindowSizeChangedEventArgs e) | |
| { | |
| _yoloCanvasActualWidth = (uint)YoloCanvas.ActualWidth; | |
| _yoloCanvasActualHeight = (uint)YoloCanvas.ActualHeight; | |
| } |
In the final example, I have also added a visual indicator as a status bar, which shows the number of frames processed per second and the real size of the elements in the App.
The full code for the example can be downloaded from
https://github.com/elbruno/Blog/tree/master/20180704%20UwpMLNet%20TinyYoloV2
Happy Coding!
Greetings @ Toronto
El Bruno
References
- YOLO: Real-time object detection
- YOLO9000: Better, Faster, Stronger by Joseph Redmon and Ali Farhadi (2016)
- ONNX Tools
- Azure AI Gallery, Tiny YOLO V2
- El Bruno, Windows Community Toolkit V 3.0 makes life incredibly easy if you need working with the camera in a UWP App
- Visual Studio Marketplace, Visual Studio Tools for AI
- Real-time object detection with YOLO
- Rene Schulte GitHub
- Sevans4067 WinML-TinyYolo
I know this is an old post, any idea if this this will work on HoloLens 2?
I tried running it in my desktop but having error System.NullReferenceException: ‘Object reference not set to an instance of an object.’
on LearningModelBindingPreview binding = new LearningModelBindingPreview(_learningModel);.
LikeLike