Windows 10 and YOLOV2 for Object Detection Series
- Introduction to YoloV2 for object detection
- Create a basic Windows10 App and use YoloV2 in the camera for object detection
- Transform YoloV2 output analysis to C# classes and display them in frames
- Resize YoloV2 output to support multiple formats and process and display frames per second
Hi!
Let me start from my previous post where I already coded a a real-time video camera feed process using with Tiny-YoloV2. The model returns results in a 416×416 size image, and that’s why the detected frame with the person looks kind of weird.
It’s not complicated to work with the output so it can be adapted to the size of the Webcam control, only a couple of lines of code.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| private uint _overlayCanvasActualWidth; | |
| private uint _overlayCanvasActualHeight; | |
| private void DrawYoloBoundingBox(YoloBoundingBox box, Canvas overlayCanvas) | |
| { | |
| // get current webcam and canvas size | |
| _overlayCanvasActualWidth = (uint)YoloCanvas.ActualWidth; | |
| _overlayCanvasActualHeight = (uint)YoloCanvas.ActualHeight; | |
| // process output boxes | |
| var x = (uint)Math.Max(box.X, 0); | |
| var y = (uint)Math.Max(box.Y, 0); | |
| var w = (uint)Math.Min(overlayCanvas.ActualWidth – x, box.Width); | |
| var h = (uint)Math.Min(overlayCanvas.ActualHeight – y, box.Height); | |
| // fit to current canvas and webcam size | |
| x = _overlayCanvasActualWidth * x / 416; | |
| y = _overlayCanvasActualHeight * y / 416; | |
| w = _overlayCanvasActualWidth * w / 416; | |
| h = _overlayCanvasActualHeight * h / 416; | |
| ... |
And in this way, we can draw the frames with the right format. In the following image it is possible to see how a person is detected, with a low precision score and as well as one of the pictures on the wall it is detected as a monitor.
From here, it’s all about optimization, for example, take the value of the webcam control only when the app is resized.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| protected override async void OnNavigatedTo(NavigationEventArgs e) | |
| { | |
| LoadYoloOnnxModel(); | |
| Window.Current.SizeChanged += Current_SizeChanged; | |
| await CameraPreview.StartAsync(); | |
| CameraPreview.CameraHelper.FrameArrived += CameraHelper_FrameArrived; | |
| } | |
| private void Current_SizeChanged(object sender, WindowSizeChangedEventArgs e) | |
| { | |
| _yoloCanvasActualWidth = (uint)YoloCanvas.ActualWidth; | |
| _yoloCanvasActualHeight = (uint)YoloCanvas.ActualHeight; | |
| } |
In the final example, I have also added a visual indicator as a status bar, which shows the number of frames processed per second and the real size of the elements in the App.
The full code for the example can be downloaded from
https://github.com/elbruno/Blog/tree/master/20180704%20UwpMLNet%20TinyYoloV2
Happy Coding!
Greetings @ Toronto
El Bruno
References
- YOLO: Real-time object detection
- YOLO9000: Better, Faster, Stronger by Joseph Redmon and Ali Farhadi (2016)
- ONNX Tools
- Azure AI Gallery, Tiny YOLO V2
- El Bruno, Windows Community Toolkit V 3.0 makes life incredibly easy if you need working with the camera in a UWP App
- Visual Studio Marketplace, Visual Studio Tools for AI
- Real-time object detection with YOLO
- Rene Schulte GitHub
- Sevans4067 WinML-TinyYolo
El Bruno 
I know this is an old post, any idea if this this will work on HoloLens 2?
I tried running it in my desktop but having error System.NullReferenceException: ‘Object reference not set to an instance of an object.’
on LearningModelBindingPreview binding = new LearningModelBindingPreview(_learningModel);.
LikeLike
Hola.. compile esta version en vs2019 y tengo problemas con var learningModel = await LearningModelPreview.LoadModelFromStorageFileAsync(file); siempre en la variable learningModel tengo null al cargar el modelo..
puede ser que este proyecto sea muy viejo ? o no .. saludos gracias
LikeLike
pues se me ocurren un par de cosas: 1 que el archivo fisico no este disponible, yo revisaria que se copie al output, 2 que no se esten utilizando las ultimas dlls de WinML, aqui es mas complicado depurar.
Hace mucho que no abro estos proyectos, si tengo un tiempo los reviso.
Saludos
LikeLike
bueno.. revisare lo comentado, gracias
LikeLike