#WinML – How to create a #Windows10 App using #YOLO for object detection (2 of 4)

Windows 10 and YOLOV2 for Object Detection Series

Hi!

Now we already have downloaded the YoloV2 ONNX file. We can create an UWP App to use the model. Here’s the step by step and some comments

1. We create a new Windows 10 UWP App. As always the name is super original

01

2. We are going to use the camera as images source to be processed with YoloV2. There are many examples of how to do this. In this case we will use a control of Windows Community Toolkit V 3.0 (as I wrote in a previous post). We add the following Packages Via NuGet

  • Microsoft.Toolkit.Uwp.UI
  • Microsoft.Toolkit.Uwp.UI.Controls

02

3. We add a control CameraPreview In our Xaml and we’re already 2 lines away of having the camera working.

03

4. We enable the necessary permissions to use the camera in the app manifest. With The following lines our app have a functional camera


using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Navigation;
namespace UwpAppYolo01
{
public sealed partial class MainPage : Page
{
public MainPage()
{
InitializeComponent();
}
protected override async void OnNavigatedTo(NavigationEventArgs e)
{
await CameraPreview.StartAsync();
CameraPreview.CameraHelper.FrameArrived += CameraHelper_FrameArrived;
}
private void CameraHelper_FrameArrived(object sender, Microsoft.Toolkit.Uwp.Helpers.FrameEventArgs e)
{
}
}
}

view raw

UwpCameraStart.cs

hosted with ❤ by GitHub

04

5. The time has come to start using YoloV2. The latest versions of Visual Studio 2017 allow us to import an ONNX model into a project and the IDE will create the classes needed to work with it. We add as an existing file to the file [Tiny-YOLOv2. Onnx].

Our solution should be similar to the following

05

6. The class that is generated in VS to work with the model It’s pretty ugly.

06

7. My suggestion, replace [8d4d0fa662b14686b1865e0e6d3c598eByTinyYoloV2]

07

8. Now we can see the generated class and within it we can find 3 classes

  • An Input class for the Model
  • An Output class for the Model
  • A class to work with the Model

It’s time to load the model into our App. We do this before initializing the camera as shown in the following Code


using System;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Navigation;
namespace UwpAppYolo01
{
public sealed partial class MainPage : Page
{
private TinyYoloV2Model _model;
public MainPage()
{
InitializeComponent();
}
protected override async void OnNavigatedTo(NavigationEventArgs e)
{
LoadYoloOnnxModel();
await CameraPreview.StartAsync();
CameraPreview.CameraHelper.FrameArrived += CameraHelper_FrameArrived;
}
private async void LoadYoloOnnxModel()
{
var file = await Windows.Storage.StorageFile.GetFileFromApplicationUriAsync(new Uri("ms-appx:///Tiny-YOLOv2.onnx"));
_model = await TinyYoloV2Model.CreateTinyYoloV2Model(file); //,
}
private void CameraHelper_FrameArrived(object sender, Microsoft.Toolkit.Uwp.Helpers.FrameEventArgs e)
{
}
}
}

view raw

UwpYoloLoadModel.cs

hosted with ❤ by GitHub

 

9. Because we will use the model as part of our App, we have to define that it be treated as a Content in the process of Build

08

10. The next step is to evaluate a Frame Of the camera with the YOLOV2 model. Here the code shows us how we use the classes we create when we import the model.


private async void CameraHelper_FrameArrived(object sender, Microsoft.Toolkit.Uwp.Helpers.FrameEventArgs e)
{
if (e?.VideoFrame?.SoftwareBitmap == null) return;
var input = new TinyYoloV2ModelInput {image = e.VideoFrame};
var output = await _model.EvaluateAsync(input);
Debug.WriteLine(output.grid.ToString());
}

view raw

UwpYoloCameraFrameEval.cs

hosted with ❤ by GitHub

11. Finally a Break Point is an useful way to show the output of the model and the information of the model output, which we will have to process later.

09

 

Well, at this point we are already using the model YoloV2 to analyze the frames of the camera. In the following posts I’ll write on how to work with the output of the model to get information which makes sense to us!

Important: At some point I should write about the model conversion capabilities that Visual Studio Tools brings to us For AI.

Happy Coding!

Greetings @ Toronto

El Bruno

References

57 comments

  1. Pingback: #WinML – How to create a #Windows10 App using #YOLO for object detection (1 of N) – El Bruno
  2. Pingback: #WinML – How to create a #Windows10 App using #YOLO for object detection (3 of N) – El Bruno
  3. Pingback: #WinML – How to create a #Windows10 App using #YOLO for object detection (4 of 4) – El Bruno

  4. Hi El Bruno,

    Excellent tutorial.

    Between steps #3 and #4, I cannot seem to initialize my Camera as I am receiving red error lines under CameraPreview in my MainPage.xaml.cs script:

    await CameraPreview.StartAsync();
    CameraPreview.CameraHelper.FrameArrived += CameraHelper_FrameArrived;

    Also, I see a difference in the MainPage.xaml code for the CameraPreview grid:

    Do you know what my issue may be?

    Cheers,
    Jason

    Liked by 1 person

    Reply

    1. Fixed it in the MainPage.xaml.cs script:

      protected override async void OnNavigatedTo(NavigationEventArgs e)
      {
      LoadYoloOnnxModel();

      await CameraPreviewControl.StartAsync();
      CameraPreviewControl.CameraHelper.FrameArrived += CameraHelper_FrameArrived;
      }

      Here is the code for the XAML:

      Liked by 1 person

      Reply

        2. Hi elbruno,
          I have a problem with adding Tiny-YOLOv2.cs file into my project. Actually, when I add another YOLO file there is no problem and the .cs file will be added automatically. But, when I wanna try the one you recommended, The .cs file does not show up automatically. Can please let me know what the problem can be?
          Thanks

          Like

          Reply

          1. Have you check that the file is not there? check the file system and delete the cs file before adding the yolo file !

            Like

  6. Pingback: #WinML – Problems With ONNX and Machine Learning.Net in the latest versions of #Windows10 Insiders – El Bruno
  7. Pingback: #WinML – How to convert Tiny-YoloV3 model in CoreML format to Onnx and use it in a #Windows10 App – El Bruno
  8. Pingback: #WinML – How to create a #Windows10 App and use #TinyYOLOV2 for object detection (the complete series) – El Bruno
  9. Pingback: #WinML – Demo actualizada para utilizar Tiny YOLOv2 1.2, Windows 10 and YOLOV2 for Object Detection Series – El Bruno
  10. Pingback: #WinML – Updated demo using Tiny YOLO V2 1.2, Windows 10 and YOLOV2 for Object Detection Series – El Bruno
  11. Pingback: #WinML – Alternativas a #TinyYolo para reconocimiento de objetos en formato #ONNX – El Bruno
  12. Pingback: #WinML – Alternatives to #Yolo for object detection in #ONNX format – El Bruno
  13. Pingback: #Event – Materiales utilizados en la sesión [#ArtificialIntelligence and #MachineLearning in #Azure] – El Bruno
  14. Pingback: #Event – Resources used on the [#ArtificialIntelligence and #MachineLearning in #Azure] event – El Bruno
  15. Pingback: #Onnx – Reconocimiento de objetos con #CustomVision y ONNX desde aplicaciones Windows WinML. – El Bruno
  16. Pingback: #Onnx – Object recognition with #CustomVision and ONNX in Windows applications using WinML – El Bruno
  17. Pingback: #Onnx – Reconocimiento de objetos con #CustomVision y ONNX desde aplicaciones Windows 10 con Windows ML. – El Bruno
  18. Pingback: #Onnx – Object recognition with #CustomVision and ONNX in Windows applications using Windows ML – El Bruno
  19. Pingback: #Onnx – Object recognition with #CustomVision and ONNX in Windows applications using Windows ML, drawing frames – El Bruno
  20. Pingback: #Onnx – Reconocimiento de objetos con #CustomVision y ONNX desde aplicaciones Windows 10 con Windows ML, Frames de objetos detectados – El Bruno
  21. Pingback: #WinML – #CustomVision, object recognition using Onnx in Windows10, calculate FPS – El Bruno
  22. Pingback: #WinML – #CustomVision, reconocimiento de objectos utilizando Onnx en Windows10, calculando FPS – El Bruno
  23. Pingback: #Windows10 – Can’t install #docker on Windows 10 Home, need Pro or Enterprise – El Bruno
  24. Pingback: #Windows10 – Problemas al instalar #docker en Windows 10 Home, necesitas Pro o Enterprise – El Bruno
  25. Pingback: #CustomVision – Running a Custom Vision project in a local #Docker Container – El Bruno
  26. Pingback: #CustomVision – Utilizando un proyecto Custom Vision en un #Docker Container local – El Bruno
  27. Pingback: #CustomVision – Analyzing images in a Console App using a #CustomVision project in a #Docker Container – El Bruno
  28. Pingback: #CustomVision – Analizando imágenes en una Console App utilizando un proyecto de #CustomVision en #Docker Container – El Bruno
  29. Pingback: #CustomVision – Analyzing images using PostMan from a #CustomVision project hosted in a #Docker Container – El Bruno
  30. Pingback: #CustomVision – Analizando imágenes con PostMan con un proyecto de #CustomVision en un #Docker Container – El Bruno
  31. Pingback: #CustomVision – Building the CustomVision project in #Docker in a #RaspberryPi – El Bruno
  32. Pingback: #CustomVision – Compilar el proyecto de CustomVision en #Docker en una #RaspberryPi – El Bruno
  33. Pingback: #Docker – Container dies immediately upon successful start in a #RaspberryPi. Of course, it’s all about #TensorFlow dependencies – El Bruno
  34. Pingback: #Docker – Container muere inmediatamente después de ser iniciado en #RaspberryPi. Obviamente, era un problema de dependencias de #TensorFlow – El Bruno
  35. Pingback: #Docker – About ports, IPs and more to access a container hosted in a #RaspberryPi – El Bruno
  36. Pingback: #Docker – Sobre puertos, IPs y mas para acceder a un container alojado en #RaspberryPi – El Bruno
  37. Pingback: #Docker – Average response times using a CustomVision.ai docker container in a #RaspberryPi and a PC – El Bruno
  38. Pingback: #Docker – Tiempos de respuesta promedio utilizando #CustomVision.ai en un contenedor con Docker en #RaspberryPi u en PC – El Bruno
  39. Pingback: #AI – My posts on CustomVision.ai, running on ONNX, Docker, on PC, RaspberryPi, MacOS and more ! – El Bruno
  40. Pingback: #CustomVision – It’s time to move your Custom Vision projects to #Azure! – El Bruno
  41. Pingback: #CustomVision – Es el momento de mover los proyectos de Custom Vision a #Azure! – El Bruno
  42. Pingback: #AI – Mis posts sobre CustomVision.ai, exportando y utilizando ONNX, Docker, en PC, RaspberryPi, MacOS y más ! – El Bruno

  43. I guess I’m missing something but even form your github I have a NullReferenceException with a null learning model on EvaluateAsync line in FrameArrived method, any clue ?

    Liked by 1 person

    Reply
  44. Pingback: #AI – CustomVision.ai project now can export directly toVision AI Developer Kit – El Bruno
  45. Pingback: #AI – Los proyectos de CustomVision.ai se pueden exportar para ser utilizados con el Vision AI Developer Kit – El Bruno
  46. Pingback: #Event – Resources used during the #GlobalAINight @metrotorontoUG – El Bruno

  47. Unable to load the Tiny-YOLOv2.onnx in Visual Studio 2017 i get the error:

    Tiny-YOLOv2.onnx -l CS -n VanUI -t VS_Marketplace’ failed with exit code 1
    mlgen failed: Node:batchnorm Required attribute ‘consumed_inputs’ is missing.

    Like

    Reply
  48. Pingback: #Event – Materiales utilizados durante #GlobalAINight con los amigos de @metrotorontoUG – El Bruno
  49. Pingback: #Event – Resources for the sessions about #DeepLearning and #CustomVision at the @ChicagoCodeCamp – El Bruno

  50. Hi,
    I’m not a specialist in VS and I’m having a hard time to understand what do you mean by “We add as an existing file to the file [Tiny-YOLOv2. Onnx].” I add the onnx file to my project but the .cs file is not created automatically. I’m using Visual Studio Community 2017, Version 15.9.16.

    Thank you very much.. your post has been very useful to me,
    Dania

    Like

    Reply

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.