#Onnx – Object recognition with #CustomVision and ONNX in Windows applications using Windows ML


One of the most interesting options that gives us Custom Vision, is the ability to export a model trained to be used on other platforms, without invoking Custom Vision own web service.

The 4 options available that we have today are

  • CoreML, iOS 11
  • TensorFlow, Android
  • ONNX, Windows ML
  • DockerFile, Azure IoT Edge, Azure Functions, AzureML

cv marvel export to ios tensorflow onnx dockerfile

I’ll share my experiences using the ONNX exported models being used in a Windows 10 Universal App.

The first thing we have to know is the version of Windows 10 with which we will work, because at the time of export we will see that we have 2 options

  • ONNX 1.0 for Windows 10 lower than 17738
  • ONNX 1.2 for Windows 10 higher than 17738

I am currently working with Win10 18317, so my examples will be for the ONNX version 1.2. The exported file is a zip that internally has the following files

  • CSharp\ObjectDetection.cs
  • python\cntk_predict.py
  • python\object_detection.py
  • labels.txt
  • model.onnx

The directories CSharp and Python have sample files to use the model with these languages. The file [labels.txt] contains the labels defined in the model, and finally the ONNX file is the model per se.

custom vision marvel onnx 1.2 exported files

For this example, I will use a blank App UWP with the following features

  • Added package NuGet Microsoft.Toolkit.Uwp.UI. Controls
  • Using the same, to access the webcam at the beginning of the app
  • We process each of the frames That are received from the webcam

Sample Code

using System;
using System.Diagnostics;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Navigation;
namespace CodeMashMarvelUwp01
public sealed partial class MainPage : Page
public MainPage()
protected override async void OnNavigatedTo(NavigationEventArgs e)
await CameraPreview.StartAsync();
CameraPreview.CameraHelper.FrameArrived += CameraHelper_FrameArrived;
private async void CameraHelper_FrameArrived(object sender, Microsoft.Toolkit.Uwp.Helpers.FrameEventArgs e)
if (e.VideoFrame.SoftwareBitmap == null) return;

At this point, we can now use our exported model to analyze images. We must add the file ONNX to our project, and configure the same to be a content and to be copied to the output build of our application.

An important detail here, is that if you have [Visual Studio Tools for AI] installed in Visual Studio, when you add this file the extension will automatically add a CS class to use with the model.  This class requires a lot of work to work, I recommend deleting it, as we will use as a base that is exported from Custom Vision, [ObjectDetection.cs].

custom vision marvel add onnx file to solution in vs

The file [ObjectDetection.cs] contains everything you need to use our model In a UWP App. At the start of the App we initialize the ONNX model, and in each Frame That is received from the camera will process the same to show the results in the window of Debug.

protected override async void OnNavigatedTo(NavigationEventArgs e)
await CameraPreview.StartAsync();
CameraPreview.CameraHelper.FrameArrived += CameraHelper_FrameArrived;
private async void CameraHelper_FrameArrived(object sender, Microsoft.Toolkit.Uwp.Helpers.FrameEventArgs e)
if (e.VideoFrame.SoftwareBitmap == null) return;
if (_objectDetection?.Model == null) return;
await LoadAndEvaluateModelAsync(e.VideoFrame);
catch (Exception exception)
{ Debug.WriteLine(exception); }
private ObjectDetection _objectDetection;
private async void InitModel()
_objectDetection = new ObjectDetection();
var modelFile = await StorageFile.GetFileFromApplicationUriAsync(new Uri($"ms-appx:///Assets/model.onnx"));
await _objectDetection.Init(modelFile);
private async Task LoadAndEvaluateModelAsync(VideoFrame videoFrame)
var result = await _objectDetection.PredictImageAsync(videoFrame);
var message = $"{DateTime.Now.ToLongTimeString()}{Environment.NewLine}============================={Environment.NewLine}";
message = result.Aggregate(message, (current, predictionModel) => current + $" {predictionModel.TagName} {(predictionModel.Probability * 100.0f):#0.00}% {Environment.NewLine}");

Now, for our app Work properly, you have to make a series of changes to the file [ObjectDetection.cs]. The changes are mainly related to the way in which WinML Processes the output when analyzing an image:

custom vision marvel uwp app running and analyzing

To be able to work this way, I added a new binding To process the output of the processing. This binding Respects the contract of ONNX with an array Long [1, 4].

public sealed class modelOutput
public TensorFloat Model_outputs0 = TensorFloat.Create(new long[] { 1, 4 });
public async Task<IList<PredictionModel>> PredictImageAsync(VideoFrame image)
var output = new modelOutput();
var imageFeature = ImageFeatureValue.CreateFromVideoFrame(image);
var bindings = new LearningModelBinding(Session);
bindings.Bind("data", imageFeature);
bindings.Bind("model_outputs0", output.Model_outputs0);
var result = await Session.EvaluateAsync(bindings, "0");
return Postprocess(output.Model_outputs0);

This way we have no mistake of those “funny” that make you spend pleasant moments.

The full app can be seen in https://github.com/elbruno/events/tree/master/2019%2001%2010%20CodeMash%20CustomVision/CSharp/CustomVisionMarvelConsole01

Happy Coding!

Greetings @ Burlington

El Bruno


My Posts

Windows 10 and YOLOV2 for Object Detection Series


Leave a comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: