#Onnx – Object recognition with #CustomVision and ONNX in Windows applications using Windows ML, drawing frames

Hi !

Custom Vision Allows us to create models of Object recognition. Once these models are trained, we can analyze an image and the model will offer us as an answer

  • A list of [Tags] objects detected in each image
  • For each TAG we will also have the probability [score] associated with it and a series of numerical values with the position of the object found within the analyzed image

In previous posts I wrote about how to perform object analysis from the feed From a Webcam In a Windows 10 application. The next step is to show the Frame of the recognized object.

01 custom vision analysis and draw frame

The following code shows an example of how to show the frames In the Windows 10 App using a Canvas. The 2 main functions are

  • DrawFrames(Where an iteration of the predictions made
  • DrawFrame() This is the function that takes care of drawing the Frame in real time. There’s a little bit of math in it to adjust the ONNX values to the actual size of the Canvas and the Webcam.

For example, these are the values that I work with in a tag of Iron Fist In the image of this post.

  • The Canvas size is Actual Width: 1356, Actual Height: 700
  • The values returned by ONNX prediction process are Top: 20.80284, Left: 73.15757, Height: 54.41817, Width: 24.3813
  • The Frame To show will be drawn with the following values Y: 140, x: 989, Height: 378, Width: 325

private async Task LoadAndEvaluateModelAsync(VideoFrame videoFrame)
_objectDetection.ProbabilityThreshold = 0.5F;
_stopwatch = Stopwatch.StartNew();
_predictions = await _objectDetection.PredictImageAsync(videoFrame);
await Dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
private void DrawFrames()
var message = $"{DateTime.Now.ToLongTimeString()}{1000f / _stopwatch.ElapsedMilliseconds,4:f1} fps{Environment.NewLine}============================={Environment.NewLine}";
if (_predictions.Count > 0)
foreach (var prediction in _predictions)
DrawFrame(prediction, OverlayCanvas);
TextBlockResults.Text = message;
private void DrawFrame(PredictionModel prediction, Canvas overlayCanvas)
_overlayCanvasActualWidth = (uint)CameraPreview.ActualWidth;
_overlayCanvasActualHeight = (uint)CameraPreview.ActualHeight;
var x = (uint)Math.Max(prediction.Box.Left, 0);
var y = (uint)Math.Max(prediction.Box.Top, 0);
var w = (uint)Math.Min(_overlayCanvasActualWidth x, prediction.Box.Width);
var h = (uint)Math.Min(_overlayCanvasActualHeight y, prediction.Box.Height);
Debug.WriteLine($"\tOverLay Canvas \tActual Width: {_overlayCanvasActualWidth}, Actual Height: {_overlayCanvasActualHeight}");
Debug.WriteLine($"\tOriginal\tY: {y}, x: {x}, Height: {h}, Width: {w}");
// fit to current size
var factorSize = 100u;
x = _overlayCanvasActualWidth * x / factorSize;
y = _overlayCanvasActualHeight * y / factorSize;
w = _overlayCanvasActualWidth * w / factorSize;
h = _overlayCanvasActualHeight * h / factorSize;
Debug.WriteLine($"\tScaled\tY: {y}, x: {x}, Height: {h}, Width: {w}");
var rectStroke = _lineBrushGreen;
switch (prediction.TagName)
case "Venom":
rectStroke = _lineBrushGray;
case "Rocket_Racoon":
rectStroke = _lineBrushYellow;
case "Iron_Fist":
rectStroke = _lineBrushGreen;
var r = new Windows.UI.Xaml.Shapes.Rectangle
Tag = prediction,
Width = w,
Height = h,
Fill = _fillBrush,
Stroke = rectStroke,
StrokeThickness = _lineThickness,
Margin = new Thickness(x, y, 0, 0)
var tb = new TextBlock
Margin = new Thickness(x + 4, y + 4, 0, 0),
Text = $"{prediction}",
FontWeight = FontWeights.Bold,
Width = 126,
Height = 21,
HorizontalTextAlignment = TextAlignment.Center
var textBack = new Windows.UI.Xaml.Shapes.Rectangle
Width = 150,
Height = 29,
Fill = rectStroke,
Margin = new Thickness(x, y, 0, 0)

In following posts I’ll comment on final details on how to measure processing time and other tips.

The full app can be seen in https://github.com/elbruno/events/tree/master/2019%2001%2010%20CodeMash%20CustomVision/CSharp/CustomVisionMarvelConsole01

Happy Coding!

Greetings @ Burlington

El Bruno


My Posts

Windows 10 and YOLOV2 for Object Detection Series


Leave a comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: