Hi!
After a couple of weeks of almost no posts, mostly because of holidays, family trips and some fever days at home, now it’s time to get back to blogging. One of the outstanding issues I have to write is related on how to use ONNX models exported from Custom Vision projects (CV from now on More). Let’s start With the definition of this service:
The Azure Custom Vision API is a cognitive service that lets you build, deploy and improve custom image classifiers. An image classifier is an AI service that sorts images into classes (tags) according to certain characteristics. Unlike the Computer Vision service, Custom Vision allows you to create your own classifications.
There are many tutorials on how to create a CV project, I recommend the official Microsoft documentation, for example:
- Quickstart: Create an image classification project with the Custom Vision .NET SDK
- Quickstart: Create an object detection project with the Custom Vision .NET SDK
For this series of posts, I’ve created a [custom object detection] model using some of my Marvel figures.
The model is successfully trained using 3 [labels]
- Venom
- Rocket_Racoon
- Iron_Fist
The metrics of this are pretty good, with 100% Precision, 100% Recall and 100% mAP. From the official documentation, let’s recap on this terms.
Precision indicates the fraction of identified classifications that were correct. For example, if the model identified 100 images as dogs, and 99 of them were actually of dogs, then the precision would be 99%.
Recall indicates the fraction of actual classifications that were correctly identified. For example, if there were actually 100 images of apples, and the model identified 80 as apples, the recall would be 80%..
The model also seems to work very well with simple images of these toys. The test from the URL of customvision.ai shows us that using photo of Venom, the service test return the Venom label with more 90% of score.
The simplest way to use this model is by making an HTTP call. The following example shows how to make this call from a app Of the console in C#. The full app can be seen in https://github.com/elbruno/events/tree/master/2019%2001%2010%20CodeMash%20CustomVision/CSharp/CustomVisionMarvelConsole01, although it is very simple
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| using System; | |
| using System.IO; | |
| using System.Net.Http; | |
| using System.Net.Http.Headers; | |
| using System.Threading.Tasks; | |
| using Newtonsoft.Json; | |
| using Newtonsoft.Json.Linq; | |
| namespace CustomVisionMarvelConsole01 | |
| { | |
| static class Program | |
| { | |
| static void Main() | |
| { | |
| MakePredictionRequest("IMG01.jpg").Wait(); | |
| Console.ReadLine(); | |
| } | |
| static async Task MakePredictionRequest(string imageFilePath) | |
| { | |
| var client = new HttpClient(); | |
| client.DefaultRequestHeaders.Add("Prediction-Key", "<Custom Vision Prediction Key>"); | |
| var url = "https://southcentralus.api.cognitive.microsoft.com/customvision/v2.0/Prediction/<Custom Vision AppKey>/image?iterationId=<Custom Vision IterationId>"; | |
| var byteData = GetImageAsByteArray(imageFilePath); | |
| using (var content = new ByteArrayContent(byteData)) | |
| { | |
| content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream"); | |
| var response = await client.PostAsync(url, content); | |
| var jsonResponse = await response.Content.ReadAsStringAsync(); | |
| var prettyJson = JToken.Parse(jsonResponse).ToString(Formatting.Indented); | |
| Console.WriteLine(prettyJson); | |
| } | |
| } | |
| static byte[] GetImageAsByteArray(string imageFilePath) | |
| { | |
| var fileStream = new FileStream(imageFilePath, FileMode.Open, FileAccess.Read); | |
| var binaryReader = new BinaryReader(fileStream); | |
| return binaryReader.ReadBytes((int)fileStream.Length); | |
| } | |
| } | |
| } |
Well, in next posts show how to export this model to ONNX format and then how to use the file ONNX in a app Windows or in a container with Docker..
Happy Coding!
Greetings @ Toronto
El Bruno
References
- Custom Vision
- Quickstart: Create an image classification project with the Custom Vision .NET SDK
- Quickstart: Create an object detection project with the Custom Vision .NET SDK
Windows 10 and YOLOV2 for Object Detection Series
- Introduction to YoloV2 for object detection
- Create a basic Windows10 App and use YoloV2 in the camera for object detection
- Transform YoloV2 output analysis to C# classes and display them in frames
- Resize YoloV2 output to support multiple formats and process and display frames per second
- How to convert Tiny-YoloV3 model in CoreML format to ONNX and use it in a Windows 10 App
- Updated demo using Tiny YOLO V2 1.2, Windows 10 and YOLOV2 for Object Detection Series
- Alternatives to Yolo for object detection in ONNX format
El Bruno 
31 comments