#CustomVision – Analyzing images in a Console App using a #CustomVision project in a #Docker Container

Hi !

This is a special post. It’s the 1st one I write completely in my MacBook, so I’m sure that Javier (@jsuarezruiz), Yeray (@JosueYeray), Braulio (@braulio_sl), Luis, Sara, Roberto and other mac users will be proud of me 馃榾

So, I build and run my Custom Vision Marvel project in Docker for Mac. Smooth build and also a fast one!

docker build -t elbruno/cvmarvel:3.0 .

01 doker build on mac

Then get the image id and run the image

03 docker list images and run image

Final step is to play around with curl in bash to post the image (the file name with @ prefix took me some bing searches). Iron Fist detected !

curl -X POST http://127.0.0.1:8080/image -F imageData=@img1.jpg聽

05 docker bash ls image analyzed and source image.png

Ok, the environment is working, so it’s time to create a .NetCore Console App to test this using amazing C# code. I have all my code in Azure Dev Ops, so I sync my repo and聽 added a new project in my current solution

06 new netcore project in visual studio for mac

Some C# lines in my console app and I was able to analyze a local picture using the Custom Vision Model in a container

07 console app in vs for mac detected image

The source code is very simple

Happy coding!

Greetings @ Toronto

El Bruno

References

My Posts

Windows 10 and YOLOV2 for Object Detection Series

Advertisements

#CustomVision – Utilizando un proyecto Custom Vision en un #Docker Container local

Buenas !

Ahora que ya tengo en funcionamiento Docker en Windows 10 es tiempo de exportar un proyecto de CustomVision y ejecutarlo dentro de un container.

Cuando exportamos hay 2 version disponibles: Linux or Windows

00 CV exported to docker

En mi caso no pude compilar la versi贸n de Windows, asi que trabajare con la versi贸n de Linux. El zip que descargamos tiene varios archivos, como el DockerFile con la definici贸n del container, el modelo de ML, los archivos python para leer y realizar an谩lisis de imagenes y varios archivos mas.

FROM python:3.5

ADD app /app

RUN pip install –upgrade pip
RUN pip install -r /app/requirements.txt

# Expose the port
EXPOSE 80

# Set the working directory
WORKDIR /app

# Run the flask server for the endpoints
CMD python app.py

Esta imagen utiliza Python 3.5, y el comando para compilar la imagen es

docker build -t elbruno/cvmarvel:3.0 .

05 CV docker build image in windows

Un par de segundos despu茅s, la imagen esta compilada y disponible en la store local

docker image ls

07 Docker local images

Ahora que se mi IMAGE ID, ya puedo iniciar la ejecuci贸n de la imagen. En este caso, utilizare el puerto 8080

docker run -p 127.0.0.1:8080:80 -d ddd1623ee694

En este ejemplo utilizare PowerShell y el comando Invoke-WebRequest para realizar una petici贸n HTTP Post con una imagen

Invoke-WebRequest -uri “http://127.0.0.1:8080/image” -Method Post -Infile “D:\docker\test01.jpg” -ContentType ‘image/jpg’

06 CV running and testing an image on docker

El resultado detecta una imagen, sin embargo no tengo todo el resultado Json disponible. Para esto, agregare un archivo de salida en la petici贸n para guardar en el mismo el resultado del an谩lisis de la imagen. Ahora ya puedo ver que detecta Iron Fist y a Venom!

Invoke-WebRequest -uri “http://127.0.0.1:8080/image” -Method Post -Infile “D:\docker\test01.jpg” -Outfile “D:\docker\result.json” -ContentType ‘image/jpg’

08 docker results in visual studio code

Happy coding!

Greetings @ Burlington

El Bruno

References

My Posts

Windows 10 and YOLOV2 for Object Detection Series

#CustomVision – Running a Custom Vision project in a local #Docker Container

Hi !

So now that I have Docker running in Windows 10, it’s time to use a Custom Vision model in Windows hosted in Docker container.

There are 2 version available to export from CustomVision.ai for each one the projects: Linux or Windows

00 CV exported to docker

I could not sucessfully built the Windows version, so I’ll work with the Linux one. Once exported the zip file have a set of python files to tun the model, the model file (model.pb) and a Dockerfile to build the docker image.

FROM python:3.5

ADD app /app

RUN pip install –upgrade pip
RUN pip install -r /app/requirements.txt

# Expose the port
EXPOSE 80

# Set the working directory
WORKDIR /app

# Run the flask server for the endpoints
CMD python app.py

The image uses Python 3.5, and the build command is as simple as

docker build -t elbruno/cvmarvel:3.0 .

05 CV docker build image in windows

After a couple of seconds, the image is build in the local store

docker image ls

07 Docker local images

Once I have my IMAGE ID, it’s time to start the image. For this demo, I’ll use the port 8080

docker run -p 127.0.0.1:8080:80 -d ddd1623ee694

And then I can submit an image using Invoke-WebRequest and view the results directly in PowerShell

Invoke-WebRequest -uri “http://127.0.0.1:8080/image” -Method Post -Infile “D:\docker\test01.jpg” -ContentType ‘image/jpg’

06 CV running and testing an image on docker

In order to get the complete output of the POST request, I must add an OutFile into the PowerShell comand. And in the complete output we can see some Iron Fist and Venom results!

Invoke-WebRequest -uri “http://127.0.0.1:8080/image” -Method Post -Infile “D:\docker\test01.jpg” -Outfile “D:\docker\result.json” -ContentType ‘image/jpg’

08 docker results in visual studio code

Happy coding!

Greetings @ Toronto

El Bruno

References

My Posts

Windows 10 and YOLOV2 for Object Detection Series

#WinML 鈥 #CustomVision, reconocimiento de objectos utilizando Onnx en Windows10, calculando FPS

Buenas !

Hoy va un post r谩pido. Y es del tipo de ayuda mental, ya que siempre que tengo que mostrar informaci贸n relacionada a proceso de frames por segundo, tengo que buscar en mis aplicaciones anteriores.

En este caso agregare esta informaci贸n al reconocimiento de im谩genes con un modelo Onnx exportado desde Custom Vision. En la UWP app que he creado en post anteriores, mostrare un label con la fecha y hora, e informaci贸n de FPS.

01 custom vision uwp frame analysis using onnx fps

El c贸digo es muy simple, especial detalle a la linea 10

Brain Backup done!

The full app can be seen in聽https://github.com/elbruno/events/tree/master/2019%2001%2010%20CodeMash%20CustomVision/CSharp/CustomVisionMarvelConsole01

Happy Coding!

Saludos @ Burlington

El Bruno

References

My Posts

Windows 10 and YOLOV2 for Object Detection Series

#WinML 鈥 #CustomVision, object recognition using Onnx in Windows10, calculate FPS

Hi !

Quick post today. And it鈥檚 mostly as a brain reminder on the best way to perform a Frames Per Second calculation when we are analyzing images using a ONNX model. In the final UWP app, I added a top right label displaying the current date and time, and the processed FPS

01 custom vision uwp frame analysis using onnx fps

And the code behind all this is very simple, specially line 10

So, I鈥檒l search for my sample next time I need to display this.

The full app can be seen in聽https://github.com/elbruno/events/tree/master/2019%2001%2010%20CodeMash%20CustomVision/CSharp/CustomVisionMarvelConsole01

Happy Coding!

Greetings @ Burlington

El Bruno

References

My Posts

Windows 10 and YOLOV2 for Object Detection Series

#Onnx 鈥 Reconocimiento de objetos con #CustomVision y ONNX desde aplicaciones Windows 10 con Windows ML, Frames de objetos detectados

Buenas !

Custom Vision nos permite crear modelos de reconocimiento de objetos. Una vez entrenados estos modelos, podemos analizar una imagen y el modelo nos ofrecer谩 como respuesta

  • Una lista de objetos [Tags] detectados en cada imagen
  • Para cada Tag tendremos tambi茅n la probabilidad [score] asociado al mismo y una serie de valores num茅ricos con la posici贸n del objeto encontrado dentro de la imagen analizada

En posts anteriores escrib铆 sobre como realizar el an谩lisis de objetos desde el feed de una WebCam en una aplicaci贸n Windows 10. El siguiente paso es mostrar el Frame del objeto reconocido.

01 custom vision analysis and draw frame

El siguiente c贸digo muestra un ejemplo sobre como mostrar los frames en la Windows 10 App utilizando un Canvas. Las 2 funciones principales son

  • DrawFrames() donde realizado una iteraci贸n sobre las predicciones realizadas
  • DrawFrame() esta es la funci贸n que se encarga de dibujar el Frame en tiempo real. Hay un poco de matem谩ticas en la misma para ajustar los valores de ONNX al tama帽o real del Canvas y de la WebCam.

Por ejemplo, estos son los valores con los que trabajo en un tag de Iron Fist en la imagen de este post.

  • El tama帽o del Canvas es de ActualWidth: 1356, ActualHeight: 700
  • Los valores que retorna ONNX son Top: 20.80284, Left: 73.15757, Height: 54.41817, Width: 24.3813
  • El Frame a mostrar se dibujar谩 con los siguientes valores Y: 140, x: 989, Height: 378, Width: 325

En siguientes posts comentare detalles finales sobre como medir el tiempo de procesamiento y otros tips m谩s.

The full app can be seen in聽https://github.com/elbruno/events/tree/master/2019%2001%2010%20CodeMash%20CustomVision/CSharp/CustomVisionMarvelConsole01

Happy Coding!

Greetings @ Burlington

El Bruno

References

My Posts

Windows 10 and YOLOV2 for Object Detection Series

#Onnx 鈥 Object recognition with #CustomVision and ONNX in Windows applications using Windows ML, drawing frames

Hi !

Custom Vision Allows us to create models of Object recognition. Once these models are trained, we can analyze an image and the model will offer us as an answer

  • A list of [Tags] objects detected in each image
  • For each TAG we will also have the probability [score] associated with it and a series of numerical values with the position of the object found within the analyzed image

In previous posts I wrote about how to perform object analysis from the feed From a Webcam In a Windows 10 application. The next step is to show the Frame of the recognized object.

01 custom vision analysis and draw frame

The following code shows an example of how to show the frames In the Windows 10 App using a Canvas. The 2 main functions are

  • DrawFrames(Where an iteration of the predictions made
  • DrawFrame() This is the function that takes care of drawing the Frame in real time. There’s a little bit of math in it to adjust the ONNX values to the actual size of the Canvas and the Webcam.

For example, these are the values that I work with in a tag of Iron Fist In the image of this post.

  • The Canvas size is Actual Width: 1356, Actual Height: 700
  • The values returned by ONNX prediction process are Top: 20.80284, Left: 73.15757, Height: 54.41817, Width: 24.3813
  • The Frame To show will be drawn with the following values Y: 140, x: 989, Height: 378, Width: 325

In following posts I’ll comment on final details on how to measure processing time and other tips.

The full app can be seen in聽https://github.com/elbruno/events/tree/master/2019%2001%2010%20CodeMash%20CustomVision/CSharp/CustomVisionMarvelConsole01

Happy Coding!

Greetings @ Burlington

El Bruno

References

My Posts

Windows 10 and YOLOV2 for Object Detection Series

#Onnx 鈥 Object recognition with #CustomVision and ONNX in Windows applications using Windows ML

Hi!

One of the most interesting options that gives us Custom Vision, is the ability to export a model trained to be used on other platforms, without invoking Custom Vision own web service.

The 4 options available that we have today are

  • CoreML, iOS 11
  • TensorFlow, Android
  • ONNX, Windows ML
  • DockerFile, Azure IoT Edge, Azure Functions, AzureML

cv marvel export to ios tensorflow onnx dockerfile

I’ll share my experiences using the ONNX exported models being used in a Windows 10 Universal App.

The first thing we have to know is the version of Windows 10 with which we will work, because at the time of export we will see that we have 2 options

  • ONNX 1.0 for Windows 10 lower than 17738
  • ONNX 1.2 for Windows 10 higher than 17738

I am currently working with Win10 18317, so my examples will be for the ONNX version 1.2. The exported file is a zip that internally has the following files

  • CSharp\ObjectDetection.cs
  • python\cntk_predict.py
  • python\object_detection.py
  • labels.txt
  • model.onnx

The directories CSharp and Python have sample files to use the model with these languages. The file [labels.txt] contains the labels defined in the model, and finally the ONNX file is the model per se.

custom vision marvel onnx 1.2 exported files

For this example, I will use a blank App UWP with the following features

  • Added package NuGet Microsoft.Toolkit.Uwp.UI. Controls
  • Using the same, to access the webcam at the beginning of the app
  • We process each of the frames That are received from the webcam

Sample Code

At this point, we can now use our exported model to analyze images. We must add the file ONNX to our project, and configure the same to be a content and to be copied to the output build of our application.

An important detail here, is that if you have [Visual Studio Tools for AI] installed in Visual Studio, when you add this file the extension will automatically add a CS class to use with the model.聽 This class requires a lot of work to work, I recommend deleting it, as we will use as a base that is exported from Custom Vision, [ObjectDetection.cs].

custom vision marvel add onnx file to solution in vs

The file [ObjectDetection.cs] contains everything you need to use our model In a UWP App. At the start of the App we initialize the ONNX model, and in each Frame That is received from the camera will process the same to show the results in the window of Debug.

Now, for our app Work properly, you have to make a series of changes to the file [ObjectDetection.cs]. The changes are mainly related to the way in which WinML Processes the output when analyzing an image:

custom vision marvel uwp app running and analyzing

To be able to work this way, I added a new binding To process the output of the processing. This binding Respects the contract of ONNX with an array Long [1, 4].

This way we have no mistake of those “funny” that make you spend pleasant moments.

The full app can be seen in聽https://github.com/elbruno/events/tree/master/2019%2001%2010%20CodeMash%20CustomVision/CSharp/CustomVisionMarvelConsole01

Happy Coding!

Greetings @ Burlington

El Bruno

References

My Posts

Windows 10 and YOLOV2 for Object Detection Series

#Onnx 鈥 Reconocimiento de objetos con #CustomVision y ONNX desde aplicaciones Windows 10 con Windows ML.

Buenas!

Una de las opciones mas interesantes que nos brinda Custom Vision, es la capacidad de exportar un modelo entrenado para ser utilizado en otras plataformas, sin invocar el servicio web propio de Custom Vision.

Las 4 opciones disponibles que tenemos hoy son

  • CoreML, iOS 11
  • TensorFlow, Android
  • ONNX, Windows ML
  • DockerFile, Azure IoT Edge, Azure Functions, AzureML

cv marvel export to ios tensorflow onnx dockerfile

En el post de hoy comentare mi experiencia utilizando el modelo exportado a formato ONNX y siendo utilizado en una Universal App en Windows 10.

Lo primero que tenemos que conocer es la version de Windows 10 con la que trabajaremos, ya que al momento de exportar veremos que tenemos 2 opciones

  • ONNX 1.0 para versiones de Windows 10 menores que 17738
  • ONNX 1.2 para versiones de Windows 10 superiores a 17738

Actualmente estoy trabajando con Win10 18317, as铆 que mis ejemplos ser谩n para la version ONNX 1.2. El archivo exportado es un zip que internamente posee los siguientes archivos

  • CSharp\ObjectDetection.cs
  • python\cntk_predict.py
  • python\object_detection.py
  • labels.txt
  • model.onnx

Los directorios CSharp y Python poseen archivos de ejemplo para utilizar el modelo con estos lenguajes. El archivo [labels.txt] posee los labels definidos en el modelo, y finalmente el archivo ONNX es el modelo.

custom vision marvel onnx 1.2 exported files

Para este ejemplo, utilizare una UWP App en blanco con las siguientes caracter铆sticas

  • Agregado el paquete NuGet Microsoft.Toolkit.Uwp.UI.Controls
  • Utilizando el mismo, para acceder a la webcam en el inicio de la app
  • Procesamos cada uno de los frames que se reciben desde la webcam

Sample Code

En este momento, ya podemos utilizar nuestro modelo exportado para analizar las im谩genes de las c谩maras. Debemos agregar el archivo ONNX a nuestro proyecto, y configurar el mismo para que sea un contenido y para que se copie al output build de nuestra aplicaci贸n.

Un detalle importante para tener en cuenta es que si tenemos instalada [Visual Studio Tools for AI] en Visual Studio, al momento de agregar este archivo, la extensi贸n nos agregara autom谩ticamente una clase CS para utilizar con el modelo.聽 Esta clase requiere bastante trabajo para funcionar, yo recomiendo borrarla, ya que utilizaremos como base la que se exporta desde Custom Vision, [ObjectDetection.cs].

custom vision marvel add onnx file to solution in vs

El archivo [ObjectDetection.cs] contiene todo lo necesario para utilizar nuestro modelo en una UWP App. En el inicio de la App inicializaremos el modelo ONNX, y en cada Frame que se reciba de la c谩mara procesaremos el mismo para mostrar los resultados en la ventana de Debug.

Ahora bien, para que nuestra app funcione correctamente, hay que hacer una serie de cambios en el archivo [ObjectDetection.cs]. Los cambios est谩n principalmente relacionados en la forma en la que WinML procesa el output cuando se analiza una imagen

custom vision marvel uwp app running and analyzing

Para poder trabajar de esta forma, he agregado un nuevo binding para procesar el output del procesamiento. Este binding respeta el contract de ONNX con un array Long [1, 4].

De esta forma no tenemos ning煤n error de esos 鈥渄ivertidos鈥 que te hacen pasar ratos agradables.

The full app can be seen in聽https://github.com/elbruno/events/tree/master/2019%2001%2010%20CodeMash%20CustomVision/CSharp/CustomVisionMarvelConsole01

Happy Coding!

Saludos @ Burlington

El Bruno

References

My Posts

Windows 10 and YOLOV2 for Object Detection Series

#Onnx 鈥 Object recognition with #CustomVision and ONNX in Windows applications using WinML

Hi!

After a couple of weeks of almost no posts, mostly because of holidays, family trips and some fever days at home, now it’s time to get back to blogging. One of the outstanding issues I have to write is related on how to use ONNX models exported from Custom Vision projects (CV from now on More).聽 Let’s start With the definition of this service:

The Azure Custom Vision API is a cognitive service that lets you build, deploy and improve custom image classifiers. An image classifier is an AI service that sorts images into classes (tags) according to certain characteristics. Unlike the Computer Vision service, Custom Vision allows you to create your own classifications.

There are many tutorials on how to create a CV project, I recommend the official Microsoft documentation, for example:

For this series of posts, I’ve created a [custom object detection] model using some of my Marvel figures.

marvel custom vision sample

The model is successfully trained using 3 [labels]

  • Venom
  • Rocket_Racoon
  • Iron_Fist

The metrics of this are pretty good, with 100% Precision, 100% Recall and 100% mAP. From the official documentation, let’s recap on this terms.

Precision indicates the fraction of identified classifications that were correct. For example, if the model identified 100 images as dogs, and 99 of them were actually of dogs, then the precision would be 99%.
Recall indicates the fraction of actual classifications that were correctly identified. For example, if there were actually 100 images of apples, and the model identified 80 as apples, the recall would be 80%.
.

marvel custom vision iteration 3 trained

The model also seems to work very well with simple images of these toys. The test from the URL of customvision.ai shows us that using photo of Venom, the service test return the Venom label with more 90% of score.

marvel custom vision test with venom toy

The simplest way to use this model is by making an HTTP call. The following example shows how to make this call from a app Of the console in C#. The full app can be seen in聽https://github.com/elbruno/events/tree/master/2019%2001%2010%20CodeMash%20CustomVision/CSharp/CustomVisionMarvelConsole01, although it is very simple

Well, in next posts show how to export this model to ONNX format and then how to use the file ONNX in a app Windows or in a container with Docker..

Happy Coding!

Greetings @ Toronto

El Bruno

References

Windows 10 and YOLOV2 for Object Detection Series