Tag: English Post

#VS2019 – ML.NET Model Builder for Object Detection, be careful with file names and VoTT

Buy Me A Coffee

Hi !

This is a sequel part after yesterday post on the new Model Builder Object Detection scenario. I’ve found this error, and it took me some time to figure out the reason.

As usual, an error on the Model Builder, with very few details and the suggestion to check more details on the Azure ML Portal.

On the Azure ML Portal, we can see that 3 experiments failed. Now is time to fig into the logs files and try to figure out what happened.

mlnet 02 azure ml portal error

On the main experiment, after reading the logs, I found out that there is a [File missing for image]. And other file related issues.

mlnet 03 azure portal error file missing

And, after some research (test, fail and test again)

I realized that some of my original file names have a space on the file name.

In example [apple 04.png].

When we create the labeling project using VoTT, the exported project makes some changes into the file names and [apple 04.png] becomes [apple%2004.png]. Here is a part of the exported project:

{
    "name": "mlnet bad file name",
    "securityToken": "mlnet bad file name Token",
    "videoSettings": {
        "frameExtractionRate": 15
    },
    "tags": [
        {
            "name": "apple",
            "color": "#5db300"
        }
    ],
    "id": "aajYgiNHg",
    "activeLearningSettings": {
        "autoDetect": false,
        "predictTag": true,
        "modelPathType": "coco"
    },
    "version": "2.2.0",
    "lastVisitedAssetId": "3d30adce06faf9bf8c6540ec6435ef61",
    "assets": {
        "3d30adce06faf9bf8c6540ec6435ef61": {
            "asset": {
                "format": "png",
                "id": "3d30adce06faf9bf8c6540ec6435ef61",
                "name": "apple%2004.png",
                "path": "file:C:/ML/FreshFood/mlnet%20bad%20file%20names/apple%2004.png",
                "size": {
                    "width": 584,
                    "height": 510
                },
                "state": 2,
                "type": 1
            },
            "regions": [
                {
                    "id": "4BDOAk0Xq",
                    "type": "RECTANGLE",
                    "tags": [
                        "apple"
                    ],
                    "boundingBox": {
                        "height": 505.0856531049251,
                        "width": 579.0878504672897,
                        "left": 1.782359813084112,
                        "top": 1.6381156316916488
                    },

The solution, was to use a tool and remove all the spaces and non-standard characters from my image data set. I also did all the labeling process again, and when I launched again my Object Recognition scenario from Model Builder, everything worked fine!

Happy coding!

Greetings

El Bruno

Resources

#VS2019 – ML.NET Model Builder for Object Detection using #Azure Compute #mlnet #objectdetection

Buy Me A Coffee

Hi !

There is a new scenario available in ML.Net Model Builder for Visual Studio 2019: Object Detection. This scenario is not just image tagging, this scenario allows us to detect objects in an image, and get the specific coordinates and size of the detected objects. As usual, it requires a starting data set with images and labels.

Model Builder Object Detection

This one is super helpful and is also very easy to use. Let’s start with the 1st step

Add a new Machine Learning element in a Visual Studio project, and select Object Detection scenario.

select object detection scenario

This scenario only supports Azure training environment. Lucky me, I already have a Azure ML environment up and running (see references for my previous posts).

Next step is to define the data source for the scenario. In order to do this, we need to define an Image Labeling format file. The suggested tool to do this is VoTT (see references). You need to install the local version for VoTT. As far as I understand, for this Model Builder scenario, it only support VoTT projects using local files.

Define Source Images and Labels

For this post, I created a sample project using Fresh Food labels. Just 4

  • apple
  • apple rotten
  • banana
  • banana rotten

Important: The source connection will define the path where for the source images that will be used for labeling. Target connection defines the project location and also the path for the project exported file.

vott project settins

Once we defined our project properties, we can start defining tags and labeling our objects in the project images. This is a manual process, and it can be tedious.

Important: VoTT support custom detailed areas as the ones you see in the image below. For the Model Builder Scenario, these custom areas will be transformed into standard rectangles.

At any moment, you can save and export your project. This will generate a new folder in the [Target Connection] location named [vott-json-export]. In this folder we can find all the used and labeled images and also the exported JSON file that we will use on Visual Studio.

This is sample content of the exported file:

{
    "name": "MLNet Fresh Food",
    "securityToken": "MLNet Fresh Food Token",
    "videoSettings": {
        "frameExtractionRate": 15
    },
    "tags": [
        {
            "name": "apple_rotten",
            "color": "#5db300"
        },
        {
            "name": "apple",
            "color": "#e81123"
        },
        {
            "name": "banana",
            "color": "#6917aa"
        },
        {
            "name": "banana_rotten",
            "color": "#015cda"
        }
    ],
    "id": "uF6EQo2eX",
    "activeLearningSettings": {
        "autoDetect": false,
        "predictTag": true,
        "modelPathType": "coco"
    },
    "version": "2.2.0",
    "lastVisitedAssetId": "27f36f1bbeb5abc8a505d677931d8e1d",
    "assets": {
        "27f36f1bbeb5abc8a505d677931d8e1d": {
            "asset": {
                "format": "png",
                "id": "27f36f1bbeb5abc8a505d677931d8e1d",
                "name": "bananarotten08.png",
                "path": "file:C:/ML/FreshFood/mlnet/bananarotten08.png",
                "size": {
                    "width": 568,
                    "height": 298
                },
                "state": 2,
                "type": 1
            },
            "regions": [
                {
                    "id": "VVg0KLN6d",
                    "type": "RECTANGLE",
                    "tags": [
                        "banana_rotten"
                    ],
                    "boundingBox": {
                        "height": 158.57173447537474,
                        "width": 470.0359550561798,
                        "left": 21.26011235955056,
                        "top": 24.248394004282655
                    },
                    "points": [
                        {
                            "x": 21.26011235955056,
                            "y": 24.248394004282655
                        },
                        {
                            "x": 491.29606741573036,
                            "y": 24.248394004282655
                        },
                        {
                            "x": 491.29606741573036,
                            "y": 182.8201284796574
                        },
                        {
                            "x": 21.26011235955056,
                            "y": 182.8201284796574
                        }
                    ]
                }
            ],
            "version": "2.2.0"
        },

Back again in Visual Studio, we can select this file as the INPUT Json file for our scenario.

mlnet object detection definicion using the exported file

Train the Model

Everything is ready, let’s train the model, using Azure Compute ;D

The process is similar to the Image Classification scenario, it all start uploading the sources images to a Blob in Azure, and then launch 4 Azure ML Experiments

  • AutoML
  • Preparation
  • Script
  • HyperDrive

Once the process is complete, we can also check output metrics like Precision and Accuracy. This one is not the best model, however I consider this a great starting point for my demo blog.

If you want to know more about Azure ML, this is an great opportunity to play around with these experiments. The Script Experiment (Run 16 in the image) has a lot to dig, like the complete output log, some training python files, the generated ONNX model, and more.

mlnet 11 object detection azure track 01

Test and consume the generated Model

Back to Visual Studio we can check the model metrics and also the Best Model (I have so many questions here!).

And we can test the generated model. As you can see in the next images, using a threshold with value 50, the results are good. Still not detecting the rotten apple with the good ones, so I maybe need to figure out some strategies to improve my model.

sample test detecting apples
sample test detecting bananas

Remember that all these features are still in preview, however they are an amazing starting point and tool to use and learn.

Happy coding!

Greetings

El Bruno

Resources

#Windows10 – My own list of prerequisites to install dlib: Visual Studio C++, CMake and Cuda dlls

Buy Me A Coffee

Hi !

Another blog post reminder, this time related on prerequisites to install dlib. And before start, a dlib description:

Dlib is a modern C++ toolkit containing machine learning algorithms and tools for creating complex software in C++ to solve real world problems. It is used in both industry and academia in a wide range of domains including robotics, embedded devices, mobile phones, and large high performance computing environments. Dlib’s open source licensing allows you to use it in any application, free of charge.

dlib.net

Dlib is a key piece in computer vision processes, so every time I deploy a new Windows 10 instance, I need to install DLib and here are some prerequisites or, as I like to call them, lessons learned on the hard way.

Visual Studio for C++

One of the key requisites is [Visual Studio for C++]. In my default VS installation, I don’t install C++, here is my excuse to install this again.

Install CMake

Install CMake is an easy one, and somehow, I always forget to do this. More information in references section, and if you don’t know what’s CMake:

CMake is a cross-platform free and open-source software tool for managing the build process of software using a compiler-independent method. It supports directory hierarchies and applications that depend on multiple libraries. It is used in conjunction with native build environments such as Make, Qt Creator, Ninja, Apple’s Xcode, and Microsoft Visual Studio. It has minimal dependencies, requiring only a C++ compiler on its own build system.

CMake, Wikipedia

All to avoid this

Cuda Path and cudnn64_7.dll

And the last one:

  • Download cudnn64_7.dll from https://developer.nvidia.com/cudnn
  • Copy the cudnn64_7.dll into the %CUDA_PATH%/bin directory
  • As reference, my CUDA Path is: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin

This one was deep in stack overflow and is super important!

Happy coding!

Greetings

El Bruno

References

#Tools – Add ZoomIt to Windows 10 Start Menu #Windows10

Buy Me A Coffee

Hi !

Here it goes another post reminder, just for me. Every time I install a new Windows 10 instance, I need to search on how to add a .exe file as an application to the Windows Start Menu. So, I’ll use ZoomIt as the sample one and here are the steps.

Disclaimer: if you share your screen often, zoomit is a must have !

  • Right click on your ZoomIt executable file and create a shortcut on your desktop
  • Open the location [%AppData%\Microsoft\Windows\Start Menu\Programs]
  • Move the shortcut you created on the Desktop to the previous location
  • Done!

Now you can search for ZoomIt on the Start Menu Programs:

start menu and zoomit

Happy coding!

Greetings

El Bruno

Resources

#Event – Resources used on my session for the @netcoreconf

Buy Me A Coffee

Hi !

Another huge and amazing events from my friends at the @netcoreconf, perfect excuse to talk about python, machine learning, computer vision and more. This one was also tricky, no slides, just code during 50 min so here are some related resources.

Code

The code used during the session is available here: https://github.com/elbruno/events/tree/main/2020%2010%2003%20Netcoreconf%20Virtual

Session Recording

Resources

Face Recognition and Detecting series in Python

  1. Detecting Faces with 20 lines in Python
  2. Face Recognition with 20 lines in Python
  3. Detecting Facial Features with 20 lines in Python
  4. Facial Features and Face Recognition with 20 lines in Python
  5. Performance improvements with code
  6. Resize the camera input with OpenCV
  7. Working with Haar Cascades and OpenCV

#Tools – Using an old #Android tablet as an additional monitor

Buy Me A Coffee

Hi !

Please don’t ask me why, however today I was in the need to have a 3rd monitor to my current setup.

And, somewhere during these days, I remember that someone commented about using an old iPad / Android tablet as an external monitor.

We don’t do Apple at home (again, don’t ask me why!), however I still have some old Android tablets, and I decided to try some of those apps.

Note: You know the feeling about installing Android apps, you never know what’s there.

So, after a couple of tests, I found a super cool and decent solution:

SpaceDesk (see references)

I won’t bother with detailed instructions;

  • Install the app on your PC, that will be the host.
  • Install the app on the Android tablet (client)
  • Both needs to be in the same network
  • and done !

Some settings to arrange the screen

And, in the Bottom left corner you may see a new screen

3 monitors

Awesome !

Happy coding!

Greetings

El Bruno

References

#Azure – How to push / export and run a docker python flask web app on Azure

Buy Me A Coffee

Hi !

Here is an important reminder before starting:

Please do not use as a guide to deploy to production environments.

I was reading a lot about Azure Container Registry and other container scenarios. And most of the demos / tutorials are based on .Net Container Web Apps. Which is great, however there are no much information about the same process using Python Container Web Apps. So, I take this as an excuse to learn something new and here I am 😁, Let’s start.

Python Web App using Flask

I started with a simple Python Flask app. For this demo, I will use a post I write about flask and multi threading (see references for the full post). 

# Bruno Capuano
# start a webserver with flask in a thread
# start a different thread +1 a shared var

from flask import Flask                                                         
import threading
import time

iCounter = 0
data = 'foo'
app = Flask(__name__)

def mainSum():
    # increment counter every second
    global iCounter
    while True:
        iCounter = iCounter + 1
        t = time.localtime()
        current_time = time.strftime("%H:%M:%S", t)    
        print(str(f"{current_time} - data {iCounter}"))
        time.sleep(1)

def startWebServer():
     app.run(host='0.0.0.0', port=8100)

@app.route("/getdata")
def main():
    global iCounter
    t = time.localtime()
    current_time = time.strftime("%H:%M:%S", t)    
    return str(f"{current_time} - data {iCounter}")

if __name__ == "__main__":
    stateThread = threading.Thread(target=mainSum)
    stateThread.daemon = True
    stateThread.start()

    webThread = threading.Thread(target=startWebServer)
    webThread.start()

This a resource consuming app, so be careful with this code!

Create a Docker container for the app

So next step was a to create a container for the app. This is my docker file. I added the commands to build and run the image at the bottom.

FROM python:3.7-slim

RUN pip install -U pip
RUN pip install --no-cache-dir flask~=1.1.2

COPY app /app

# Expose the port, default 8100
EXPOSE 8100

# Set the working directory
WORKDIR /app

# Run the flask server for the endpoints
CMD python -u app.py

# Commands to build and run the image
# docker build --pull --rm -f "Dockerfile" -t dockertoazure-webapp-service "."
# docker run -d -p 8100:8100 --privileged --name dockertoazure-webapp-service dockertoazure-webapp-service:latest

So, build and run my docker image and everything works fine !

Double check in Portainer, because I’m not a command line person

docker to azure build and run local ok in portainer

So, here my 1st milestone:

A local python web app using flask, in a docker container ready to be used elsewhere.

Push local Image to Azure

Our next step will be to have a Azure container registry, that will be the place that we will use as repository in Azure to host our images. In the references section the official definition is:

An Azure container registry is a private Docker registry in Azure where you can store and manage private Docker container images and related artifacts. In this quickstart, you create a container registry with the Azure portal. Then, use Docker commands to push a container image into the registry, and finally pull and run the image from your registry.

Quickstart: Create an Azure container registry using the Azure portal

I will assume Azure CLI is installed, so next step will be to login to azure and also login to your container.

# LOGIN TO AZURE
az login
az acr login --name <container name>.azurecr.io

Next step will require to have a tagged image for the Azure Container Registry.

# TAG LOCAL IMAGE
docker tag dockertoazure-webapp-service <container name>.azurecr.io/dockertoazure-webapp-service:v1

And, time to push the local image to the cloud ! 

# PUSH LOCAL IMAGE
docker push <container name>.azurecr.io/dockertoazure-webapp-service:v1

And we have 2 ways to check if the image was successfully pushed. Using the command line

# LIST CURRENT REPOSITORIES
az acr repository list -n <container name>.azurecr.io
docker to azure list azure container registry repositories

We can also use the Azure Portal to check the available repositories

docker to azure azure container repositories list

Running the Docker image in a WebApp

And here comes the tricky part. Once you select a repository, you can see the tags of the image and you can deploy a WebApp directly from there.

docker to azure docker image available options

However, if you haven’t created a WebApp using other steps, you won’t have this option enabled.

Note: My guess is that until you have an Linux usage plan created, the [Run instance] and [Deploy to web app] options will be disabled. In this scenario, you may want to skip next steps and go directly to option 2.

Let’s see both options. From the context menu, from the previous image, create a web app is just a single step

docker to azure create using the context menu

A couple of seconds later we have our App up and running

The 2nd option to create your Web App is creating a new [Web App for Containers]

And here we found a couple of interesting options, like create a web site based on a docker container:

And in the next step, we can choose the image that we uploaded in previous steps

new web app docker image details

And, again, a couple of seconds later, we also have this new App up and running

Conclusion

So, nothing new here. I haven’t found an article or post, that describes this, so I put all the pieces together and dump this lines. Again an important warning

There are much better ways to do this ! Think on a DevOps approach.

I promise to write a follow up on this one, with the step-by-step details on this using GitHub actions. Which is the right way.

Happy coding!

Greetings

El Bruno

Resources

#VisualStudioCode – Create and use Code Snippets in @code, and create custom snippets just for your workspace

Buy Me A Coffee

Hi !

Code Snippets are super useful, and create your custom Code Snippets is super easy in Visual Studio Code. The official Documentation (see references) explains how to create snippets, with the full details of the JSON file, sections, contents and more. However there are some extra tips on top of this information that are super useful.

Snippet Creator

This is an amazing Visual Studio Code Extension. Just search for [snippet creator] on the Extension and install the extension.

visual studio code snippet creator extension

Once the extension is installed, the way to use this is super simple.

  • Select some code
  • CTRL + SHIFT + P, and type [create snippet]
  • Follow the steps, define snippet prefix, and optionally description
  • That’s it!
visual studio code snippet creator add new snippet

I created a custom snippet with the prefix [cameraCalcFPS] and now the snippet is available for any python file !

visual studio code custom code snippet available

In order to check all the custom code snippets

  • CTRL + SHIFT + P
  • Type [configure user snippets]
  • Open the specific one, for this sample python.json

The code snippets file has all the custom created snippets

visual studio code open custom snippets for python

Workspace specific snippets

There is also another specific scenario which I found very interesting:

The need for specific snippets inside a workspace

In order to do this, just create a file [mysnippets.code-snippets] and paste your own snippets here. This snippets will be only available for this workspace!

snippets for a workspace

Happy coding!

Greetings

El Bruno

References

#Event – Resources used on “Getting Started with Machine Learning .NET” for the Central Ohio .NET Developer’s Group (@CONDG)

Buy Me A Coffee

Hi !

Great session yesterday with Matthew and the Central Ohio .NET Developer’s Group (CONDG). And the excuse was to talk about Machine Learning .Net.

As usual, now it’s time for slides and code

Slides

Code

The code used during the session is available here: https://github.com/elbruno/events/tree/main/2020%2009%2024%20CONDG%20MLNet

Session Recording

(I’ll update this block when the recording is available!)

Resources