#VS2019 – ML.NET Model Builder for Object Detection, be careful with file names and VoTT

Buy Me A Coffee

Hi !

This is a sequel part after yesterday post on the new Model Builder Object Detection scenario. I’ve found this error, and it took me some time to figure out the reason.

As usual, an error on the Model Builder, with very few details and the suggestion to check more details on the Azure ML Portal.

On the Azure ML Portal, we can see that 3 experiments failed. Now is time to fig into the logs files and try to figure out what happened.

mlnet 02 azure ml portal error

On the main experiment, after reading the logs, I found out that there is a [File missing for image]. And other file related issues.

mlnet 03 azure portal error file missing

And, after some research (test, fail and test again)

I realized that some of my original file names have a space on the file name.

In example [apple 04.png].

When we create the labeling project using VoTT, the exported project makes some changes into the file names and [apple 04.png] becomes [apple%2004.png]. Here is a part of the exported project:

{
    "name": "mlnet bad file name",
    "securityToken": "mlnet bad file name Token",
    "videoSettings": {
        "frameExtractionRate": 15
    },
    "tags": [
        {
            "name": "apple",
            "color": "#5db300"
        }
    ],
    "id": "aajYgiNHg",
    "activeLearningSettings": {
        "autoDetect": false,
        "predictTag": true,
        "modelPathType": "coco"
    },
    "version": "2.2.0",
    "lastVisitedAssetId": "3d30adce06faf9bf8c6540ec6435ef61",
    "assets": {
        "3d30adce06faf9bf8c6540ec6435ef61": {
            "asset": {
                "format": "png",
                "id": "3d30adce06faf9bf8c6540ec6435ef61",
                "name": "apple%2004.png",
                "path": "file:C:/ML/FreshFood/mlnet%20bad%20file%20names/apple%2004.png",
                "size": {
                    "width": 584,
                    "height": 510
                },
                "state": 2,
                "type": 1
            },
            "regions": [
                {
                    "id": "4BDOAk0Xq",
                    "type": "RECTANGLE",
                    "tags": [
                        "apple"
                    ],
                    "boundingBox": {
                        "height": 505.0856531049251,
                        "width": 579.0878504672897,
                        "left": 1.782359813084112,
                        "top": 1.6381156316916488
                    },

The solution, was to use a tool and remove all the spaces and non-standard characters from my image data set. I also did all the labeling process again, and when I launched again my Object Recognition scenario from Model Builder, everything worked fine!

Happy coding!

Greetings

El Bruno

Resources

#VS2019 – ML.NET Model Builder for Object Detection using #Azure Compute #mlnet #objectdetection

Buy Me A Coffee

Hi !

There is a new scenario available in ML.Net Model Builder for Visual Studio 2019: Object Detection. This scenario is not just image tagging, this scenario allows us to detect objects in an image, and get the specific coordinates and size of the detected objects. As usual, it requires a starting data set with images and labels.

Model Builder Object Detection

This one is super helpful and is also very easy to use. Let’s start with the 1st step

Add a new Machine Learning element in a Visual Studio project, and select Object Detection scenario.

select object detection scenario

This scenario only supports Azure training environment. Lucky me, I already have a Azure ML environment up and running (see references for my previous posts).

Next step is to define the data source for the scenario. In order to do this, we need to define an Image Labeling format file. The suggested tool to do this is VoTT (see references). You need to install the local version for VoTT. As far as I understand, for this Model Builder scenario, it only support VoTT projects using local files.

Define Source Images and Labels

For this post, I created a sample project using Fresh Food labels. Just 4

  • apple
  • apple rotten
  • banana
  • banana rotten

Important: The source connection will define the path where for the source images that will be used for labeling. Target connection defines the project location and also the path for the project exported file.

vott project settins

Once we defined our project properties, we can start defining tags and labeling our objects in the project images. This is a manual process, and it can be tedious.

Important: VoTT support custom detailed areas as the ones you see in the image below. For the Model Builder Scenario, these custom areas will be transformed into standard rectangles.

At any moment, you can save and export your project. This will generate a new folder in the [Target Connection] location named [vott-json-export]. In this folder we can find all the used and labeled images and also the exported JSON file that we will use on Visual Studio.

This is sample content of the exported file:

{
    "name": "MLNet Fresh Food",
    "securityToken": "MLNet Fresh Food Token",
    "videoSettings": {
        "frameExtractionRate": 15
    },
    "tags": [
        {
            "name": "apple_rotten",
            "color": "#5db300"
        },
        {
            "name": "apple",
            "color": "#e81123"
        },
        {
            "name": "banana",
            "color": "#6917aa"
        },
        {
            "name": "banana_rotten",
            "color": "#015cda"
        }
    ],
    "id": "uF6EQo2eX",
    "activeLearningSettings": {
        "autoDetect": false,
        "predictTag": true,
        "modelPathType": "coco"
    },
    "version": "2.2.0",
    "lastVisitedAssetId": "27f36f1bbeb5abc8a505d677931d8e1d",
    "assets": {
        "27f36f1bbeb5abc8a505d677931d8e1d": {
            "asset": {
                "format": "png",
                "id": "27f36f1bbeb5abc8a505d677931d8e1d",
                "name": "bananarotten08.png",
                "path": "file:C:/ML/FreshFood/mlnet/bananarotten08.png",
                "size": {
                    "width": 568,
                    "height": 298
                },
                "state": 2,
                "type": 1
            },
            "regions": [
                {
                    "id": "VVg0KLN6d",
                    "type": "RECTANGLE",
                    "tags": [
                        "banana_rotten"
                    ],
                    "boundingBox": {
                        "height": 158.57173447537474,
                        "width": 470.0359550561798,
                        "left": 21.26011235955056,
                        "top": 24.248394004282655
                    },
                    "points": [
                        {
                            "x": 21.26011235955056,
                            "y": 24.248394004282655
                        },
                        {
                            "x": 491.29606741573036,
                            "y": 24.248394004282655
                        },
                        {
                            "x": 491.29606741573036,
                            "y": 182.8201284796574
                        },
                        {
                            "x": 21.26011235955056,
                            "y": 182.8201284796574
                        }
                    ]
                }
            ],
            "version": "2.2.0"
        },

Back again in Visual Studio, we can select this file as the INPUT Json file for our scenario.

mlnet object detection definicion using the exported file

Train the Model

Everything is ready, let’s train the model, using Azure Compute ;D

The process is similar to the Image Classification scenario, it all start uploading the sources images to a Blob in Azure, and then launch 4 Azure ML Experiments

  • AutoML
  • Preparation
  • Script
  • HyperDrive

Once the process is complete, we can also check output metrics like Precision and Accuracy. This one is not the best model, however I consider this a great starting point for my demo blog.

If you want to know more about Azure ML, this is an great opportunity to play around with these experiments. The Script Experiment (Run 16 in the image) has a lot to dig, like the complete output log, some training python files, the generated ONNX model, and more.

mlnet 11 object detection azure track 01

Test and consume the generated Model

Back to Visual Studio we can check the model metrics and also the Best Model (I have so many questions here!).

And we can test the generated model. As you can see in the next images, using a threshold with value 50, the results are good. Still not detecting the rotten apple with the good ones, so I maybe need to figure out some strategies to improve my model.

sample test detecting apples
sample test detecting bananas

Remember that all these features are still in preview, however they are an amazing starting point and tool to use and learn.

Happy coding!

Greetings

El Bruno

Resources

#VS2019 – How to fix the [obj\project.assets.json’ not found] when building a #NetCore project

Buy Me A Coffee

Hi!

Quick post today. This one is to save me 30 minutes of internet search next time I face this error.

Context:

New .Net Core Console App.

When I build the console app for the 1st time I got this error.

  • Error: NETSDK1004
  • Assets file ‘<Path> \EventConsoleApp01\obj\project.assets.json’ not found.
  • Run a NuGet package restore to generate this file.
  • C:\Program Files\dotnet\sdk\3.0.100\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets
visual studio 2019 new dotnet core console error on 1st build, cant find assets json.jpg

It’s a very silly error. However, if you are old school, the standard methods to fix this won’t work.

So the right way to fix this is to open the NuGet Package Manager Console [Tools // Nuget Package Manager / Package Manager Console] and type the command

dotnet restore
visual studio 2019 dotnet restore on nuget package manager console.jpg

Then rebuild the project, and that’s it! The new build will restore all the package, and also install everything necessary for a console app to run

visual studio 2019 project after dotnet restore.jpg

Happy coding!

Greetings @ Toronto

El Bruno

#Event – Resources used on the Visual Studio 2019 Launch Party event on [Machine Learning.Net] #VS2019 #MLNET

Hi!

Another post-event post, this time with a big thanks to the Mississauga .Net User Group who organized the Visual Studio 2019 Launch Party.

I was part of the team sharing some demos, and I mostly speak about Machine Learning.Net.

As usual, now it’s time to share slides, code and more.

Source Code in GitHub https://github.com/elbruno/events/tree/master/2019%2004%2027%20GAB%20MLNet

And some Machine Learning.Net resources:

Resources

Happy coding!

Greetings @ Toronto

El Bruno

#MLNET – Testing Machine Learning Model Builder preview. It’s so cool !

Hi !

Last week Machine Learning.Net 1.0 was officially announced during Build 2019, and the ML.Net team also announced a set of ML tools related to ML.Net.

One of the most interesting ones is Machine Learning Model Builder. You can get more information about Model Builder in the official website.

ML.NET Model Builder provides an easy to understand visual interface to build, train, and deploy custom machine learning models. Prior machine learning expertise is not required. Model Builder supports AutoML, which automatically explores different machine learning algorithms and settings to help you find the one that best suits your scenario.

Machine Learning Model Builder

The tool is on Preview, but it’s still an amazing one to play around with ML. So I decided to give it a try with my small data set of kids, the one I use on the Machine Learning.Net demos.

The structure of my CSV file is very simple with just 3 columns: Age, Gender and Label.

However the first time I run the scenario I found the following error.

Inferring Columns ...
Creating Data loader ...
Loading data ...
Exploring multiple ML algorithms and settings to find you the best model for ML task: regression
For further learning check: https://aka.ms/mlnet-cli
|     Trainer                             RSquared Absolute-loss Squared-loss RMS-loss  Duration #Iteration      |
[Source=AutoML, Kind=Trace] Channel started
Exception occured while exploring pipelines:
Provided label column 'Label' was of type String, but only type Single is allowed.
System.ArgumentException: Provided label column 'Label' was of type String, but only type Single is allowed.
   at Microsoft.ML.CLI.Program.<>c__DisplayClass1_0.<Main>b__0(NewCommandSettings options)
   at Microsoft.ML.CLI.CodeGenerator.CodeGenerationHelper.GenerateCode()
Please see the log file for more info.
Exiting ...

Which makes a lot of sense, my Label column is a String and the Model Builder expects a Single data type. So, I updated my data file replacing the labels with numbers and I was ready for a 2nd test.

This time the training process started fine, however I noticed that using just a small training dataset didn’t trigger any comparing between different algorithms. So I created a much bigger training dataset, and now I got the training process up and running.

At the end the results are the ones below. And it’s very interesting. I do most of my demos using a MultiClass SDCA trainer and AutoML suggest me to use a LightGBM trainer. This will be part of my Machine Learning.Net speech for sure in the future.

You can download the Visual Studio extension from https://dotnet.microsoft.com/apps/machinelearning-ai/ml-dotnet/model-builder and remember that we can talk about this on the Visual Studio 2019 event with the Mississauga .Net User Group in a couple of weeks!

Happy Coding!

Greetings @ Toronto

El Bruno

#VS2019 – Lanzamiento de Visual Studio 2019 el 2 de Abril 9:00am PT

 

visual studio 2019 launch event

Buenas !

Es tiempo de preparar el 2do monitor para el próximo 2 de Abril, cuando se presenta la nueva versión de Visual Studio 2019 !

Grandes speakers como Scott Hanselman y otros, estaran en una sesion de 7 horas presentando la ultima version de Visual Studio. El evento oficial y los detalles se pueden encontrar en https://visualstudio.microsoft.com/vs2019-launch/ , es tiempo de agregar el reminder y disfrutar el nuevo IDE !

Saludos @ Burlington

El Bruno

#VS2019 – Visual Studio Launch event on the next April 2nd, 9:00am PT

 

visual studio 2019 launch event

Hi !

You may want to get your 2nd monitor ready on the next April 2nd, 9:00am PT, because is the official launch of Visual Studio 2019 !

Scott Hanselman and other amazing set of speakers will host a 7 hour session introducing the latest version of Visual Studio. So, go to https://visualstudio.microsoft.com/vs2019-launch/ , add the reminder to your calendar and get ready also for some local launch events !

Greetings @ Burlington

El Bruno

#VS2019 – Working with #Python Environments now is so cool !

Hi !

Let me start with IANAPU [I am not a Python user], and that’s maybe why, when I need to work and understand what is in my current environment it took me a lot of time to get and deploy the correct tools and the right packages to work with. I’m not a fan of console environments, and if we add this to the 90% of the Python work, that maybe the main problem.

Visual Studio Code is amazing environment to work with Python. So far, is good enough for me and my machine learning devs. I’ll became a brand-new Mac user in the next couple of days, so I hope that everything I’ve learned is the same on Mac.

I was trying some of the new features in Visual Studio 2019 Preview 2, and the options to manage Python environments blown my mind.

Let’s start with a single Python Application in the solution explorer. With 2 clicks I know what I have in my environment: current python version, packages and more.

01 python environment in vs2019 preview 2 solution explorer

With 2 clicks I can add a new environment, where I can choose the Python version, a base Anaconda environment, and tons of options more.

02 new python environment

If we decided to create an Anaconda working environment, we can load our working packages from a file, or from a cool drop-down list.

03 new python environment based on conda

Adding and upgrading packages is done with the usual Visual Studio integrated experience

04 install new packages05 upgrade packages

Of course, we have the option to review the Console logs, to understand the background processes

Virtual environment is being created at '...\Python\PythonApplication1\envBlog01'
Virtual environment was successfully created at '...\Python\PythonApplication1\envBlog01'
----- Creating 'envBlogConda01' -----
Solving environment: ...working... done
## Package Plan ##
  environment location: d:\ProgramData\Anaconda3\envs\envBlogConda01
==> WARNING: A newer version of conda exists. <==
Preparing transaction: ...working... done
  current version: 4.5.4
  latest version: 4.6.1
Please update conda by running
    $ conda update -n base conda
Verifying transaction: ...working... done
Executing transaction: ...working... done
#
# To activate this environment, use:
# > activate envBlogConda01
#
# To deactivate an active environment, use:
# > deactivate
#
# * for power-users using bash, you must source
#
----- Successfully created 'envBlogConda01' -----


---- Installing 'numpy' -----
Collecting numpy
  Downloading https://files.pythonhosted.org/packages/31/7e/8905636f7e4f9b9d7078aa0e701500634f832f145855a11beb098d3b0fb1/numpy-1.16.0-cp36-cp36m-win_amd64.whl (11.9MB)
Installing collected packages: numpy
Successfully installed numpy-1.16.0
You are using pip version 10.0.1, however version 19.0.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
----- Successfully installed 'numpy' -----


---- Installing 'pip' -----
Collecting pip==19.0.1
  Downloading https://files.pythonhosted.org/packages/46/dc/7fd5df840efb3e56c8b4f768793a237ec4ee59891959d6a215d63f727023/pip-19.0.1-py2.py3-none-any.whl (1.4MB)
Installing collected packages: pip
  Found existing installation: pip 10.0.1
    Uninstalling pip-10.0.1:
      Successfully uninstalled pip-10.0.1
Successfully installed pip-19.0.1
----- Successfully installed 'pip' -----

Happy Coding!

Greetings @ Toronto

El Bruno

References