#Anaconda – My steps to install a virtual environment with #TensorFlow, #Keras and more

Hi!

So today post is not a post, just a selfish reminder of the steps I do when I setup a new dev machine

  • Install Anaconda (see references). I use the default settings, and important: I don’t add Anaconda to Windows PATH.
  • Open Anaconda command prompt as administrator
open anaconda as administrator

Need to be open as Admin in order to install updates

  • Install updates with the command
conda update conda 
conda update –all
  • Create a new development environment named “tfEnv” with tensorflow. Activate the environment
conda create -n tfenv tensorflow 
conda activate tfenv
  • The command to install keras is
pip install
keras

However, if it doesn’t work, I install keras with the following packages

pip install matplotlib 
pip install pillow
pip install tensorflow==1.14
conda install mingw libpython
pip install git+git://github.com/Theano/Theano.git
pip install git+git://github.com/fchollet/keras.git
  • Finally, install Jupyter notebook kernel and create a new kernel for the current virtual environment
pip install ipykernel 
ipython kernel install --user --name=tfEnv
  • There seems to be an issue to install OpenCV using pip with the command
pip install
opencv-python

So, I Install the OpenCV nonofficial package. 1st I download a compatible package from

https://www.lfd.uci.edu/~gohlke/pythonlibs/#pyopencl

Install with

pip install
c:\temp\opencv_python-4.1.1-cp36-cp36m-win_amd64.whl

Happy coding!

Greetings @ Toronto

El Bruno

References

Advertisements

#Python – Can’t install TensorFlow on Anaconda, maybe is the Visual Studio distribution

Hi!

This is the 2nd time I get a weird error when I install TensorFlow in my Anaconda distribution. And this is the 2nd time I realize that I’m using the Anaconda version that is preinstalled with Visual Studio. I’m not sure if the spaces in the path affects the creation of environments or it’s something else, however my current and big and amazing solution is:

  • Uninstall Anaconda
  • Install Anaconda again

And then, follow the simple commands in the official Anaconda and TensorFlow doc (see references)

conda create -n tensorflow_env tensorflow
conda activate tensorflow_env

Once tensorflow is installed, I usually test this in python

> Python 
import tensorflow as tf
print(tf.__version__)

Note: please ignore the typos!

anaconda start python and test anaconda version

Now TensorFlow is installed and it’s time to move forward with a new development environment.

Happy Coding!

Greetings @ Burlington

El Bruno

References

#Event – Webinar: Getting Started with ML.NET and Windows Machine Learning, Today with the GLUG .Net User Group

giphy-downsized

Hi !

Another virtual event about Windows ML and Machine Learning .Net, and this time with my good friends from Okemos, MI. The event will be

Getting Started with ML.NET and Windows Machine Learning

One more time, we have new features and improvements on the platform since the previous event, so it’s time for me to review and prepare new content. In example, use TensorFlow and ML.Net !!!

Description  

Machine Learning has moved out of the lab and into production systems. Understanding how to work with this technology is one of the essential skills for developers today. In this session, you will learn the basics of machine learning, how to use existing models and services in your apps, and how to get started with creating your own simple models.

In other words, if you are a .NET developer, this session is for you! We will cover the basics of ML.NET, a complete machine learning framework to work with C#, F# or any other .NET Core language.

Online Register https://www.meetup.com/GLUGnet/events/qkmgpkyzdbcc/

Happy Coding!

Greetings @ Burlington

El Bruno

#Docker – Container muere inmediatamente después de ser iniciado en #RaspberryPi. Obviamente, era un problema de dependencias de #TensorFlow

Buenas !

La creación de imágenes en Docker es un proceso divertido. Cuando cree la imagen de CustomVision.ai para ser ejecutada en Docker en Raspberry Pi, me encontré con unos errores interesantes, así que aprovechare este post para escribir sobre los mismos.

La compilación de cada imagen suele tardar alrededor de unos 15 minutos. Ver que la misma compila correctamente es un momento de alegría, que se veía arruinado cuando al momento intentar iniciarla, el container se destruía automáticamente. El comando con el que iniciaba el mismo es el siguiente

sudo docker run -p 127.0.0.1:8080:80 -d <IMAGE ID>

Estuve leyendo mucho y encontré varias opciones para intentar comprender que sucede. Al final opte por intentar analizar los eventos en tiempo real que Docker publica con el comando

sudo docker events&

01 docker events

En la consola podemos ver un buffer lleno de eventos de Docker. Después de varios intentos con mi imagen, me encontré con mensajes similares a los siguientes.

2019-02-12T07:34:46.195722938-05:00 container start cdcdcc410518db46e09967412bd583c33cff6f4e8eee0f10e8baeec860f9c9a2 (image=295, io.balena.architecture=armv7hf, io.balena.device-type=raspberry-pi2, io.balena.qemu.version=3.0.0+resin-arm, name=musing_zhukovsky)

2019-02-12T07:34:46.195722938-05:00 container die cdcdcc410518db46e09967412bd583c33cff6f4e8eee0f10e8baeec860f9c9a2 (image=295, io.balena.architecture=armv7hf, io.balena.device-type=raspberry-pi2, io.balena.qemu.version=3.0.0+resin-arm, name=musing_zhukovsky)

Es fácil interpretar que después de la fecha y hora del evento, la descripciones “container start” y “container die”, describen el comportamiento que estoy analizando. Estaba un poco mas cerca.

Sin embargo, el evento no presenta mucha información sobre el error. Es por esto, que utilizando el <LOG ID> podemos obtener mas información con el siguiente comando.

sudo docker logs cdcdcc410518db46e09967412bd583c33cff6f4e8eee0f10e8baeec860f9c9a2

02 docker event details

Esto ya es mucho mejor! Ya puedo ver un archivo de código fuente en python y ademas el error, que en este caso, se da al intentar importar el modulo Pillow. Ahora ya puedo abrir python y todo cobra sentido.

03 app python details

Pues bien, ahora solo queda ver las dependencias y herramientas que necesita TensorFlow para instalar las mismas en el orden correcto antes de compilar la imagen.

Happy coding!

Greetings @ Toronto

El Bruno

References

My Posts

  1. Object recognition with Custom Vision and ONNX in Windows applications using WinML
  2. Object recognition with Custom Vision and ONNX in Windows applications using WinML
  3. Object recognition with Custom Vision and ONNX in Windows applications using Windows ML, drawing frames
  4. Object recognition with Custom Vision and ONNX in Windows applications using Windows ML, calculate FPS
  5. Can’t install Docker on Windows 10 Home, need Pro or Enterprise
  6. Running a Custom Vision project in a local Docker Container
  7. Analyzing images in a Console App using a Custom Vision project in a Docker Container
  8. Analyzing images using PostMan from a Custom Vision project hosted in a Docker Container
  9. Building the CustomVision.ai project in Docker in a RaspberryPi

Windows 10 and YOLOV2 for Object Detection Series

#Docker – Container dies immediately upon successful start in a #RaspberryPi. Of course, it’s all about #TensorFlow dependencies

Hi !

Creating Docker images is a fun process. When I created the CustomVision.ai custom image to be executed in my Raspberry Pi, I faced a couple of errors, so now it’s time to save / share some lessons learned.

One of the most frustrating steps was after my 15 min wait time to build an image to find that the image was successfully built, however it dies after I run the image with a command like this one

sudo docker run -p 127.0.0.1:8080:80 -d <IMAGE ID>

There are a couple of options to understand what’s happen here. I decided to launch and trace the live events from Docker with the command

sudo docker events&

01 docker events

This windows is a full buffer of Docker events, after a while I detected that after I tried to start my docker image I got 2 messages similar to this one

2019-02-12T07:34:46.195722938-05:00 container start cdcdcc410518db46e09967412bd583c33cff6f4e8eee0f10e8baeec860f9c9a2 (image=295, io.balena.architecture=armv7hf, io.balena.device-type=raspberry-pi2, io.balena.qemu.version=3.0.0+resin-arm, name=musing_zhukovsky)

2019-02-12T07:34:46.195722938-05:00 container die cdcdcc410518db46e09967412bd583c33cff6f4e8eee0f10e8baeec860f9c9a2 (image=295, io.balena.architecture=armv7hf, io.balena.device-type=raspberry-pi2, io.balena.qemu.version=3.0.0+resin-arm, name=musing_zhukovsky)

As you probably detected (much faster than me!) the events were container start and container die. But the docker events does not display much more information with details of the event.

What we can use is the <LOG ID> included in the event line. And with the following command we can get more details of the event.

sudo docker logs cdcdcc410518db46e09967412bd583c33cff6f4e8eee0f10e8baeec860f9c9a2

02 docker event details

This is much better! Now I know that we can’t import a Python module named PIL on the file [app.py], in the line 10. When I open the file, it all makes sense.

03 app python details

So now it’s time to check the dependencies and tools required to use TensorFlow in a Raspberry Pi. I’ll write more about this tomorrow 😀

Happy coding!

Greetings @ Toronto

El Bruno

References

My Posts

  1. Object recognition with Custom Vision and ONNX in Windows applications using WinML
  2. Object recognition with Custom Vision and ONNX in Windows applications using WinML
  3. Object recognition with Custom Vision and ONNX in Windows applications using Windows ML, drawing frames
  4. Object recognition with Custom Vision and ONNX in Windows applications using Windows ML, calculate FPS
  5. Can’t install Docker on Windows 10 Home, need Pro or Enterprise
  6. Running a Custom Vision project in a local Docker Container
  7. Analyzing images in a Console App using a Custom Vision project in a Docker Container
  8. Analyzing images using PostMan from a Custom Vision project hosted in a Docker Container
  9. Building the CustomVision.ai project in Docker in a RaspberryPi

Windows 10 and YOLOV2 for Object Detection Series

#MLNet – Adding NuGet Packages in Preview mode from MyGet, ie: Microsoft.ML-0.6.0 Version

Hi!

A few days ago I wrote about the new Machine Learning.Net version 0,5. Yesterday when .NetConf started, the release was made official with the following post from Cesar de la Torre

Announcing ML.NET 0.5

Now, a detail that is a little Deep in the documentation is the way to access packages in preview mode that are used in the following repositories Machine Learning  and Machine Learning Samples

Well, when we open and try to compile one of the projects we can see that in the references of this are being used Packages in preview state

01 mlnet 0.60 preview solution explorer

The problem we have is that, when built, these packages are not available in nuget.org

Unable to find package Microsoft.ML with version (>= 0.6.0-preview-26912-3)

  – Found 5 version(s) in nuget.org [ Nearest version: 0.5.0 ]

 TensorFlowMLNETInceptionv3ModelScoring  

02 mlnet 060 preview error on build

The solution for this is to add the necessary packages in manual mode from MyGet.

PM> Install-Package Microsoft.ML -Version 0.6.0-preview-26913-1 -Source https://dotnet.myget.org/F/dotnet-core/api/v3/index.json

or set up a new NuGet source in Visual Studio, with the following information

03 MyGet ML.Net Sources NuGet

In this way, when a NuGet Restore action is performed for example, when we build a project, and we will have all the necessary packages available to work.

04 mlnet 0.60 preview solution explorer build OK

Happy Coding!

Greetings @ Toronto

El Bruno

References

My Posts

#MLNet – Agregando paquetes en modo Preview desde MyGet, por ejemplo Microsoft.ML -Version 0.6.0

Buenas!

Hace unos días hable de la nueva version 0.5 de Machine Learning.Net. Ayer cuando comenzó .NetConf, se hizo oficial el lanzamiento de esta version con el siguiente post de Cesar de la Torre

Announcing ML.NET 0.5

Ahora bien, un detalle que está un poco Deep en la documentación es la forma de acceder a paquetes en modo Preview que se utilizan en los repositorios de Machine Learning  y en Machine Learning Samples

Pues bien, cuando abrimos e intentamos compilar uno de los proyectos podemos ver que en las referencias de este se están utilizando packages en modo Preview

01 mlnet 0.60 preview solution explorer

El problema que tenemos es que, al compilar, estos paquetes no están disponibles en nuget.org

Unable to find package Microsoft.ML with version (>= 0.6.0-preview-26912-3)

  – Found 5 version(s) in nuget.org [ Nearest version: 0.5.0 ]

 TensorFlowMLNETInceptionv3ModelScoring  

02 mlnet 060 preview error on build

La solución para esto es agregar los paquetes necesarios en modo manual desde MyGet.

PM> Install-Package Microsoft.ML -Version 0.6.0-preview-26913-1 -Source https://dotnet.myget.org/F/dotnet-core/api/v3/index.json

O configurar un nuevo source de NuGet en Visual Studio, con la siguiente información

Name: Microsoft.ML

Source: https://dotnet.myget.org/F/dotnet-core/api/v3/index.json

03 MyGet ML.Net Sources NuGet

De esta manera, cuando se realice una acción de NuGet restore, por ejemplo, al compilar, tendremos disponibles todos los paquetes necesarios para trabajar.

04 mlnet 0.60 preview solution explorer build OK

Happy Coding!

Saludos @ Toronto

El Bruno

References

My Posts

#Event – Webinar: Getting Started with ML.NET and Windows Machine Learning, Dec 11 with the Tech Valley .Net User Group

giphy-downsized

Hi !

Another virtual event about Machine Learning .Net, and this time with my good friends from Albany, NY. The event will be

Getting Started with ML.NET and Windows Machine Learning

One more time, we have new features and improvements on the platform since the previous event, so it’s time for me to review and prepare new content. In example, use TensorFlow and ML.Net !!!

Description  

Machine Learning has moved out of the lab and into production systems. Understanding how to work with this technology is one of the essential skills for developers today. In this session, you will learn the basics of machine learning, how to use existing models and services in your apps, and how to get started with creating your own simple models.

In other words, if you are a .NET developer, this session is for you! We will cover the basics of ML.NET, a complete machine learning framework to work with C#, F# or any other .NET Core language.

Online Register https://www.meetup.com/TechValleyNETUserGroup/events/254466580/

Happy Coding!

Greetings @ Burlington

El Bruno

#MLNET – ML.Net 0.5 initial support for #TensorFlow

Hi!

New version of Machine Learning.Net and in this version, we have the ability to use TensorFlow frozen models in ML.Net.

During the process of creating a pipeline, we can now use TensorFlow frozen models models and use them to train a model and make predictions. In a console application, at the time of adding the ML.Net packages we can see a new series of packages to work with TensorFlow

01 ms ml tensor flow

The following code is the best way to understand how a pipeline with a TF model works. While I’m building my Pipeline, I use a couple of trainers to load and modify images; and then on line 40 the TF model is added to the pipeline

At this time the file [Cifar_model/FrozenModel. PB] is not part of the ML.Net repository. If you want to try the code, you can download a trial version from https://github.com/deeplearning4j/dl4j-test-resources/tree/master/src/main/resources/tf_graphs/examples/yolov2_608x608

02 mlnet cifar pb

Happy Coding!

Greetings @ Toronto

El Bruno

References

My Posts

#MLNET – ML.Net 0.5 con soporte para #TensorFlow

Buenas!

Nueva version de Machine Learning .Net y esta vez la novedad principal es la capacidad de utilizar modelos de TensorFlow en ML.Net.

Durante el proceso de creación de una Pipeline, ahora podemos utilizar modelos frezados de TF y utilizar los mismos para entrenar un modelo y realizar predicciones. En una aplicación de Consola, al momento de agregar los paquetes de ML.Net podemos ver una serie nueva de paquetes para trabajar con TensorFlow

01 ms ml tensor flow

El siguiente código es la mejor forma de entender como funciona una Pipeline con un modelo de TF. En la definición de la misma se utilizan un par de trainers para cargar y modificar imágenes; y luego en la línea 40 se carga el modelo para su utilización.

En este momento el archivo [cifar_model/frozen_model.pb] no es parte del repositorio de ML.Net. Si quieres probar el código, puedes descargar una version de prueba desde https://github.com/deeplearning4j/dl4j-test-resources/tree/master/src/main/resources/tf_graphs/examples/yolov2_608x608

02 mlnet cifar pb

Happy Coding!

Saludos @ Toronto

El Bruno

References

My Posts