#Event – Resources used with @ivanatilca during the “Lessons Learned creating a multiplatform AI project for Azure Kinect and Hololens 2” for the Global XR Talks

Buy Me A Coffee

Hi !

We had an amazing time last week with Ivana at the Global XR Talks, sharing some of our lessons learned creating an Mixed Reality app to work in Hololens 2 and Azure Kinect.

As usual, now it’s time for slides and code

Slides

Code

The main scripts for Camera and Cognitive Services interaction are available here https://github.com/elbruno/events/tree/main/20200806%20Global%20XR%20HL2%20to%20Azure%20Kinect%20Lessons%20Learned

Session Recording

Resources

#Unity3D – Making a CustomVision.ai HTTP Post call to have a better #MRTK experience with #CognitiveServices @ivanatilca

Buy Me A Coffee

Hi !

Quick post today, with mostly sample code. And, it’s all about a scenario that we faced with Ivana a couple of days ago while we were using MRTK and we were trying to use some Cognitive Services.

As of today, not all the services in Cognitive Services are supported and have official Unity3D support. At the end, it’s not a problem, we can just make an HTTP Post call, and that’s it. However, this is not as easy as is supposed to be.

So, after facing some issues with the System.Net.HttpClient library, I decided to use UnityWebRequest. This library main objective is to work with HTTP Forms, however we can manage to send an image with a sample like this:

string DetectImage(byte[] image, string imageUrl)
{
string body = string.Empty;
using (var request = UnityWebRequest.Post(imageUrl, ""))
{
request.SetRequestHeader("Content-Type", "application/octet-stream");
request.uploadHandler = new UploadHandlerRaw(image);
request.SendWebRequest();
while (request.isDone == false)
{
var wfs = new WaitForSeconds(1);
}
if (request.isNetworkError || request.isHttpError)
{
Debug.Log(request.error);
}
else
{
body = request.downloadHandler.text;
}
}
return body;
}

As we can see in the previous post, there is no async / await support here. So I added a couple of ugly lines of code to support this. We can improve this to have a timeout or delay applied here. As for this sample, this works great.

As a bonus, you can watch the full presentation in the Global XR YouTube channel here

Happy coding!

Greetings

El Bruno

References

#CognitiveServices – #AnomalyDetector Service running on Containers (some #docker time!)

Buy Me A Coffee

Hi!

I was chatting about the new features and the use cases supported using the brand new (and still in Preview) Cognitive Services Anomaly Detector and I realized that we can use the service with local containers (instead of the cloud service), but there is something different in this Service.

anomaly detector sample data chart

Why to use containers?

Azure Cognitive Services allow developers to easily add cognitive features—such as object detection, vision recognition, and language understanding—into their applications without having direct AI or data science skills or knowledge. Containerization is an approach to software distribution in which an application or service is packaged so that it can be deployed in a container host with little or no modification

I first will recommend reading the article [Getting started with Azure Cognitive Services in containers] and then I will remark 2 main advantages of using containers

  • You can get a better control of your internet usage. I mean, all the HTTP calls will be performed in your intranet
  • You will have a better data governance.

Related to the 2nd bullet, I’ll quote the official launch sample

For example, let’s look at a typical hospital system that works with patients. After many years of taking care of patients, they have numerous doctor’s notes, intake records, or other files that they want to process and derive insights about key trends. Using Cognitive Services containers, they can process all of these files, index millions of documents and find commonalities, and improve the patient experience while keeping the data in-house. Another example would be a large manufacturing plant that has limited connectivity where they want to track assets on the edge using remote sensors and cameras, using AI to predict maintenance needs.

As a downside point, you need to manage your own docker instances, which may require some extra work. However, it’s likely that you are already doing this, so this is “just more containers”

How about Anomaly Detector Containers?

And, after this small introduction, and starting to use Anomaly Detector Service I found that if you want to use Anomaly Detector with containers, you must first complete and submit the Anomaly Detector Container Request form to request access to the container.

Anomaly Detector Container Request form

https://forms.office.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbRxSkyhztUNZCtaivu8nmhd1UOTdDRzlPSDRBOEdZVFcxR0lTUUlBRzY1OS4u

The form requests information about you, your company, and the user scenario for which you’ll use the container. After you’ve submitted the form, the Azure Cognitive Services team reviews it to ensure that you meet the criteria for access to the private container registry.

Once you submitted your request and been approved, you can start to use the Anomaly Detector service locally with containers. And just a reminder about this: even if your data requests are not going to hit and Azure HTPP Endpoint, your containers need to be able to connect to Azure.

The container needs the billing argument values to run. These values allow the container to connect to the billing endpoint. The container reports usage about every 10 to 15 minutes. If the container doesn’t connect to Azure within the allowed time window, the container continues to run but doesn’t serve queries until the billing endpoint is restored. The connection is attempted 10 times at the same time interval of 10 to 15 minutes. If it can’t connect to the billing endpoint within the 10 tries, the container stops running.

Happy Coding!

Greetings @ Toronto

El Bruno

References

#CognitiveServices – Easy lines to convert CSV to JSON to be used on the #AnomalyDetector service

Buy Me A Coffee

Hi!

After the event “Building an Anomaly Detector System with a few or no lines of code” at MsftReactor, some people asked for the 2 lines that I used to convert a CSV file to JSON, to be used with Cognitive Services Anomaly Detector, so here they are.

static string CsvToJson(string csvPath, string granularity = "daily", bool hasHeaders = true)
{
char[] fieldSeparator = { ',' };
var lines = System.IO.File.ReadAllLines(csvPath);
// remove header
if (hasHeaders)
lines = lines.Skip(1).ToArray();
// build series
var arraySeries = new JArray();
foreach (var line in lines)
{
if (string.IsNullOrEmpty(line)) continue;
var fields = line.Split(fieldSeparator);
var jsonSerie = new JObject
{
["timestamp"] = fields[0],
["value"] = fields[1]
};
arraySeries.Add(jsonSerie);
}
var jobjectMain = new JObject
{
["granularity"] = granularity,
["series"] = arraySeries,
};
var jsonComplete = new JArray {jobjectMain};
return jsonComplete.ToString();
}

Important: you need Newtonsoft.Json to build the json content.

The input CSV is part of the Machine Learning.Net sample data, and has this sample content:

Month,ProductSales
1-Jan,271
2-Jan,150.9
3-Jan,188.1
...

And as a bonus, the full console project can be downloaded from here.

https://github.com/elbruno/Blog/tree/master/20191125%20CSV%20to%20JSON%20for%20Anomaly%20Detector

This project also have a second function which creates the JSON content using simple string, without the need of Newtonsoft.Json.

Happy coding!

Greetings @ Burlingon

El Bruno

References

#Event – Resources used during my session “Building an Anomaly Detector System with a few or no lines of code” @MsftReactor

Buy Me A Coffee

Hi!

My 1st session at Microsoft Reactor and it was amazing. We had engaging conversations around Machine Learning and Anomaly Detection, and as I promised here are the resources used.

Building an Anomaly Detector System with a few or no lines of code

More Information https://www.microsoftevents.com/profile/form/index.cfm?PKformID=0x8330114abcd

Slides

Source Code

https://github.com/elbruno/events/tree/master/2019%2011%2021%20Anomaly%20Detections%20Reactor

Reference Links

General

Machine Learning.Net

Cognitive Services Anomaly Detector

Azure Machine Learning

Happy coding!

Greetings @ Toronto

El Bruno

#CognitiveServices – Nuevo Servicio: Anomaly Detector

Buenas!

Durante estos días, estoy totalmente centrado en un increíble evento en Avanade Canada: Canada !nnovate Event, y casi me pierdo el lanzamiento de este nuevo servicio, en la familia Cognitive Services: Anomaly Detector.

Este fin de semana tenia pensado terminar mi sesión para la próxima Global AI Night (link), sin embargo, me parece que dedicaré un poco de tiempo para poder hablar del mismo en el próximo evento.

Después de pasar hoy en día leyendo sobre el servicio, parece que funciona con series de tiempo de datos y el uso de modelos específicos, se centran en las anomalías de los datos de la serie y .. ¡Magia! Creo que la documentación oficial lo explica mejor:

The Anomaly Detector API enables you to monitor and detect abnormalities in your time series data with machine learning. The Anomaly Detector API adapts by automatically identifying and applying the best-fitting models to your data, regardless of industry, scenario, or data volume. Using your time series data, the API determines boundaries for anomaly detection, expected values, and which data points are anomalies.

Detect pattern changes in service requests

More information: https://azure.microsoft.com/en-us/services/cognitive-services/anomaly-detector/

Saludos @ Burlington

El Bruno

#CognitiveServices – New service: Anomaly Detector

Hi !

So, I’m fully focused on have an amazing Canadian !nnovate Event in Avanade and I almost miss this new service, in the Cognitive Services Family: Anomaly Detector.

I was planning to run my 30K Around the Bay this weekend and also finish my session for the next Global AI Night (link), however I may play around with this service and add a quick reference on the next event.

After spending sometime today reading about the service, it seems that it works with time series of data and using specific models, it focus on anomalies of data on the series and .. magic! I think the official documentation explains this better:

The Anomaly Detector API enables you to monitor and detect abnormalities in your time series data with machine learning. The Anomaly Detector API adapts by automatically identifying and applying the best-fitting models to your data, regardless of industry, scenario, or data volume. Using your time series data, the API determines boundaries for anomaly detection, expected values, and which data points are anomalies.

Detect pattern changes in service requests

More information: https://azure.microsoft.com/en-us/services/cognitive-services/anomaly-detector/

Greetings @ Toronto

El Bruno

#Azure – Single Key for all Services in #CognitiveServices, that’s cool :D

Hi !

Quick Friday post. And an amazing one, because now we have the chance to create a single Key to be use around a bunch of Cognitive Services. That’s mean we don’t need to remember and store different keys for LUIS, Face Emotion, and more. A single key will cover most of these scenarios with 3 simple steps

 

More information in What’s New? A Single Key for Cognitive Services on Channel 9 by Noelle LaCharite

Happy coding!

Greetings @ Burlington

El Bruno

#Event – I’ll be at @CodeMash on Ohio in 17 days sharing some #AI and #CustomVision experiences

codemash-logo

Hi!

I’m going to be part of one of the most amazing developer events in NA: CodeMash (http://www.codemash.org/). It will be my first time in Ohio, and also it will be an amazing opportunity to network and have some face-to-face chats with some amazing people. (Just look at the Speaker List)

I was also lucky to host a session around Artificial Intelligence with Cognitive Services at Enterprise Level. The latest announcements of containers and Cognitive Services are ready on time for this!

How a PoC at home can scale to Enterprise Level using Custom Vision APIs

It all started with a DIY project to use Computer Vision for security cameras at home. A custom Machine Learning model is the core component used to analyze pictures to detect people, animals and more in a house environment. The AI processing is performed at the edge, in dedicated hardware and the collected information is stored in the cloud.
The same idea can be applied to several CCTV scenarios, like parking lots, train stations, malls and more. However, moving this into enterprise scale brings a set of challenges, which are going to be described and explained in this session.

Session List: http://www.codemash.org/session-list/

Happy coding and see you there!

Greetings @ Toronto

El Bruno

#AI – Some news in Cognitive Services presented at #MSBuild 2018

Hi!

Again, it’s time to write about some topics what has most caught my attention in the news presented during Microsoft Build 2018. In this case I will only comment on some news related to Vision and Speech.

Vision

  • Computer Vision, now supports Object Detection. We have the ability to detect objects in an image. I have to see more in depth that we can both exploit this capacity in Custom Vision.
  • Custom Vision, new formats to export models. Until now we had the ability to export Custom Vision models to CoreML and TensorFlow.
    Now we have 2 new options that really are impressive

    • Export to ONNX. About this I wrote about it. Now we can use these models natively as part of our UWP Apps in Windows 10.
    • Export to Docker File. Especially designed for mixed scenarios with Azure Functions and Azure IOT Edge

I1

Speech

The first thing to comment is a big but very necessary change.

We now have a single service that handles: Speech to Text, Text to Speech and Speech Intent Recognition.

The 2nd point to note is that we now have the ability to Create our own Voice Models. This means that we could create Alexa or Cortana style assistants using our own voice. Ideal to give to your partner, your mother or your worst enemy.

And with this I put pause for today. Happy coding!

Greetings @ Toronto

El Bruno