QnA Maker managed: now in public preview

If you try to create a QnA Maker today, you will notice the following option. “Managed (preview)”. But what is this about?

What’s new or which new features are introduced:

  1. Deep learnt ranker with enhanced relevance of results across all supported languages, based on Turing multilingual language model (T-ULRv2). Computation has been moved out of the user subscription, so there is no dependency on the customers for scaling and availability management. This allowed us to use SOTA deep learnt model for L2 ranker which enhances the L2 ranker horizontally across all the languages, so now we support all the 50+ languages with better and enhanced precision.
  2. Precise phrase/short answer extraction from answer passages.
    undefined
  3. Simplified resource management by reducing the number of resources deployed.
    There will be only two resources:
    1. QnA Maker service for authoring and computation
    2. Azure Cognitive Search for storage and L1 ranking
  4. E2E region support for Authoring…

View original post 138 more words

#dotnet – Detecting Faces, DNN vs Haar Cascades from the 🎦 camera feed using #OpenCV and #net5

Buy Me A Coffee

Hi !

In one session around computer vision, someone ask the question about which approach is better

Haar Cascades or DNN?

And the answer can be show using the video below

net5 opencv face detection comparison between dnn and haar cascades

As you can see Haar Cascades works great for faces looking directly to the camera, with good lights and in optimal conditions. However, once my face stop looking at the camera, HC stop detecting faces; and the custom DNN model still working.

But, and this is important, the DNN model will rely only on the set of faces that has been used for training. If the model doesn’t include any demographics, in example Asian friends, the model won’t work with Asian subjects.

At the end, DNN models usually works great than Haar Cascades, however it’s really important to know the limitations of each model.

Happy coding!

Greetings

El Bruno

References

#Event – Resources used during the session “Getting started with Machine Learning.Net and AutoML” at the MVP Days Israel 2020

Buy Me A Coffee

Hi !

I had a great time early in the day (04:00AM!) with the MVP Days Israel team talking about how to start with Machine Learning .Net. As usual, time for slides and code:

Slides

Code

https://github.com/elbruno/events/tree/main/20201119%20MVP%20Day%20Israel%20MLNet

Resources

Recording

Coming soon !

Happy coding!

Greetings

El Bruno

#dotnet – Detecting Faces using DNN from the 🎦 camera feed in a WinForm using #OpenCV and #net5

Buy Me A Coffee

Hi !

Let’s do some face detection using a DNN model (See references). As yesterday, I won’t write about details, there are almost 20 years of online documentation available.

And, IMHO opinion code is much more useful that long writing, so let’s go there. 1st load the Caffe model and the config file.

// download model and prototxt from https://github.com/spmallick/learnopencv/tree/master/FaceDetectionComparison/models
const string configFile = "deploy.prototxt";
const string faceModel = "res10_300x300_ssd_iter_140000_fp16.caffemodel";
_faceNet = CvDnn.ReadNetFromCaffe(configFile, faceModel);

And, once we grab the camera frame, let’s perform face detection using the dnn model:

int frameHeight = newImage.Rows;
int frameWidth = newImage.Cols;

using var blob = CvDnn.BlobFromImage(newImage, 1.0, new Size(300, 300),
    new Scalar(104, 117, 123), false, false);
_faceNet.SetInput(blob, "data");

using var detection = _faceNet.Forward("detection_out");
using var detectionMat = new Mat(detection.Size(2), detection.Size(3), MatType.CV_32F,
    detection.Ptr(0));
for (int i = 0; i < detectionMat.Rows; i++)
{
    float confidence = detectionMat.At<float>(i, 2);

    if (confidence > 0.7)
    {
        int x1 = (int)(detectionMat.At<float>(i, 3) * frameWidth);
        int y1 = (int)(detectionMat.At<float>(i, 4) * frameHeight);
        int x2 = (int)(detectionMat.At<float>(i, 5) * frameWidth);
        int y2 = (int)(detectionMat.At<float>(i, 6) * frameHeight);

        Cv2.Rectangle(newImage, new Point(x1, y1), new Point(x2, y2), Scalar.Green);
        Cv2.PutText(newImage, "Face Dnn", new Point(x1 + 2, y2 + 20),
            HersheyFonts.HersheyComplexSmall, 1, Scalar.Green, 2);
    }
}

And, the full code is here

using System;
using System.Threading;
using System.Windows.Forms;
using OpenCvSharp;
using OpenCvSharp.Dnn;
using OpenCvSharp.Extensions;
using Point = OpenCvSharp.Point;
using Size = OpenCvSharp.Size;
namespace Demo08_WinFormFaceDetectionDNN
{
public partial class Form1 : Form
{
private bool _run = false;
private bool _doFaceDetection = false;
private VideoCapture _capture;
private Mat _image;
private Thread _cameraThread;
private bool _fps = false;
private Net _faceNet;
public Form1()
{
InitializeComponent();
Load += Form1_Load;
Closed += Form1_Closed;
}
private void Form1_Closed(object sender, EventArgs e)
{
_cameraThread.Interrupt();
_capture.Release();
}
private void btnStart_Click(object sender, EventArgs e)
{
_run = true;
}
private void btnStop_Click(object sender, EventArgs e)
{
_run = false;
}
private void btnFDDNN_Click(object sender, EventArgs e)
{
_doFaceDetection = !_doFaceDetection;
}
private void buttonFPS_Click(object sender, EventArgs e)
{
_fps = !_fps;
}
private void Form1_Load(object sender, EventArgs e)
{
// download model and prototxt from https://github.com/spmallick/learnopencv/tree/master/FaceDetectionComparison/models
const string configFile = "deploy.prototxt";
const string faceModel = "res10_300x300_ssd_iter_140000_fp16.caffemodel";
_faceNet = CvDnn.ReadNetFromCaffe(configFile, faceModel);
_capture = new VideoCapture(0);
_image = new Mat();
_cameraThread = new Thread(new ThreadStart(CaptureCameraCallback));
_cameraThread.Start();
}
private void CaptureCameraCallback()
{
while (true)
{
if (!_run) continue;
var startTime = DateTime.Now;
_capture.Read(_image);
if (_image.Empty()) return;
var imageRes = new Mat();
Cv2.Resize(_image, imageRes, new Size(320, 240));
var newImage = imageRes.Clone();
if (_doFaceDetection)
{
int frameHeight = newImage.Rows;
int frameWidth = newImage.Cols;
using var blob = CvDnn.BlobFromImage(newImage, 1.0, new Size(300, 300),
new Scalar(104, 117, 123), false, false);
_faceNet.SetInput(blob, "data");
using var detection = _faceNet.Forward("detection_out");
using var detectionMat = new Mat(detection.Size(2), detection.Size(3), MatType.CV_32F,
detection.Ptr(0));
for (int i = 0; i < detectionMat.Rows; i++)
{
float confidence = detectionMat.At<float>(i, 2);
if (confidence > 0.7)
{
int x1 = (int)(detectionMat.At<float>(i, 3) * frameWidth);
int y1 = (int)(detectionMat.At<float>(i, 4) * frameHeight);
int x2 = (int)(detectionMat.At<float>(i, 5) * frameWidth);
int y2 = (int)(detectionMat.At<float>(i, 6) * frameHeight);
Cv2.Rectangle(newImage, new Point(x1, y1), new Point(x2, y2), Scalar.Green);
Cv2.PutText(newImage, "Face Dnn", new Point(x1 + 2, y2 + 20),
HersheyFonts.HersheyComplexSmall, 1, Scalar.Green, 2);
}
}
}
if (_fps)
{
var diff = DateTime.Now startTime;
var fpsInfo = $"FPS: Nan";
if (diff.Milliseconds > 0)
{
var fpsVal = 1.0 / diff.Milliseconds * 1000;
fpsInfo = $"FPS: {fpsVal:00}";
}
Cv2.PutText(imageRes, fpsInfo, new Point(10, 20), HersheyFonts.HersheyComplexSmall, 1, Scalar.White);
}
var bmpWebCam = BitmapConverter.ToBitmap(imageRes);
var bmpEffect = BitmapConverter.ToBitmap(newImage);
pictureBoxWebCam.Image = bmpWebCam;
pictureBoxEffect.Image = bmpEffect;
}
}
}
}

That’s all for today!

Happy coding!

Greetings

El Bruno

References

#dotnet – Detecting Faces using Cascades from the 🎦 camera feed in a WinForm using #OpenCV and #net5

Buy Me A Coffee

Hi !

Let’s do some face detection using one of the most popular methods: Haar Casacades (See references). I won’t write about Cascades, there are almost 20 years of online documentation available.

And, IMHO opinion code is much more useful that long writing, so let’s go there.

1st load the cascade definition file.

_faceCascade = new CascadeClassifier();
_faceCascade.Load("haarcascade_frontalface_default.xml");

And, once we grab the camera frame, let’s perform some face detection:

using var gray = new Mat();
Cv2.CvtColor(newImage, gray, ColorConversionCodes.BGR2GRAY);

var faces = _faceCascade.DetectMultiScale(gray, 1.3, 5);
foreach (var face in faces)
{
  Cv2.Rectangle(newImage, face, Scalar.Red);
  Cv2.PutText(newImage, "Face Cascade", new Point(face.Left + 2, face.Top + face.Width + 20), HersheyFonts.HersheyComplexSmall, 1, Scalar.Red, 2);
}

And, the full code is here

using System;
using System.Threading;
using System.Windows.Forms;
using OpenCvSharp;
using OpenCvSharp.Extensions;
using Point = OpenCvSharp.Point;
using Size = OpenCvSharp.Size;
namespace Demo07_WinFormFaceDetectionCascades
{
public partial class Form1 : Form
{
private bool _run = true;
private bool _doFaceDetection = false;
private VideoCapture _capture;
private Mat _image;
private Thread _cameraThread;
private bool _fps = false;
private CascadeClassifier _faceCascade;
public Form1()
{
InitializeComponent();
Load += Form1_Load;
Closed += Form1_Closed;
}
private void Form1_Closed(object sender, EventArgs e)
{
_cameraThread.Interrupt();
_capture.Release();
}
private void btnStart_Click(object sender, EventArgs e)
{
_run = true;
}
private void btnStop_Click(object sender, EventArgs e)
{
_run = false;
}
private void btnFaceDetectionCascades_Click(object sender, EventArgs e)
{
_doFaceDetection = !_doFaceDetection;
}
private void buttonFPS_Click(object sender, EventArgs e)
{
_fps = !_fps;
}
private void Form1_Load(object sender, EventArgs e)
{
_faceCascade = new CascadeClassifier();
_faceCascade.Load("haarcascade_frontalface_default.xml");
_capture = new VideoCapture(0);
_image = new Mat();
_cameraThread = new Thread(new ThreadStart(CaptureCameraCallback));
_cameraThread.Start();
}
private void CaptureCameraCallback()
{
while (true)
{
if (!_run) continue;
var startTime = DateTime.Now;
_capture.Read(_image);
if (_image.Empty()) return;
var imageRes = new Mat();
Cv2.Resize(_image, imageRes, new Size(320, 240));
var newImage = imageRes.Clone();
if (_doFaceDetection)
{
using var gray = new Mat();
Cv2.CvtColor(newImage, gray, ColorConversionCodes.BGR2GRAY);
var faces = _faceCascade.DetectMultiScale(gray, 1.3, 5);
foreach (var face in faces)
{
Cv2.Rectangle(newImage, face, Scalar.Red);
Cv2.PutText(newImage, "Face Cascade", new Point(face.Left + 2, face.Top + face.Width + 20),
HersheyFonts.HersheyComplexSmall, 1, Scalar.Red, 2);
}
}
if (_fps)
{
var diff = DateTime.Now startTime;
var fpsInfo = $"FPS: Nan";
if (diff.Milliseconds > 0)
{
var fpsVal = 1.0 / diff.Milliseconds * 1000;
fpsInfo = $"FPS: {fpsVal:00}";
}
Cv2.PutText(newImage, fpsInfo, new Point(10, 20), HersheyFonts.HersheyComplexSmall, 1, Scalar.White);
}
var bmpWebCam = BitmapConverter.ToBitmap(imageRes);
var bmpEffect = BitmapConverter.ToBitmap(newImage);
pictureBoxWebCam.Image = bmpWebCam;
pictureBoxEffect.Image = bmpEffect;
}
}
}
}

That’s all for today!

Happy coding!

Greetings

El Bruno

References

#dotnet – Display the 🎦 camera feed in a WinForm using #OpenCV and #net5

Buy Me A Coffee

Hi !

Back on the Windows Forms days, cameras were tricky. We didn’t have a lot of libraries to work, and they usually require some extra work to handle unexpected errors. With Net 5 and OpenCVSharp, we can create a simple WebCam viewer like this one.

net5 opencv live camera and effects

Let’s start with a [Take a Photo] Windows Form app. This App have a PictureBox and a Button, and it’s all connected with these lines.

using System;
using System.Windows.Forms;
using OpenCvSharp;
namespace Demo03_WinForm
{
public partial class Form1 : Form
{
private VideoCapture _capture;
private Mat _image;
public Form1()
{
InitializeComponent();
}
private void Form1_Load(object sender, EventArgs e)
{
_capture = new VideoCapture(0);
_image = new Mat();
}
private void button1_Click(object sender, EventArgs e)
{
_capture.Read(_image);
if (_image.Empty()) return;
pictureBox1.Image = OpenCvSharp.Extensions.BitmapConverter.ToBitmap(_image);
}
}
}

As you can see it’s super simple, and the main lesson to learn here is on line 27, when we need to convert the OpenCV Mat object to a standard Bitmap to be used on a PictureBox.

We can some more features, like the one in the animation before

  • Start / Stop Camera Preview
  • Calculate FPS
  • Add Canny effect

with the following lines

using System;
using System.Threading;
using System.Windows.Forms;
using OpenCvSharp;
using OpenCvSharp.Extensions;
using Point = OpenCvSharp.Point;
using Size = OpenCvSharp.Size;
namespace Demo04_WinForm
{
public partial class Form1 : Form
{
private bool _run = false;
private bool _canny = false;
private VideoCapture _capture;
private Mat _image;
private Thread _cameraThread;
private bool _fps = false;
public Form1()
{
InitializeComponent();
Load += Form1_Load;
Closed += Form1_Closed;
}
private void Form1_Closed(object sender, EventArgs e)
{
_cameraThread.Interrupt();
_capture.Release();
}
private void btnStart_Click(object sender, EventArgs e)
{
_run = true;
}
private void btnStop_Click(object sender, EventArgs e)
{
_run = false;
}
private void btnCanny_Click(object sender, EventArgs e)
{
_canny = !_canny;
}
private void buttonFPS_Click(object sender, EventArgs e)
{
_fps = !_fps;
}
private void Form1_Load(object sender, EventArgs e)
{
_capture = new VideoCapture(0);
_image = new Mat();
_cameraThread = new Thread(new ThreadStart(CaptureCameraCallback));
_cameraThread.Start();
}
private void CaptureCameraCallback()
{
while (true)
{
if (!_run) continue;
var startTime = DateTime.Now;
_capture.Read(_image);
if (_image.Empty()) return;
var imageRes = new Mat();
Cv2.Resize(_image, imageRes, new Size(320, 240));
var newImage = imageRes.Clone();
if (_canny)
Cv2.Canny(imageRes, newImage, 50, 200);
if (_fps)
{
var diff = DateTime.Now startTime;
var fpsInfo = $"FPS: Nan";
if (diff.Milliseconds > 0)
{
var fpsVal = 1.0 / diff.Milliseconds * 1000;
fpsInfo = $"FPS: {fpsVal:00}";
}
Cv2.PutText(imageRes, fpsInfo, new Point(10, 20), HersheyFonts.HersheyComplexSmall, 1, Scalar.White);
}
var bmpWebCam = BitmapConverter.ToBitmap(imageRes);
var bmpEffect = BitmapConverter.ToBitmap(newImage);
pictureBoxWebCam.Image = bmpWebCam;
pictureBoxEffect.Image = bmpEffect;
}
}
}
}

Less than 100 lines of code to do some camera image processing in WinForms!

That’s all for today!

Happy coding!

Greetings

El Bruno

References

#dotnet – detecting corners on the 🎦 camera feed with FAST algorithm, #OpenCV and #net5

Buy Me A Coffee

Hi !

Today is a quick one: FAST corner detection algorithm.

FAST (Features from Accelerated Segment Test) algorithm was proposed by Edward Rosten and Tom Drummond in their paper “Machine learning for high-speed corner detection” in 2006 (Later revised it in 2010). A basic summary of the algorithm is presented below. Refer original paper for more details (All the images are taken from original paper).

OpenCV includes the FAST algorithm as part of the base set of operations to perform on an image. You only need 2 lines of code to get this output.

opencv demo using fast algorithm

The OpenCV FAST tutorial (see references) explain the different steps and, I’ll keep this simple, just sharing the code !

public Mat applyEffect(Mat image)
{
var newImage = new Mat();
Cv2.CvtColor(image, newImage, ColorConversionCodes.BGR2GRAY, 0);
KeyPoint[] keypoints = Cv2.FAST(newImage, 50, true);
foreach (KeyPoint kp in keypoints)
newImage.Circle((Point)kp.Pt, 3, Scalar.Red, 1, LineTypes.AntiAlias, 0);
return newImage;
}

That’s all for today!

Happy coding!

Greetings

El Bruno

References

Tooling para presentar Charlas/Eventos/Demos

Azure Mask is also supported on Edge !

Juanlu, ElGuerre

Durante estos días está teniendo lugar la #dotNETConf en la que no solo estamos viento novedades de .NET, también es momento para compartir todo tipo de experiencias. En concreto en este post, quiero hacer mención a dos de las herramientas que usan los Presentadores, y que ademas @shanselman contaba ayer en una de sus charlas.

ZoomIt

ZoomIt es la herramienta de Sysinternals, para anotaciones y zoom de pantalla.

La siguiente imagen muestra un rectangulo y una flecha señalando la caja de texto de “Power Toys Run” :

Aunque con 5 min de dedicación es facil entenderla al completo, los paso seguidos son estos para este ejemplo en concreto son los siguientes:

  • Ctrl + 1. Realiza el zoom de la pantalla. ¡Si pulsamos Ctrl + 2 en su lugar evitamos hacer el zoom, si no lo necesitamos!
  • Mouse Left Click. Fijamos el zoom en el punto en…

View original post 177 more words

#dotnet – detecting edges on the 🎦 camera feed with Canny algorithm, #OpenCV and #net5

Buy Me A Coffee

Hi !

The Canny edge detection is one the most popular algorithms, and it’s also +30 years old.

The Canny edge detector is an edge detection operator that uses a multi-stage algorithm to detect a wide range of edges in images. It was developed by John F. Canny in 1986.

OpenCV includes the Canny algorithm as part of the base set of operations to perform on an image. You only need 2 lines of code to get this output.

canny demo effect using camera

The OpenCV canny tutorial (see references) explain the different steps to detect edges:

  • Noise Reduction
  • Finding Intensity Gradient of the Image
  • Non-maximum Suppression
  • Hysteresis Thresholding

I’ll keep this simple, just sharing the code !

using OpenCvSharp;
var capture = new VideoCapture(0);
var window = new Window("El Bruno – OpenCVSharp Effects demo");
var image = new Mat();
var imageNew = new Mat();
bool applyCanny = false;
bool run = true;
while (run)
{
capture.Read(image);
if (image.Empty()) break;
if (applyCanny)
imageNew = applyCannyEffect(image);
else
imageNew = image.Clone();
window.ShowImage(imageNew);
switch ((char)Cv2.WaitKey(100))
{
case (char)27: // Esc – Exit
run = false;
break;
case 'c':
applyCanny = !applyCanny;
break;
}
}
Mat applyCannyEffect(Mat image)
{
var newImage = new Mat();
Cv2.Canny(image, newImage, 50, 200);
return newImage;
}

That’s all for today!

Happy coding!

Greetings

El Bruno

References

#dotnet – less than 20 lines to display a 🎦 camera feed with #OpenCV and #net5

Buy Me A Coffee

Hi !

This post is the start of a series of posts around OpenCV and DotNet Core .Net 5. Most of my samples are going to be camera based, so I’ll start with the +20 13 lines needed to access a local camera and show the frames in a window.

opencvsharp camera sample

And the code using Net Core 5:

using OpenCvSharp;
var capture = new VideoCapture(0);
var window = new Window("El Bruno – OpenCVSharp demo");
var image = new Mat();
while (true)
{
capture.Read(image);
if (image.Empty())
break;
window.ShowImage(image);
if (Cv2.WaitKey(1) == 113) // Q
break;
}

A couple of notes on previous code

  • Pressing Q key will stop the program
  • Line 2 define the camera to open based on local indexes, in example
// 0 Logitech Brio
// 1 Msft Lifecam
// ... 
// 4 Logitech Capture
// 5 OBS VirtualCam
  • This is a standard .Net 5 Console App
  • I’m using the great OpenCVSharp nuget packages to have OpenCV capabilities in my app.
opencvsharp nuget packages

And because I’m using Net 5, here goes a before / after preview

That’s all for today!

Happy coding!

Greetings

El Bruno

References