#Event – “Let’s code a drone to follow faces syncing everything with Azure IoT” at the Codegen 2021, Verona, Italy. Session Resources

Buy Me A Coffee

Hi !

I had a great time early in the day with some friend from Italy.

drone telemetry in azure IoT

As usual, time for slides and code:

Slides

Code

https://github.com/elbruno/events/tree/main/2021%2002%2013%20CodeGen%20Verona%20Italy%20Drone%20Azure%20IoT

My Session – Video

All Sessions – Videos

All +40 sessions here http://codegen2021.azurewebsites.net/videos

Resources

Happy coding!

Greetings

El Bruno


#OpenCV – Open a video file 🎥 and save each frame as a PNG 🖼 file to a folder 📂#Net5

Buy Me A Coffee

Hi !

A couple of days ago I wrote this post, and made the sample with Python. So today, same scenario, however with C# 9 and Net5.

This is a non-usual scenario, however I’m on a point where I need to extract all the frames from a video file. The reason: some of these frames will be used to train a Machine Learning model.

There are tools that can do this, however it’s a nice moment to do some OpenCV code. Let’s go for it. A couple of remarks

  • Video file must be in the same folder as python file, and the name is defined in video_file, line 7
  • There is a live preview of the video, comment line 24 to avoid this
  • You can stop the process at any time pressing the Q letter
using System;
using OpenCvSharp;

var videoFile = "01.mp4";
System.IO.Directory.CreateDirectory("frames");

var capture = new VideoCapture(videoFile);
var window = new Window("El Bruno - OpenCVSharp Save Video Frame by Frame");
var image = new Mat();

var i = 0;
while (capture.IsOpened())
{
    capture.Read(image);
    if (image.Empty())
        break;

    i++;
    var imgNumber = i.ToString().PadLeft(8, '0');

    var frameImageFileName = $@"frames\image{imgNumber}.png";
    Cv2.ImWrite(frameImageFileName, image);

    window.ShowImage(image);
    if (Cv2.WaitKey(1) == 113) // Q
        break;
}

Console.WriteLine("Complete !");

Happy coding!

Greetings

El Bruno


#OpenCV – Open a video file 🎥 and save each frame as a PNG 🖼 file to a folder 📂#Python

Buy Me A Coffee

Hi !

This is a non-usual scenario, however I’m on a point where I need to extract all the frames from a video file. The reason: some of these frames will be used to train a Machine Learning model.

There are tools that can do this, however it’s a nice moment to do some OpenCV code. Let’s go for it. A couple of remarks

  • Video file must be in the same folder as python file, and the name is defined in video_file, line 11
  • I resize the frame to 640 x 480, remove line 20 and fix vars name
  • There is a live preview of the video, comment line 27 to avoid this
  • You can stop the process at any time pressing the Q letter
# Bruno Capuano 2020
# open a video file and save each frame as a PNG file to a folder

import cv2
import os

# Camera Settings
camera_Width  = 640 # 1024 # 1280 # 640
camera_Heigth = 480 # 780  # 960  # 480
frameSize = (camera_Width, camera_Heigth)
video_file = "03.mp4"
video_capture = cv2.VideoCapture(video_file)

i = 0
while video_capture.isOpened():

    ret, frameOrig = video_capture.read()
    if ret == True:
        # resize frame, optional you may not need this
        frame = cv2.resize(frameOrig, frameSize)

        i += 1
        imgNumber = str(i).zfill(5)
        frameImageFileName = str(f'03\image{imgNumber}.png')
        cv2.imwrite(frameImageFileName, frameOrig)

        cv2.imshow('Video', frame)
    else:
        break

    # key controller
    key = cv2.waitKey(1) & 0xFF    
    if key == ord("q"):
        break

video_capture.release()
cv2.destroyAllWindows()

Happy coding!

Greetings

El Bruno


#dotnet – Display a video file🎥 in Winform using #OpenCV and #net5

Buy Me A Coffee

Hi !

Super short post today, however is the base of the next series of posts around OpenCV and .Net 5. I already wrote on how to work with a webcam, and now it’s time to share a simple piece of code to show

How to open and process a video frame by frame

I’ve downloaded a “traffic cam” video from Youtube, and this is the output in a Winform.

Super easy ! And as usual, the source code

using System;
using System.Threading;
using System.Windows.Forms;
using OpenCvSharp;
using OpenCvSharp.Extensions;
using Size = OpenCvSharp.Size;
namespace Demo13_WinFormVideoFromFile
{
public partial class Form1 : Form
{
private bool _run = false;
private VideoCapture _capture;
private Mat _image;
private Thread _cameraThread;
private string _videoFile = "4K camera example for Traffic Monitoring (Road).mp4";
private delegate void SafeCallDelegate(string text);
public Form1()
{
InitializeComponent();
Load += Form1_Load;
Closed += Form1_Closed;
}
private void Form1_Closed(object sender, EventArgs e)
{
_cameraThread.Interrupt();
_capture.Release();
}
private void btnStart_Click(object sender, EventArgs e)
{
_capture = new VideoCapture(_videoFile);
_run = true;
}
private void btnStop_Click(object sender, EventArgs e)
{
_run = false;
}
private void Form1_Load(object sender, EventArgs e)
{
_image = new Mat();
_cameraThread = new Thread(new ThreadStart(CaptureCameraCallback));
_cameraThread.Start();
}
private void CaptureCameraCallback()
{
while (true)
{
if (!_run) continue;
var startTime = DateTime.Now;
_capture.Read(_image);
if (_image.Empty()) return;
var imageRes = new Mat();
Cv2.Resize(_image, imageRes, new Size(320, 240));
var bmpWebCam = BitmapConverter.ToBitmap(imageRes);
pictureBoxWebCam.Image = bmpWebCam;
}
}
}
}

And the source video

Happy coding!

Greetings

El Bruno


References

#Event – Resources used during the session “Hack a drone, hack the camera and use AI” at the Global AI Tour, Lahore Pakistan, 2020

Buy Me A Coffee

Hi !

I had a great time early in the day with my Microsoft Student Partners from Lahore Pakistan, for the Global AI Tour. As usual, time for slides and code:

Slides

Code

https://github.com/elbruno/events/tree/main/2020%2011%2030%20Global%20AI%20Tour%20Pakistan

Resources

Happy coding!

Greetings

El Bruno


#dotnet – Packaging my WinForm #OpenCV and #net5 app in a one-self contained file (Thanks @shanselman!)

Buy Me A Coffee

Hi !

A couple of days ago, Scott Hanselman posted about a super cool feature (see references):

… you can make a single EXE that will run on any Windows Machine in two commands

How to make a WinForms app with .NET 5 entirely from the command line and publish as one self-contained file

and I was wondering, how this would work with my Winform tests with OpenCV. So, I pick up my Face Recognition using DNN app and decided to give it a try.

When I just build my Face Recognition using DNN app, it creates an output with

  • 22 files
  • 9 folders

Of course, including the Caffe model and the proto file. And I know, I can remove some unnecessary files, this is just for reference.

opencv net5 winform app with all the included files

Back to Scott’s post, I decided to clean this and run the publish command

dotnet publish -r win-x64 /p:PublishSingleFile=true /p:IncludeNativeLibrariesForSelfExtract=true

And the output was amazing:

  • My application (exe) self-contained file
  • My ML Models, prototxt and caffe model
  • OpenCVSharp dependency for Windows
opencv net5 winform app self contained file

This is great ! But wait, there is still an option here to PACK everything, that’s mean add the parameter IncludeAllContentForSelfExtract=True

dotnet publish -r win-x64 /p:PublishSingleFile=true /p:IncludeNativeLibrariesForSelfExtract=true /p:IncludeAllContentForSelfExtract=True

And, it really packs everything. However, it does not work in my default code. OpenCVSharp load the models from a physical location, and using the IncludeNativeLibrariesForSelfExtract option, the caffe model and prototxt are embedded as resources.

_netPose = CvDnn.ReadNetFromCaffe(PoseProtoTxt, PoseCaffeModel);
_netPose.SetPreferableBackend(Net.Backend.OPENCV);

There should be a trick here, for sure. I’ll spend some time to figure out, how we can load models directly from a single package app file!

Happy coding!

Greetings

El Bruno


References

#dotnet – Pose detection from the 🎦 camera feed using #OpenCV and #net5. Home-made #kinect!

Buy Me A Coffee

Hi !

LearnOpenCV is an amazing resource to learn about OpenCV. And, it has lot of scenarios of real life problem solved with OpenCV. Most of the samples are in C++ or Python, so I decided to pick one related to pose estimation, and using .Net 5 in a Winforms App, build something like this:

net5 opencv pose estimation on real camera feed

The main model is OpenPose (see references). The model is amazing, and also works fast: ~1 FPS. There are other variations here, detecting face, Body, Foot, Face, and Hands Estimation, and more. I’ll try and share some of the other models usage in C# in next posts.

Now as usual, a huge code snippet with only the frame recognition and processing to detect the body joints.

private void CaptureCameraCallback()
{
    while (true)
    {
        if (!_run) continue;
        var startTime = DateTime.Now;

        _capture.Read(_image);
        if (_image.Empty()) return;
        var imageRes = new Mat();
        Cv2.Resize(_image, imageRes, new Size(320, 240));
        if (_detectPose)
        {

            var frameWidth = imageRes.Cols;
            var frameHeight = imageRes.Rows;

            const int inWidth = 368;
            const int inHeight = 368;

            // Convert Mat to batch of images
            using var inpBlob = CvDnn.BlobFromImage(imageRes, 1.0 / 255, new Size(inWidth, inHeight), new Scalar(0, 0, 0), false, false);

            _netPose.SetInput(inpBlob);

            using var output = _netPose.Forward();
            var H = output.Size(2);
            var W = output.Size(3);

            var points = new List<Point>();

            for (var n = 0; n < nPoints; n++)
            {
                // Probability map of corresponding body's part.
                using var probMap = new Mat(H, W, MatType.CV_32F, output.Ptr(0, n));
                var p = new Point2f(-1, -1);

                Cv2.MinMaxLoc(probMap, out _, out var maxVal, out _, out var maxLoc);

                var x = (frameWidth * maxLoc.X) / W;
                var y = (frameHeight * maxLoc.Y) / H;

                if (maxVal > thresh)
                {
                    p = maxLoc;
                    p.X *= (float)frameWidth / W;
                    p.Y *= (float)frameHeight / H;

                    Cv2.Circle(imageRes, (int)p.X, (int)p.Y, 8, Scalar.Azure, -1);
                    //Cv2.PutText(imageRes, Cv2.Format(n), new Point((int)p.X, (int)p.Y), HersheyFonts.HersheyComplex, 1, new Scalar(0, 0, 255), 1);
                }

                points.Add((Point)p);
            }

            WriteTextSafe(@$"Joints {nPoints} found");

            var nPairs = 14; //(POSE_PAIRS).Length / POSE_PAIRS[0].Length;

            for (var n = 0; n < nPairs; n++)
            {
                // lookup 2 connected body/hand parts
                var partA = points[posePairs[n][0]];
                var partB = points[posePairs[n][1]];
                if (partA.X <= 0 || partA.Y <= 0 || partB.X <= 0 || partB.Y <= 0)
                    continue;
                Cv2.Line(imageRes, partA, partB, new Scalar(0, 255, 255), 8);
                Cv2.Circle(imageRes, partA.X, partA.Y, 8, new Scalar(0, 0, 255), -1);
                Cv2.Circle(imageRes, partB.X, partB.Y, 8, new Scalar(0, 0, 255), -1);
            }

        }
// rest of the code to calc FPS and display the image
    }
}

Super fun ! and check the references for the model and support files download location.

Happy coding!

Greetings

El Bruno


References

#dotnet – GoogleNet detection from the 🎦 camera feed using #OpenCV and #net5. Bonus: C++ to C# time!

Buy Me A Coffee

Hi !

So I was browsing in the OpenCV documentation and I find a nice sample that uses opencv_dnn module for image classification by using GoogLeNet trained network from Caffe model zoo.

So I give it a try, and get a decent .Net 5 Winforms App running at ~30 FPS.

opencv net5 load and analyze camera frames with googlenet

The model was trained with 1000 classes, and once you get the main focus on the camera it work great with objects like a machine, mug, bottle, etc. There is a nice amount of code here, and because the DNN analysis is performed in a separated thread, I need to update the label details using PInvoke functions.

using System;
using System.IO;
using System.Linq;
using System.Threading;
using System.Windows.Forms;
using OpenCvSharp;
using OpenCvSharp.Dnn;
using OpenCvSharp.Extensions;
using Point = OpenCvSharp.Point;
using Size = OpenCvSharp.Size;
namespace Demo11_WinFormGoogleNet
{
public partial class Form1 : Form
{
private bool _run = true;
private bool _useGoogleNet = false;
private VideoCapture _capture;
private Mat _image;
private Thread _cameraThread;
private bool _fps = false;
private Net _netGoogleNet;
private string[] _classNames;
const string ProtoTxt = @"models\bvlc_googlenet.prototxt";
const string CaffeModel = @"models\bvlc_googlenet.caffemodel";
const string SynsetWords = @"models\synset_words.txt";
private delegate void SafeCallDelegate(string text);
public Form1()
{
InitializeComponent();
Load += Form1_Load;
Closed += Form1_Closed;
}
private void Form1_Closed(object sender, EventArgs e)
{
_cameraThread.Interrupt();
_capture.Release();
}
private void btnStart_Click(object sender, EventArgs e)
{
_run = true;
}
private void btnStop_Click(object sender, EventArgs e)
{
_run = false;
}
private void btnGoogleNet_Click(object sender, EventArgs e)
{
_useGoogleNet = !_useGoogleNet;
}
private void buttonFPS_Click(object sender, EventArgs e)
{
_fps = !_fps;
}
private void Form1_Load(object sender, EventArgs e)
{
_classNames = File.ReadAllLines(SynsetWords)
.Select(line => line.Split(' ').Last())
.ToArray();
_netGoogleNet = CvDnn.ReadNetFromCaffe(ProtoTxt, CaffeModel);
_capture = new VideoCapture(0);
_image = new Mat();
_cameraThread = new Thread(new ThreadStart(CaptureCameraCallback));
_cameraThread.Start();
}
private void CaptureCameraCallback()
{
while (true)
{
if (!_run) continue;
var startTime = DateTime.Now;
_capture.Read(_image);
if (_image.Empty()) return;
var imageRes = new Mat();
Cv2.Resize(_image, imageRes, new Size(320, 240));
if (_useGoogleNet)
{
// Convert Mat to batch of images
using var inputBlob = CvDnn.BlobFromImage(imageRes, 1, new Size(224, 224), new Scalar(104, 117, 123));
_netGoogleNet.SetInput(inputBlob, "data");
using var prob = _netGoogleNet.Forward("prob");
// find the best class
GetMaxClass(prob, out int classId, out double classProb);
var msg = @$"Best class: #{classId} '{_classNames[classId]}' – Probability: {classProb:P2}";
// display output
WriteTextSafe(msg);
}
if (_fps)
{
var diff = DateTime.Now startTime;
var fpsInfo = $"FPS: Nan";
if (diff.Milliseconds > 0)
{
var fpsVal = 1.0 / diff.Milliseconds * 1000;
fpsInfo = $"FPS: {fpsVal:00}";
}
Cv2.PutText(imageRes, fpsInfo, new Point(10, 20), HersheyFonts.HersheyComplexSmall, 1, Scalar.White);
}
var bmpWebCam = BitmapConverter.ToBitmap(imageRes);
pictureBoxWebCam.Image = bmpWebCam;
}
}
private void WriteTextSafe(string text)
{
if (lblOutputAnalysis.InvokeRequired)
{
var d = new SafeCallDelegate(WriteTextSafe);
lblOutputAnalysis.Invoke(d, new object[] { text });
}
else
{
lblOutputAnalysis.Text = text;
}
}
private static void GetMaxClass(Mat probBlob, out int classId, out double classProb)
{
// reshape the blob to 1×1000 matrix
using (var probMat = probBlob.Reshape(1, 1))
{
Cv2.MinMaxLoc(probMat, out _, out classProb, out _, out var classNumber);
classId = classNumber.X;
}
}
}
}

Super fun ! and check the references for the model and support files download location.

Happy coding!

Greetings

El Bruno


References

#dotnet – Age and Gender estimation from the 🎦 camera feed using #OpenCV and #net5

Buy Me A Coffee

Hi !

Face detected, so next step is to use some prebuild models to perform additional actions: like estimate the Age of a face, and also the Gender. In order to do this, I downloaded a couple of models from here.

Disclaimer: these models are just sample models, do not use them in production. These model does not covers all the necessary scenarios for a real implementation.

And the final winform app is kind of cute!

Below you can find the complete Form1 source code, before let’s take a look at the sample analyzing a magazine photo.

2020-11-23_16-37-43 opencv net 5 detecting multiple faces

So let’s analyze the code. For this sample, we load 3 models to work with age, faces and gender.

// # detect faces, age and gender using models from https://github.com/spmallick/learnopencv/tree/08e61fe80b8c0244cc4029ac11e44cd0fbb008c3/AgeGender
const string faceProto = "models/deploy.prototxt";
const string faceModel = "models/res10_300x300_ssd_iter_140000_fp16.caffemodel";
const string ageProto = @"models/age_deploy.prototxt";
const string ageModel = @"models/age_net.caffemodel";
const string genderProto = @"models/gender_deploy.prototxt";
const string genderModel = @"models/gender_net.caffemodel";
_ageNet = CvDnn.ReadNetFromCaffe(ageProto, ageModel);
_genderNet = CvDnn.ReadNetFromCaffe(genderProto, genderModel);
_faceNet = CvDnn.ReadNetFromCaffe(faceProto, faceModel);

Once the models are loaded, in the loop to analyze camera frames, we perform face detection, and then age and gender estimation.

while (true)
{
    if (!_run) continue;
    var startTime = DateTime.Now;

    _capture.Read(_image);
    if (_image.Empty()) return;
    var imageRes = new Mat();
    Cv2.Resize(_image, imageRes, new Size(320, 240));
    var newImage = imageRes.Clone();

    if (_doFaceDetection) DetectFaces(newImage, imageRes);

    if (_fps) CalculateFps(startTime, newImage);

    var bmpWebCam = BitmapConverter.ToBitmap(imageRes);
    var bmpEffect = BitmapConverter.ToBitmap(newImage);

    pictureBoxWebCam.Image = bmpWebCam;
    pictureBoxEffect.Image = bmpEffect;
}

For each detected face, we perform the age and gender estimation. In order to do this, we crop the detected face (plus a padding), and perform the estimation on the cropped image.

private void AnalyzeAgeAndGender(int x1, int y1, int x2, int y2, Mat imageRes, Mat newImage)
{
    // get face frame
    var x = x1 - Padding;
    var y = y1 - Padding;
    var w = (x2 - x1) + Padding * 3;
    var h = (y2 - y1) + Padding * 3;
    Rect roiNew = new Rect(x, y, w, h);
    var face = imageRes[roi: roiNew];

    var meanValues = new Scalar(78.4263377603, 87.7689143744, 114.895847746);
    var blobGender = CvDnn.BlobFromImage(face, 1.0, new Size(227, 227), mean: meanValues,
        swapRB: false);
    _genderNet.SetInput(blobGender);
    var genderPreds = _genderNet.Forward();

    GetMaxClass(genderPreds, out int classId, out double classProbGender);
    var gender = _genderList[classId];

    _ageNet.SetInput(blobGender);
    var agePreds = _ageNet.Forward();
    GetMaxClass(agePreds, out int classIdAge, out double classProbAge);
    var age = _ageList[classIdAge];

    var label = $"{gender},{age}";
    Cv2.PutText(newImage, label, new Point(x1 - 10, y2 + 20), HersheyFonts.HersheyComplexSmall, 1, Scalar.Yellow, 1);
}

private void GetMaxClass(Mat probBlob, out int classId, out double classProb)
{
    // reshape the blob to 1x1000 matrix
    using var probMat = probBlob.Reshape(1, 1);
    Cv2.MinMaxLoc(probMat, out _, out classProb, out _, out var classNumber);
    classId = classNumber.X;
    Debug.WriteLine($"X: {classNumber.X} - Y: {classNumber.Y} ");
}

It’s also important to mention to the GetMaxClass() function, to retrieve the best detected element in the prob result.

And the complete source code:

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading;
using System.Windows.Forms;
using OpenCvSharp;
using OpenCvSharp.Dnn;
using OpenCvSharp.Extensions;
using Point = OpenCvSharp.Point;
using Size = OpenCvSharp.Size;
namespace Demo10_WinFormAgeAndGender
{
public partial class Form1 : Form
{
private bool _run = true;
private bool _doFaceDetection = true;
private bool _doAgeGender = false;
private VideoCapture _capture;
private Mat _image;
private Thread _cameraThread;
private bool _fps = false;
private Net _faceNet;
private Net _ageNet;
private Net _genderNet;
private const int LineThickness = 2;
private const int Padding = 10;
private readonly List<string> _genderList = new List<string> { "Male", "Female" };
private readonly List<string> _ageList = new List<string> { "(0-2)", "(4-6)", "(8-12)", "(15-20)", "(25-32)", "(38-43)", "(48-53)", "(60-100)" };
public Form1()
{
InitializeComponent();
Load += Form1_Load;
Closed += Form1_Closed;
}
private void Form1_Closed(object sender, EventArgs e)
{
_cameraThread.Interrupt();
_capture.Release();
}
private void btnStart_Click(object sender, EventArgs e)
{
_run = true;
}
private void btnStop_Click(object sender, EventArgs e)
{
_run = false;
}
private void btnFDDNN_Click(object sender, EventArgs e)
{
_doFaceDetection = !_doFaceDetection;
}
private void buttonFPS_Click(object sender, EventArgs e)
{
_fps = !_fps;
}
private void btnAgeGender_Click(object sender, EventArgs e)
{
_doAgeGender = !_doAgeGender;
}
private void Form1_Load(object sender, EventArgs e)
{
// # detect faces, age and gender using models from https://github.com/spmallick/learnopencv/tree/08e61fe80b8c0244cc4029ac11e44cd0fbb008c3/AgeGender
const string faceProto = "models/deploy.prototxt";
const string faceModel = "models/res10_300x300_ssd_iter_140000_fp16.caffemodel";
const string ageProto = @"models/age_deploy.prototxt";
const string ageModel = @"models/age_net.caffemodel";
const string genderProto = @"models/gender_deploy.prototxt";
const string genderModel = @"models/gender_net.caffemodel";
_ageNet = CvDnn.ReadNetFromCaffe(ageProto, ageModel);
_genderNet = CvDnn.ReadNetFromCaffe(genderProto, genderModel);
_faceNet = CvDnn.ReadNetFromCaffe(faceProto, faceModel);
_capture = new VideoCapture(0);
_image = new Mat();
_cameraThread = new Thread(new ThreadStart(CaptureCameraCallback));
_cameraThread.Start();
}
private void CaptureCameraCallback()
{
while (true)
{
if (!_run) continue;
var startTime = DateTime.Now;
_capture.Read(_image);
if (_image.Empty()) return;
var imageRes = new Mat();
Cv2.Resize(_image, imageRes, new Size(320, 240));
var newImage = imageRes.Clone();
if (_doFaceDetection) DetectFaces(newImage, imageRes);
if (_fps) CalculateFps(startTime, newImage);
var bmpWebCam = BitmapConverter.ToBitmap(imageRes);
var bmpEffect = BitmapConverter.ToBitmap(newImage);
pictureBoxWebCam.Image = bmpWebCam;
pictureBoxEffect.Image = bmpEffect;
}
}
private static void CalculateFps(DateTime startTime, Mat imageRes)
{
var diff = DateTime.Now startTime;
var fpsInfo = $"FPS: Nan";
if (diff.Milliseconds > 0)
{
var fpsVal = 1.0 / diff.Milliseconds * 1000;
fpsInfo = $"FPS: {fpsVal:00}";
}
Cv2.PutText(imageRes, fpsInfo, new Point(10, 20), HersheyFonts.HersheyComplexSmall, 1, Scalar.White);
}
private void DetectFaces(Mat newImage, Mat imageRes)
{
// DNN
int frameHeight = newImage.Rows;
int frameWidth = newImage.Cols;
using var blob = CvDnn.BlobFromImage(newImage, 1.0, new Size(300, 300), new Scalar(104, 117, 123), false, false);
_faceNet.SetInput(blob, "data");
using var detection = _faceNet.Forward("detection_out");
using var detectionMat = new Mat(detection.Size(2), detection.Size(3), MatType.CV_32F, detection.Ptr(0));
for (int i = 0; i < detectionMat.Rows; i++)
{
float confidence = detectionMat.At<float>(i, 2);
if (confidence > 0.7)
{
int x1 = (int)(detectionMat.At<float>(i, 3) * frameWidth);
int y1 = (int)(detectionMat.At<float>(i, 4) * frameHeight);
int x2 = (int)(detectionMat.At<float>(i, 5) * frameWidth);
int y2 = (int)(detectionMat.At<float>(i, 6) * frameHeight);
Cv2.Rectangle(newImage, new Point(x1, y1), new Point(x2, y2), Scalar.Green, LineThickness);
if (_doAgeGender)
AnalyzeAgeAndGender(x1, y1, x2, y2, imageRes, newImage);
}
}
}
private void AnalyzeAgeAndGender(int x1, int y1, int x2, int y2, Mat imageRes, Mat newImage)
{
// get face frame
var x = x1 Padding;
var y = y1 Padding;
var w = (x2 x1) + Padding * 3;
var h = (y2 y1) + Padding * 3;
Rect roiNew = new Rect(x, y, w, h);
var face = imageRes[roi: roiNew];
var meanValues = new Scalar(78.4263377603, 87.7689143744, 114.895847746);
var blobGender = CvDnn.BlobFromImage(face, 1.0, new Size(227, 227), mean: meanValues,
swapRB: false);
_genderNet.SetInput(blobGender);
var genderPreds = _genderNet.Forward();
GetMaxClass(genderPreds, out int classId, out double classProbGender);
var gender = _genderList[classId];
_ageNet.SetInput(blobGender);
var agePreds = _ageNet.Forward();
GetMaxClass(agePreds, out int classIdAge, out double classProbAge);
var age = _ageList[classIdAge];
var label = $"{gender},{age}";
Cv2.PutText(newImage, label, new Point(x1 10, y2 + 20), HersheyFonts.HersheyComplexSmall, 1, Scalar.Yellow, 1);
}
private void GetMaxClass(Mat probBlob, out int classId, out double classProb)
{
// reshape the blob to 1×1000 matrix
using var probMat = probBlob.Reshape(1, 1);
Cv2.MinMaxLoc(probMat, out _, out classProb, out _, out var classNumber);
classId = classNumber.X;
Debug.WriteLine($"X: {classNumber.X} – Y: {classNumber.Y} ");
}
}
}

Happy coding!

Greetings

El Bruno


References

#dotnet – Detecting Faces, DNN vs Haar Cascades from the 🎦 camera feed using #OpenCV and #net5

Buy Me A Coffee

Hi !

In one session around computer vision, someone ask the question about which approach is better

Haar Cascades or DNN?

And the answer can be show using the video below

net5 opencv face detection comparison between dnn and haar cascades

As you can see Haar Cascades works great for faces looking directly to the camera, with good lights and in optimal conditions. However, once my face stop looking at the camera, HC stop detecting faces; and the custom DNN model still working.

But, and this is important, the DNN model will rely only on the set of faces that has been used for training. If the model doesn’t include any demographics, in example Asian friends, the model won’t work with Asian subjects.

At the end, DNN models usually works great than Haar Cascades, however it’s really important to know the limitations of each model.

Happy coding!

Greetings

El Bruno


References