#Event – “Let’s code a drone to follow faces syncing everything with Azure IoT” at the Codegen 2021, Verona, Italy. Session Resources

Hi !

I had a great time early in the day with some friend from Italy.

As usual, time for slides and code:

Slides

Code

https://github.com/elbruno/events/tree/main/2021%2002%2013%20CodeGen%20Verona%20Italy%20Drone%20Azure%20IoT

My Session – Video

All Sessions – Videos

All +40 sessions here http://codegen2021.azurewebsites.net/videos

Resources

DJI Tello Drone,
https://www.amazon.com/Tello-CP-PT-00000252-01-Quadcopter-Drone/dp/B07BDHJJTH
Ryze Tello SDK, https://www.ryzerobotics.com/tello
Viola Jones, Object Detection Framework,
https://en.wikipedia.org/wiki/Viola%E2%80%93Jones_object_detection_framework
Cognitive Services Custom Vision, https://customvision.ai

Coding4Fun Drone posts

Happy coding!

Greetings

El Bruno

#OpenCV – Open a video file 🎥 and save each frame as a PNG 🖼 file to a folder 📂#Net5

Hi !

A couple of days ago I wrote this post, and made the sample with Python. So today, same scenario, however with C# 9 and Net5.

This is a non-usual scenario, however I’m on a point where I need to extract all the frames from a video file. The reason: some of these frames will be used to train a Machine Learning model.

There are tools that can do this, however it’s a nice moment to do some OpenCV code. Let’s go for it. A couple of remarks

Video file must be in the same folder as python file, and the name is defined in video_file, line 7
There is a live preview of the video, comment line 24 to avoid this
You can stop the process at any time pressing the Q letter

using System;
using OpenCvSharp;

var videoFile = "01.mp4";
System.IO.Directory.CreateDirectory("frames");

var capture = new VideoCapture(videoFile);
var window = new Window("El Bruno - OpenCVSharp Save Video Frame by Frame");
var image = new Mat();

var i = 0;
while (capture.IsOpened())
{
    capture.Read(image);
    if (image.Empty())
        break;

    i++;
    var imgNumber = i.ToString().PadLeft(8, '0');

    var frameImageFileName = $@"frames\image{imgNumber}.png";
    Cv2.ImWrite(frameImageFileName, image);

    window.ShowImage(image);
    if (Cv2.WaitKey(1) == 113) // Q
        break;
}

Console.WriteLine("Complete !");

Happy coding!

Greetings

El Bruno

Net 5 and OpenCV

#OpenCV – Open a video file 🎥 and save each frame as a PNG 🖼 file to a folder 📂#Python

Hi !

This is a non-usual scenario, however I’m on a point where I need to extract all the frames from a video file. The reason: some of these frames will be used to train a Machine Learning model.

There are tools that can do this, however it’s a nice moment to do some OpenCV code. Let’s go for it. A couple of remarks

Video file must be in the same folder as python file, and the name is defined in video_file, line 11
I resize the frame to 640 x 480, remove line 20 and fix vars name
There is a live preview of the video, comment line 27 to avoid this
You can stop the process at any time pressing the Q letter

# Bruno Capuano 2020
# open a video file and save each frame as a PNG file to a folder

import cv2
import os

# Camera Settings
camera_Width  = 640 # 1024 # 1280 # 640
camera_Heigth = 480 # 780  # 960  # 480
frameSize = (camera_Width, camera_Heigth)
video_file = "03.mp4"
video_capture = cv2.VideoCapture(video_file)

i = 0
while video_capture.isOpened():

    ret, frameOrig = video_capture.read()
    if ret == True:
        # resize frame, optional you may not need this
        frame = cv2.resize(frameOrig, frameSize)

        i += 1
        imgNumber = str(i).zfill(5)
        frameImageFileName = str(f'03\image{imgNumber}.png')
        cv2.imwrite(frameImageFileName, frameOrig)

        cv2.imshow('Video', frame)
    else:
        break

    # key controller
    key = cv2.waitKey(1) & 0xFF    
    if key == ord("q"):
        break

video_capture.release()
cv2.destroyAllWindows()

Happy coding!

Greetings

El Bruno

OpenCV and Python

#dotnet – Display a video file🎥 in Winform using #OpenCV and #net5

Hi !

Super short post today, however is the base of the next series of posts around OpenCV and .Net 5. I already wrote on how to work with a webcam, and now it’s time to share a simple piece of code to show

How to open and process a video frame by frame

I’ve downloaded a “traffic cam” video from Youtube, and this is the output in a Winform.

Super easy ! And as usual, the source code

      
        using System;
      
        using System.Threading;
      
        using System.Windows.Forms;
      
        using OpenCvSharp;
      
        using OpenCvSharp.Extensions;
      
        using Size = OpenCvSharp.Size;
      
        namespace Demo13_WinFormVideoFromFile
      
        {
      
            public partial class Form1 : Form
      
            {
      
                private bool _run = false;
      
                private VideoCapture _capture;
      
                private Mat _image;
      
                private Thread _cameraThread;
      
                private string _videoFile = "4K camera example for Traffic Monitoring (Road).mp4";
      
                private delegate void SafeCallDelegate(string text);
      
                public Form1()
      
                {
      
                    InitializeComponent();
      
                    Load += Form1_Load;
      
                    Closed += Form1_Closed;
      
                }
      
                private void Form1_Closed(object sender, EventArgs e)
      
                {
      
                    _cameraThread.Interrupt();
      
                    _capture.Release();
      
                }
      
                private void btnStart_Click(object sender, EventArgs e)
      
                {
      
                    _capture = new VideoCapture(_videoFile);
      
                    _run = true;
      
                }
      
                private void btnStop_Click(object sender, EventArgs e)
      
                {
      
                    _run = false;
      
                }
      
                private void Form1_Load(object sender, EventArgs e)
      
                {
      
                    _image = new Mat();
      
                    _cameraThread = new Thread(new ThreadStart(CaptureCameraCallback));
      
                    _cameraThread.Start();
      
                }
      
                private void CaptureCameraCallback()
      
                {
      
                    while (true)
      
                    {
      
                        if (!_run) continue;
      
                        var startTime = DateTime.Now;
      
                        _capture.Read(_image);
      
                        if (_image.Empty()) return;
      
                        var imageRes = new Mat();
      
                        Cv2.Resize(_image, imageRes, new Size(320, 240));
      
                        var bmpWebCam = BitmapConverter.ToBitmap(imageRes);
      
                        pictureBoxWebCam.Image = bmpWebCam;
      
                    }
      
                }
      
            }
      
        }

view raw opencvnet5winformvideofile.cs hosted with ❤ by GitHub

And the source video

Happy coding!

Greetings

El Bruno

References

OpenCVSharp, https://github.com/shimat/opencvsharp

Net 5 and OpenCV

#Event – Resources used during the session “Hack a drone, hack the camera and use AI” at the Global AI Tour, Lahore Pakistan, 2020

Hi !

I had a great time early in the day with my Microsoft Student Partners from Lahore Pakistan, for the Global AI Tour. As usual, time for slides and code:

Slides

Code

https://github.com/elbruno/events/tree/main/2020%2011%2030%20Global%20AI%20Tour%20Pakistan

Resources

DJI Tello Drone,
https://www.amazon.com/Tello-CP-PT-00000252-01-Quadcopter-Drone/dp/B07BDHJJTH
Ryze Tello SDK, https://www.ryzerobotics.com/tello
Viola Jones, Object Detection Framework,
https://en.wikipedia.org/wiki/Viola%E2%80%93Jones_object_detection_framework
Cognitive Services Custom Vision, https://customvision.ai

Coding4Fun Drone posts

Happy coding!

Greetings

El Bruno

#dotnet – Packaging my WinForm #OpenCV and #net5 app in a one-self contained file (Thanks @shanselman!)

Hi !

A couple of days ago, Scott Hanselman posted about a super cool feature (see references):

… you can make a single EXE that will run on any Windows Machine in two commands
How to make a WinForms app with .NET 5 entirely from the command line and publish as one self-contained file

and I was wondering, how this would work with my Winform tests with OpenCV. So, I pick up my Face Recognition using DNN app and decided to give it a try.

Thanks to @shanselman, I Just tested the "publish as a one self contained file" in a Net5, Winforms app using OpenCV and it rocks !

Single .exe file and all my ML models ready to use 😀 pic.twitter.com/1TEHdLE32q
— El Bruno @🇨🇦 (@elbruno) November 24, 2020

When I just build my Face Recognition using DNN app, it creates an output with

22 files
9 folders

Of course, including the Caffe model and the proto file. And I know, I can remove some unnecessary files, this is just for reference.

opencv net5 winform app with all the included files

Back to Scott’s post, I decided to clean this and run the publish command

dotnet publish -r win-x64 /p:PublishSingleFile=true /p:IncludeNativeLibrariesForSelfExtract=true

And the output was amazing:

My application (exe) self-contained file
My ML Models, prototxt and caffe model
OpenCVSharp dependency for Windows

opencv net5 winform app self contained file

This is great ! But wait, there is still an option here to PACK everything, that’s mean add the parameter IncludeAllContentForSelfExtract=True

dotnet publish -r win-x64 /p:PublishSingleFile=true /p:IncludeNativeLibrariesForSelfExtract=true /p:IncludeAllContentForSelfExtract=True

And, it really packs everything. However, it does not work in my default code. OpenCVSharp load the models from a physical location, and using the IncludeNativeLibrariesForSelfExtract option, the caffe model and prototxt are embedded as resources.

_netPose = CvDnn.ReadNetFromCaffe(PoseProtoTxt, PoseCaffeModel);
_netPose.SetPreferableBackend(Net.Backend.OPENCV);

There should be a trick here, for sure. I’ll spend some time to figure out, how we can load models directly from a single package app file!

Happy coding!

Greetings

El Bruno

References

OpenCVSharp, https://github.com/s
Scott Hanselman, How to make a WinForms app with .NET 5 entirely from the command line and publish as one self-contained file

Net 5 and OpenCV

#dotnet – Pose detection from the 🎦 camera feed using #OpenCV and #net5. Home-made #kinect!

Hi !

LearnOpenCV is an amazing resource to learn about OpenCV. And, it has lot of scenarios of real life problem solved with OpenCV. Most of the samples are in C++ or Python, so I decided to pick one related to pose estimation, and using .Net 5 in a Winforms App, build something like this:

net5 opencv pose estimation on real camera feed

The main model is OpenPose (see references). The model is amazing, and also works fast: ~1 FPS. There are other variations here, detecting face, Body, Foot, Face, and Hands Estimation, and more. I’ll try and share some of the other models usage in C# in next posts.

Now as usual, a huge code snippet with only the frame recognition and processing to detect the body joints.

private void CaptureCameraCallback()
{
    while (true)
    {
        if (!_run) continue;
        var startTime = DateTime.Now;

        _capture.Read(_image);
        if (_image.Empty()) return;
        var imageRes = new Mat();
        Cv2.Resize(_image, imageRes, new Size(320, 240));
        if (_detectPose)
        {

            var frameWidth = imageRes.Cols;
            var frameHeight = imageRes.Rows;

            const int inWidth = 368;
            const int inHeight = 368;

            // Convert Mat to batch of images
            using var inpBlob = CvDnn.BlobFromImage(imageRes, 1.0 / 255, new Size(inWidth, inHeight), new Scalar(0, 0, 0), false, false);

            _netPose.SetInput(inpBlob);

            using var output = _netPose.Forward();
            var H = output.Size(2);
            var W = output.Size(3);

            var points = new List<Point>();

            for (var n = 0; n < nPoints; n++)
            {
                // Probability map of corresponding body's part.
                using var probMap = new Mat(H, W, MatType.CV_32F, output.Ptr(0, n));
                var p = new Point2f(-1, -1);

                Cv2.MinMaxLoc(probMap, out _, out var maxVal, out _, out var maxLoc);

                var x = (frameWidth * maxLoc.X) / W;
                var y = (frameHeight * maxLoc.Y) / H;

                if (maxVal > thresh)
                {
                    p = maxLoc;
                    p.X *= (float)frameWidth / W;
                    p.Y *= (float)frameHeight / H;

                    Cv2.Circle(imageRes, (int)p.X, (int)p.Y, 8, Scalar.Azure, -1);
                    //Cv2.PutText(imageRes, Cv2.Format(n), new Point((int)p.X, (int)p.Y), HersheyFonts.HersheyComplex, 1, new Scalar(0, 0, 255), 1);
                }

                points.Add((Point)p);
            }

            WriteTextSafe(@$"Joints {nPoints} found");

            var nPairs = 14; //(POSE_PAIRS).Length / POSE_PAIRS[0].Length;

            for (var n = 0; n < nPairs; n++)
            {
                // lookup 2 connected body/hand parts
                var partA = points[posePairs[n][0]];
                var partB = points[posePairs[n][1]];
                if (partA.X <= 0 || partA.Y <= 0 || partB.X <= 0 || partB.Y <= 0)
                    continue;
                Cv2.Line(imageRes, partA, partB, new Scalar(0, 255, 255), 8);
                Cv2.Circle(imageRes, partA.X, partA.Y, 8, new Scalar(0, 0, 255), -1);
                Cv2.Circle(imageRes, partB.X, partB.Y, 8, new Scalar(0, 0, 255), -1);
            }

        }
// rest of the code to calc FPS and display the image
    }
}

Super fun ! and check the references for the model and support files download location.

Happy coding!

Greetings

El Bruno

References

OpenCVSharp, https://github.com/shimat/opencvsharp
Wikipedia, Canny Edge Detector
OpenCV, Canny tutorial
OpenCV, FAST tutorial
Wikipedia, FAST
OpenCV, Perform face detection using Haar Cascades
OpenCV, Load Caffe framework models
Caffe Model Zoo
LearnOpenCV, Multi-Person Pose Estimation in OpenCV using OpenPose
GitHub, CMU-Perceptual, OpenPose

Net 5 and OpenCV

#dotnet – GoogleNet detection from the 🎦 camera feed using #OpenCV and #net5. Bonus: C++ to C# time!

Hi !

So I was browsing in the OpenCV documentation and I find a nice sample that uses opencv_dnn module for image classification by using GoogLeNet trained network from Caffe model zoo.

So I give it a try, and get a decent .Net 5 Winforms App running at ~30 FPS.

opencv net5 load and analyze camera frames with googlenet

The model was trained with 1000 classes, and once you get the main focus on the camera it work great with objects like a machine, mug, bottle, etc. There is a nice amount of code here, and because the DNN analysis is performed in a separated thread, I need to update the label details using PInvoke functions.

      
        using System;
      
        using System.IO;
      
        using System.Linq;
      
        using System.Threading;
      
        using System.Windows.Forms;
      
        using OpenCvSharp;
      
        using OpenCvSharp.Dnn;
      
        using OpenCvSharp.Extensions;
      
        using Point = OpenCvSharp.Point;
      
        using Size = OpenCvSharp.Size;
      
        namespace Demo11_WinFormGoogleNet
      
        {
      
            public partial class Form1 : Form
      
            {
      
                private bool _run = true;
      
                private bool _useGoogleNet = false;
      
                private VideoCapture _capture;
      
                private Mat _image;
      
                private Thread _cameraThread;
      
                private bool _fps = false;
      
                private Net _netGoogleNet;
      
                private string[] _classNames;
      
                const string ProtoTxt = @"models\bvlc_googlenet.prototxt";
      
                const string CaffeModel = @"models\bvlc_googlenet.caffemodel";
      
                const string SynsetWords = @"models\synset_words.txt";
      
                private delegate void SafeCallDelegate(string text);
      
                public Form1()
      
                {
      
                    InitializeComponent();
      
                    Load += Form1_Load;
      
                    Closed += Form1_Closed;
      
                }
      
                private void Form1_Closed(object sender, EventArgs e)
      
                {
      
                    _cameraThread.Interrupt();
      
                    _capture.Release();
      
                }
      
                private void btnStart_Click(object sender, EventArgs e)
      
                {
      
                    _run = true;
      
                }
      
                private void btnStop_Click(object sender, EventArgs e)
      
                {
      
                    _run = false;
      
                }
      
                private void btnGoogleNet_Click(object sender, EventArgs e)
      
                {
      
                    _useGoogleNet = !_useGoogleNet;
      
                }
      
                private void buttonFPS_Click(object sender, EventArgs e)
      
                {
      
                    _fps = !_fps;
      
                }
      
                private void Form1_Load(object sender, EventArgs e)
      
                {
      
                    _classNames = File.ReadAllLines(SynsetWords)
      
                        .Select(line => line.Split(' ').Last())
      
                        .ToArray();
      
                    _netGoogleNet = CvDnn.ReadNetFromCaffe(ProtoTxt, CaffeModel);
      
                    _capture = new VideoCapture(0);
      
                    _image = new Mat();
      
                    _cameraThread = new Thread(new ThreadStart(CaptureCameraCallback));
      
                    _cameraThread.Start();
      
                }
      
                private void CaptureCameraCallback()
      
                {
      
                    while (true)
      
                    {
      
                        if (!_run) continue;
      
                        var startTime = DateTime.Now;
      
                        _capture.Read(_image);
      
                        if (_image.Empty()) return;
      
                        var imageRes = new Mat();
      
                        Cv2.Resize(_image, imageRes, new Size(320, 240));
      
                        if (_useGoogleNet)
      
                        {
      
                            // Convert Mat to batch of images
      
                            using var inputBlob = CvDnn.BlobFromImage(imageRes, 1, new Size(224, 224), new Scalar(104, 117, 123));
      
                            _netGoogleNet.SetInput(inputBlob, "data");
      
                            using var prob = _netGoogleNet.Forward("prob");
      
                            // find the best class
      
                            GetMaxClass(prob, out int classId, out double classProb);
      
                            var msg = @$"Best class: #{classId} '{_classNames[classId]}' – Probability: {classProb:P2}";
      
                            // display output
      
                            WriteTextSafe(msg);
      
                        }
      
                        if (_fps)
      
                        {
      
                            var diff = DateTime.Now – startTime;
      
                            var fpsInfo = $"FPS: Nan";
      
                            if (diff.Milliseconds > 0)
      
                            {
      
                                var fpsVal = 1.0 / diff.Milliseconds * 1000;
      
                                fpsInfo = $"FPS: {fpsVal:00}";
      
                            }
      
                            Cv2.PutText(imageRes, fpsInfo, new Point(10, 20), HersheyFonts.HersheyComplexSmall, 1, Scalar.White);
      
                        }
      
                        var bmpWebCam = BitmapConverter.ToBitmap(imageRes);
      
                        pictureBoxWebCam.Image = bmpWebCam;
      
                    }
      
                }
      
                private void WriteTextSafe(string text)
      
                {
      
                    if (lblOutputAnalysis.InvokeRequired)
      
                    {
      
                        var d = new SafeCallDelegate(WriteTextSafe);
      
                        lblOutputAnalysis.Invoke(d, new object[] { text });
      
                    }
      
                    else
      
                    {
      
                        lblOutputAnalysis.Text = text;
      
                    }
      
                }
      
                private static void GetMaxClass(Mat probBlob, out int classId, out double classProb)
      
                {
      
                    // reshape the blob to 1×1000 matrix
      
                    using (var probMat = probBlob.Reshape(1, 1))
      
                    {
      
                        Cv2.MinMaxLoc(probMat, out _, out classProb, out _, out var classNumber);
      
                        classId = classNumber.X;
      
                    }
      
                }
      
            }
      
        }

view raw opencvnet5googlenet.cs hosted with ❤ by GitHub

Super fun ! and check the references for the model and support files download location.

Happy coding!

Greetings

El Bruno

References

OpenCVSharp, https://github.com/shimat/opencvsharp
Wikipedia, Canny Edge Detector
OpenCV, Canny tutorial
OpenCV, FAST tutorial
Wikipedia, FAST
OpenCV, Perform face detection using Haar Cascades
OpenCV, Load Caffe framework models
Caffe Model Zoo

Net 5 and OpenCV

#dotnet – Age and Gender estimation from the 🎦 camera feed using #OpenCV and #net5

Hi !

Face detected, so next step is to use some prebuild models to perform additional actions: like estimate the Age of a face, and also the Gender. In order to do this, I downloaded a couple of models from here.

Disclaimer: these models are just sample models, do not use them in production. These model does not covers all the necessary scenarios for a real implementation.

And the final winform app is kind of cute!

Below you can find the complete Form1 source code, before let’s take a look at the sample analyzing a magazine photo.

2020-11-23_16-37-43 opencv net 5 detecting multiple faces

So let’s analyze the code. For this sample, we load 3 models to work with age, faces and gender.

// # detect faces, age and gender using models from https://github.com/spmallick/learnopencv/tree/08e61fe80b8c0244cc4029ac11e44cd0fbb008c3/AgeGender
const string faceProto = "models/deploy.prototxt";
const string faceModel = "models/res10_300x300_ssd_iter_140000_fp16.caffemodel";
const string ageProto = @"models/age_deploy.prototxt";
const string ageModel = @"models/age_net.caffemodel";
const string genderProto = @"models/gender_deploy.prototxt";
const string genderModel = @"models/gender_net.caffemodel";
_ageNet = CvDnn.ReadNetFromCaffe(ageProto, ageModel);
_genderNet = CvDnn.ReadNetFromCaffe(genderProto, genderModel);
_faceNet = CvDnn.ReadNetFromCaffe(faceProto, faceModel);

Once the models are loaded, in the loop to analyze camera frames, we perform face detection, and then age and gender estimation.

while (true)
{
    if (!_run) continue;
    var startTime = DateTime.Now;

    _capture.Read(_image);
    if (_image.Empty()) return;
    var imageRes = new Mat();
    Cv2.Resize(_image, imageRes, new Size(320, 240));
    var newImage = imageRes.Clone();

    if (_doFaceDetection) DetectFaces(newImage, imageRes);

    if (_fps) CalculateFps(startTime, newImage);

    var bmpWebCam = BitmapConverter.ToBitmap(imageRes);
    var bmpEffect = BitmapConverter.ToBitmap(newImage);

    pictureBoxWebCam.Image = bmpWebCam;
    pictureBoxEffect.Image = bmpEffect;
}

For each detected face, we perform the age and gender estimation. In order to do this, we crop the detected face (plus a padding), and perform the estimation on the cropped image.

private void AnalyzeAgeAndGender(int x1, int y1, int x2, int y2, Mat imageRes, Mat newImage)
{
    // get face frame
    var x = x1 - Padding;
    var y = y1 - Padding;
    var w = (x2 - x1) + Padding * 3;
    var h = (y2 - y1) + Padding * 3;
    Rect roiNew = new Rect(x, y, w, h);
    var face = imageRes[roi: roiNew];

    var meanValues = new Scalar(78.4263377603, 87.7689143744, 114.895847746);
    var blobGender = CvDnn.BlobFromImage(face, 1.0, new Size(227, 227), mean: meanValues,
        swapRB: false);
    _genderNet.SetInput(blobGender);
    var genderPreds = _genderNet.Forward();

    GetMaxClass(genderPreds, out int classId, out double classProbGender);
    var gender = _genderList[classId];

    _ageNet.SetInput(blobGender);
    var agePreds = _ageNet.Forward();
    GetMaxClass(agePreds, out int classIdAge, out double classProbAge);
    var age = _ageList[classIdAge];

    var label = $"{gender},{age}";
    Cv2.PutText(newImage, label, new Point(x1 - 10, y2 + 20), HersheyFonts.HersheyComplexSmall, 1, Scalar.Yellow, 1);
}

private void GetMaxClass(Mat probBlob, out int classId, out double classProb)
{
    // reshape the blob to 1x1000 matrix
    using var probMat = probBlob.Reshape(1, 1);
    Cv2.MinMaxLoc(probMat, out _, out classProb, out _, out var classNumber);
    classId = classNumber.X;
    Debug.WriteLine($"X: {classNumber.X} - Y: {classNumber.Y} ");
}

It’s also important to mention to the GetMaxClass() function, to retrieve the best detected element in the prob result.

And the complete source code:

	using System;
	using System.Collections.Generic;
	using System.Diagnostics;
	using System.Threading;
	using System.Windows.Forms;
	using OpenCvSharp;
	using OpenCvSharp.Dnn;
	using OpenCvSharp.Extensions;
	using Point = OpenCvSharp.Point;
	using Size = OpenCvSharp.Size;

	namespace Demo10_WinFormAgeAndGender
	{
	public partial class Form1 : Form
	{
	private bool _run = true;
	private bool _doFaceDetection = true;
	private bool _doAgeGender = false;

	private VideoCapture _capture;
	private Mat _image;
	private Thread _cameraThread;
	private bool _fps = false;
	private Net _faceNet;
	private Net _ageNet;
	private Net _genderNet;
	private const int LineThickness = 2;
	private const int Padding = 10;
	private readonly List<string> _genderList = new List<string> { "Male", "Female" };
	private readonly List<string> _ageList = new List<string> { "(0-2)", "(4-6)", "(8-12)", "(15-20)", "(25-32)", "(38-43)", "(48-53)", "(60-100)" };


	public Form1()
	{
	InitializeComponent();
	Load += Form1_Load;
	Closed += Form1_Closed;
	}

	private void Form1_Closed(object sender, EventArgs e)
	{
	_cameraThread.Interrupt();
	_capture.Release();
	}

	private void btnStart_Click(object sender, EventArgs e)
	{
	_run = true;
	}

	private void btnStop_Click(object sender, EventArgs e)
	{
	_run = false;
	}

	private void btnFDDNN_Click(object sender, EventArgs e)
	{
	_doFaceDetection = !_doFaceDetection;
	}

	private void buttonFPS_Click(object sender, EventArgs e)
	{
	_fps = !_fps;
	}

	private void btnAgeGender_Click(object sender, EventArgs e)
	{
	_doAgeGender = !_doAgeGender;
	}


	private void Form1_Load(object sender, EventArgs e)
	{
	// # detect faces, age and gender using models from https://github.com/spmallick/learnopencv/tree/08e61fe80b8c0244cc4029ac11e44cd0fbb008c3/AgeGender
	const string faceProto = "models/deploy.prototxt";
	const string faceModel = "models/res10_300x300_ssd_iter_140000_fp16.caffemodel";
	const string ageProto = @"models/age_deploy.prototxt";
	const string ageModel = @"models/age_net.caffemodel";
	const string genderProto = @"models/gender_deploy.prototxt";
	const string genderModel = @"models/gender_net.caffemodel";
	_ageNet = CvDnn.ReadNetFromCaffe(ageProto, ageModel);
	_genderNet = CvDnn.ReadNetFromCaffe(genderProto, genderModel);
	_faceNet = CvDnn.ReadNetFromCaffe(faceProto, faceModel);

	_capture = new VideoCapture(0);
	_image = new Mat();
	_cameraThread = new Thread(new ThreadStart(CaptureCameraCallback));
	_cameraThread.Start();
	}

	private void CaptureCameraCallback()
	{
	while (true)
	{
	if (!_run) continue;
	var startTime = DateTime.Now;

	_capture.Read(_image);
	if (_image.Empty()) return;
	var imageRes = new Mat();
	Cv2.Resize(_image, imageRes, new Size(320, 240));
	var newImage = imageRes.Clone();

	if (_doFaceDetection) DetectFaces(newImage, imageRes);

	if (_fps) CalculateFps(startTime, newImage);

	var bmpWebCam = BitmapConverter.ToBitmap(imageRes);
	var bmpEffect = BitmapConverter.ToBitmap(newImage);

	pictureBoxWebCam.Image = bmpWebCam;
	pictureBoxEffect.Image = bmpEffect;
	}
	}

	private static void CalculateFps(DateTime startTime, Mat imageRes)
	{
	var diff = DateTime.Now – startTime;
	var fpsInfo = $"FPS: Nan";
	if (diff.Milliseconds > 0)
	{
	var fpsVal = 1.0 / diff.Milliseconds * 1000;
	fpsInfo = $"FPS: {fpsVal:00}";
	}

	Cv2.PutText(imageRes, fpsInfo, new Point(10, 20), HersheyFonts.HersheyComplexSmall, 1, Scalar.White);
	}

	private void DetectFaces(Mat newImage, Mat imageRes)
	{
	// DNN
	int frameHeight = newImage.Rows;
	int frameWidth = newImage.Cols;

	using var blob = CvDnn.BlobFromImage(newImage, 1.0, new Size(300, 300), new Scalar(104, 117, 123), false, false);
	_faceNet.SetInput(blob, "data");

	using var detection = _faceNet.Forward("detection_out");
	using var detectionMat = new Mat(detection.Size(2), detection.Size(3), MatType.CV_32F, detection.Ptr(0));
	for (int i = 0; i < detectionMat.Rows; i++)
	{
	float confidence = detectionMat.At<float>(i, 2);

	if (confidence > 0.7)
	{
	int x1 = (int)(detectionMat.At<float>(i, 3) * frameWidth);
	int y1 = (int)(detectionMat.At<float>(i, 4) * frameHeight);
	int x2 = (int)(detectionMat.At<float>(i, 5) * frameWidth);
	int y2 = (int)(detectionMat.At<float>(i, 6) * frameHeight);

	Cv2.Rectangle(newImage, new Point(x1, y1), new Point(x2, y2), Scalar.Green, LineThickness);

	if (_doAgeGender)
	AnalyzeAgeAndGender(x1, y1, x2, y2, imageRes, newImage);
	}
	}
	}

	private void AnalyzeAgeAndGender(int x1, int y1, int x2, int y2, Mat imageRes, Mat newImage)
	{
	// get face frame
	var x = x1 – Padding;
	var y = y1 – Padding;
	var w = (x2 – x1) + Padding * 3;
	var h = (y2 – y1) + Padding * 3;
	Rect roiNew = new Rect(x, y, w, h);
	var face = imageRes[roi: roiNew];

	var meanValues = new Scalar(78.4263377603, 87.7689143744, 114.895847746);
	var blobGender = CvDnn.BlobFromImage(face, 1.0, new Size(227, 227), mean: meanValues,
	swapRB: false);
	_genderNet.SetInput(blobGender);
	var genderPreds = _genderNet.Forward();

	GetMaxClass(genderPreds, out int classId, out double classProbGender);
	var gender = _genderList[classId];

	_ageNet.SetInput(blobGender);
	var agePreds = _ageNet.Forward();
	GetMaxClass(agePreds, out int classIdAge, out double classProbAge);
	var age = _ageList[classIdAge];

	var label = $"{gender},{age}";
	Cv2.PutText(newImage, label, new Point(x1 – 10, y2 + 20), HersheyFonts.HersheyComplexSmall, 1, Scalar.Yellow, 1);
	}

	private void GetMaxClass(Mat probBlob, out int classId, out double classProb)
	{
	// reshape the blob to 1×1000 matrix
	using var probMat = probBlob.Reshape(1, 1);
	Cv2.MinMaxLoc(probMat, out _, out classProb, out _, out var classNumber);
	classId = classNumber.X;
	Debug.WriteLine($"X: {classNumber.X} – Y: {classNumber.Y} ");
	}
	}
	}

view raw opencvnet5agegenderestimation.cs hosted with ❤ by GitHub

Happy coding!

Greetings

El Bruno

References

OpenCVSharp, https://github.com/shimat/opencvsharp
Wikipedia, Canny Edge Detector
OpenCV, Canny tutorial
OpenCV, FAST tutorial
Wikipedia, FAST
OpenCV, Perform face detection using Haar Cascades

Net 5 and OpenCV

#dotnet – Detecting Faces, DNN vs Haar Cascades from the 🎦 camera feed using #OpenCV and #net5

Hi !

In one session around computer vision, someone ask the question about which approach is better

Haar Cascades or DNN?

And the answer can be show using the video below

net5 opencv face detection comparison between dnn and haar cascades

As you can see Haar Cascades works great for faces looking directly to the camera, with good lights and in optimal conditions. However, once my face stop looking at the camera, HC stop detecting faces; and the custom DNN model still working.

But, and this is important, the DNN model will rely only on the set of faces that has been used for training. If the model doesn’t include any demographics, in example Asian friends, the model won’t work with Asian subjects.

At the end, DNN models usually works great than Haar Cascades, however it’s really important to know the limitations of each model.

Happy coding!

Greetings

El Bruno

References

OpenCVSharp, https://github.com/shimat/opencvsharp
Wikipedia, Canny Edge Detector
OpenCV, Canny tutorial
OpenCV, FAST tutorial
Wikipedia, FAST
OpenCV, Perform face detection using Haar Cascades

Slides

Code

My Session – Video

All Sessions – Videos

Resources

Coding4Fun Drone posts

Share this:

Like this:

Net 5 and OpenCV

Share this:

Like this:

OpenCV and Python

Share this:

Like this:

How to open and process a video frame by frame

References

Net 5 and OpenCV

Share this:

Like this:

Slides

Code

Resources

Coding4Fun Drone posts

Share this:

Like this:

References

Net 5 and OpenCV

Share this:

Like this:

References

Net 5 and OpenCV

Share this:

Like this:

References

Net 5 and OpenCV

Share this:

Like this:

References

Net 5 and OpenCV

Share this:

Like this:

References

Net 5 and OpenCV

Share this:

Like this: