Trending February 2024 # Learn How To Build Face Detection System # Suggested March 2024 # Top 2 Popular

You are reading the article Learn How To Build Face Detection System updated in February 2024 on the website Moimoishop.com. We hope that the information we have shared is helpful to you. If you find the content interesting and meaningful, please share it with your friends and continue to follow and support us for the latest updates. Suggested March 2024 Learn How To Build Face Detection System

This article was published as a part of the Data Science Blogathon.

d different pre-trained models for the image-classification tasks and we have a fair understanding of object detection, and we looked at the different architectures that can be used for solving the object detection problems.

To summarize object detection involves identifying objects along with their locations. In this article, we’ll understand one of the use cases of object detection, which is face detection.

Table of Contents

1. Introduction to Face Detection

2. Application of Face Detection

3. Understanding the Problem Statement: WIDER FACE

4. Converting the annotations of the WIDER FACE dataset as per Detectron2

5. Steps to solve the Face Detection problems

Let’s start…….

Introduction to Face Detection

Similar to object detection, in face detection problems, the purpose is to identify faces from the image, along with their locations.

And we can see in the above images that we can have a single face in an image and multiple faces. In this article, we’ll learn both of these tasks. That is, how to detect faces when there is a single face in the image and the same for multiple phases in the image.

So by the end of this article, you’ll be able to build models that can detect either a single face or multiple faces from the image.

Let’s first look at some interesting applications of face detection.

Application of Face Detection

Face detection can be used to check if a person is wearing a mask or not. The steps for building such a model would be to detect the faces from the image first and then predict if the faces contain a mask or not.

This is just one of the use cases of face detection. Face Recognition can be another use case. Here, we have to identify the person present in the image. So, to solve this problem, we’ll first have to detect the faces from the image and then identify the faces.

There can be multiple other applications, like installing CCTV cameras that can identify persons to prevent ATM machine robberies or to build a user-friendly check-in system at the airports.

Understanding the Problem Statement: WIDER FACE

So in this section, we’ll understand the problem statement that we have picked for this article.

The objective of this article is to build a model that can detect faces from the images. So by the end of this article, we’ll be able to build a model that can take images as shown here. As an input, detect the faces in that image that returns the bounding boxes for the locations of these faces present in the image.

So we’ll be training the model, and we know that in order to train the model, we need the training data. So let’s look at the dataset that we’ll be using in order to train the face detection model.

In this module, we’ll be using the wider face dataset, which is benchmark data for the face detection tasks. It contains more than 32000 images comprising of approximately 0.4 million faces and this data set contains images and faces which have a large variation in scale, occlusion pose and so on.

So scale basically refers to the image size, and based on the size of the image, the dataset is grouped into three scales: small, medium and large.

Similarly, occlusion also has three categories, which is no occlusion, partial occlusion and heavy occlusion. The wider dataset contains images from a range of 60 even categories like the parade, festival, football, tennis and many more.

So the diversity of the data set is huge, and these even categories are further divided into three sets based on the ease of detection which is easy, medium and hard.

The categories marked in the above image,  green belong to the easy set, those in red belong to the medium Set, and the blue ones belong to the hard Set. In this article, we’ll be working with images belonging to the easy Set.

Also, we’ll be using the detectoron2 library in order to build the face detection model, since it provides the state-of-the-art implementation for object detection tasks.

And by now, we know that in order to work with detectron2, our dataset must follow a specific format. So in the next section, we’ll first look at the format of the wider phase data set and then see how the dataset should be pre-processed in order to use it for the directron2 library.

Converting the Annotations of the WIDER FACE Dataset as per Detectron2

Till now we have understood the face detection problem and the dataset which we’ll be using to build the face detection model. In the above section, we discussed that we’ll be using detectron2 to build the model.

So, let’s quickly have an overview of detectron2. detectoron2 is a platform for object detection and segmentation tasks. It is created by the research team of Facebook AI. And detectron2 implements state-of-the-art architectures like faster R-CNN retina net and so on.

Now, let’s look at the format of the dataset we currently have. So, this is what the annotations of the wider face dataset look like. We have images containing faces, and corresponding to these images, we have the annotations.

The first thing here represents the file name then we have the number of faces present in a particular image. And since this image has a single face the value here is one. Finally, we have the bounding box coordinates for this image. These bounding boxes follow the following format. We have the x-min,y-min values followed by the height and width of the bounding box. Also, We have the values representing the different measures of variabilities that we have discussed in the above section, Including the blur expression, illumination, occlusion and so on.

So this is what the annotations of wider face data sets look like. Now here is another example from the dataset.

So first of all we have the image name, followed by the number of faces in the image that is 3 in this case, followed by the bounding box coordinates. The bounding boxes for each face are represented using a separate dictionary. For this image, since we have three faces, there will be three different dictionaries.

Now, the above image is the annotation from the wider phase dataset. And in order to use this dataset in detectron2, we must convert the annotations to this format as shown below.

So, we first have a dictionary for the bounding boxes. and instead of having x-min, y-min, width and height for detectron2. We should have the x-min, y-min, x-max and y-max values that we’ll be using here. The category id here represents the category of this image. Then we have to pass the file name, which will be the name of the above image, followed by the height, width and image id.

So this is the format that detectron2 expects as input and hence we’ll have to pre-process our data accordingly. So while creating the face detection model we’ll be pre-processing these input formats into the format required for detectron2.

So that’s all on the data preparation part. in the next section, we’ll discuss the steps that we’ll be following to build the face detection model, and then finally we will work on the implementation.

Steps to Solve the Face Detection Problem

In this section, we will look at the steps that we’ll be following, while building the face detection model using detectron2. So we’ll start with these steps:-

Install Dependencies

Loading and pre-processing the data

Creating annotations as per Detectron2

Register the dataset

Fine Tuning the model

Evaluating model performance

I  have already uploaded the dataset on the drive and in order to run the notebooks at your end, you should also upload the data set on your drive.

So, here are the files which are available in the dataset. we have an chúng tôi file, that includes the names of classes belonging to the easy set. Then we have the wider space chúng tôi which contains the annotations for images in the training and the validation set. WIDER_train contains the training images and WIDER_val contains the validation images.

Building a face detection model to detect single faces

Till now we have understood what is face detection, how we have to annotate the data set as per detectron2 and the steps that we’ll be following to build our face detection model.

Now as already discussed, there can be scenarios where we have only a single face in the image and there can be situations where the task is to detect multiple faces in the images.

So in this article, we’ll solve both of these tasks. We’ll first build a model that can detect a single face from the image, and then we’ll build another model which will be able to detect multiple objects from the image. So, in this article, our focus will be on building a model that can detect a single face. So to build a face detection system that can detect a single face from the image, a few steps that we are going to follow, which we have discussed in the last section.

1. Install Dependencies

We’ll first start with installing all the dependencies. First of all, we are installing the 5.1 version of the PYML library. This is a prerequisite for detectoron2. And if we have an older version of this library some of the functionalities of detectron2 might not work correctly.

Then we are installing the detectron2 library and we have seen this step earlier while we were implementing the RetinaNet model.

# install required version !pip install pyyaml==5.1 # installing detectron2 wheels/cu101/torch1.5/index.html 2. Loading and pre-processing the data

The next step is to load the dataset and preprocess it. since the data set is present in google drive, we have to first mount the drive.

# mounting drive from google.colab import drive drive.mount('drive')

Next, we are extracting the files which are in the zip format. So we are extracting our training, validation, and annotations. Now we are reading the annotation files for both the training and the validation set.

# extracting files !unzip '/content/drive/My Drive/Wider_dataset/WIDER_train.zip' !unzip '/content/drive/My Drive/Wider_dataset/WIDER_val.zip' !unzip '/content/drive/My Drive/Wider_dataset/wider_face_split.zip'

So to read the files, we’ll be using the pandas library and hence importing that here first. And then we are specifying the paths for the annotation files of the training and the validation set and then read them using the read_csv function of pandas.

# reading files import pandas as pd # specify path of the data path_train = '/content/wider_face_split/wider_face_train_bbx_gt.txt' path_val = '/content/wider_face_split/wider_face_val_bbx_gt.txt' # reading data train = pd.read_csv(path_train,header=None) val = pd.read_csv(path_val,header=None)

Let’s quickly look at the CSV files. So here are the first 10 rows from the train CSV file that we have. Here we have the file names followed by the number of faces in that image, and then the bounding box coordinates and the same part. And you can see is repeated. So currently the format is not readable. So let’s convert this format to a more meaningful and readable form. Which should look something like this. We would want the format where we have names in a separate column then numbers of faces as a separate variable and then the bounding box coordinates. hence we’ll reformat the data set accordingly.

# pre-processing data # this function accepts the dataframe and returns modified dataframe def reformat(df): # fetch values of first column values = df[0].values # creating empty lists names=[] num_faces=[] bbox=[] # fetch values into corresponding lists for i in range(len(values)): # if an image if ".jpg" in values[i]: # no. of faces num=int(values[i+1]) # append image name to list names.append(values[i]) # append no. of faces to list num_faces.append(num) # create bbox list box=[] for j in range(i+2,i+2+num): box.append(values[j]) # append bbox list to list bbox.append(box) return pd.DataFrame({'name':names,'num_faces':num_faces,'bbox':bbox})

we are going to convert the training and the validation data sets and store it as train and val.

# pre-processing the data train = reformat(train) val = reformat(val)

So let’s look at the first few rows of these data sets. So we are first printing the head of the train. you can see that this data set is now in the required format.

# first 5 rows of the pre-processed data train.head()

Source:- Author

let us also look at the shape of the training data and the shape of the validation data. so we can see that in the training data we have 12 880 rows and in the validation, we have 3226 rows.

# shape of the training data train.shape

The output is:- (12880, 3)

# shape of validation data val.shape

The output is:-  (3226, 3)

Next, we’ll do some pre-processing on this data set so here we are adding the complete path before the file name and for training, we are adding the path wider_ train/images and for validation, the path will be wider_val/images.

# adding full path train['name'] = train['name'].apply(lambda x: 'WIDER_train/images/'+x ) val['name'] = val['name'].apply(lambda x: 'WIDER_val/images/'+x )

After applying this the new dataset would look something like the below image. So we’ll have the complete path for all of these images.

# first 5 rows train.head()

Source:- Author

Next, we are converting the bounding box coordinates to floating points using the float_function of the numpy library. Hence we are first importing the numpy library here and then converting the bounding box coordinates for both training and the validation set and again printing the first five rows of the data set.

# converting bbox to floating point import numpy as np train['bbox'] = train['bbox'].apply (lambda row:[ np.float_(annos.split()) for annos in row] ) val['bbox'] = val['bbox'].apply (lambda row:[ np.float_(annos.split()) for annos in row] ) # first 5 rows train.head()

Source:- Author

So the above image, the output confirms that the bounding boxes are converted into floating points. Here so you can see this is bbox column value 27.0, 26.0, and so on.

well as the validation data sets.

# extracting class names train['class']= train['name'].apply(lambda x:x.split("/")[2].split("--")[1]) val['class'] = val['name'].apply(lambda x:x.split("/")[2].split("--")[1]) # first 5 rows train.head()

Source:- Author

you can see that now we have a new column class that has the class against each of these images.

Next, we are going to read the chúng tôi file which contains the names of the classes belonging to the easy set. So we are reading this file here using the read_csv function and printing the values that we get.

# reading class names easy_df = pd.read_csv('drive/My Drive/Wider_dataset/easy.txt',header=None) easy_labels = easy_df[0].values # easy labels easy_labels

Here is a list of the classes that belong to the easy set. so we have a total of 20 classes here.

Source:- Author

Now we’ll select only those images which belong to any of these above-mentioned classes. So here we are running a for loop for only the easy classes and fetching the rows which have the easy categories.

# creating empty dataframes train_df, val_df= pd.DataFrame(), pd.DataFrame() # fetching rows of easy classes only for i in easy_labels: train_df = pd.concat( [train_df, train[train['class']==i]] ) val_df = pd.concat( [val_df, val[val['class']==i]] )

We have taken a subset from the data set for both training and validations. As discussed earlier the aim of this article is to build a face detection model that will work for single faces and hence we are only selecting those images which have a single face so we are doing this for both our training and validation.

Now before we go ahead, let’s quickly check the train shape and the validation shape.

# shape of dataframe train_df.shape, val_df.shape

So we have 1000 images in train and 274 images in the validation data. Next, We will see how to convert the annotations of a wider dataset to the annotations of Detectron2.

3. Creating annotations as per Detectron2

Now, we’ll convert the annotations of our dataset as per the requirements of the detectoron2 library. So here is what the annotations should look like. We’ll first have the file name and then the number of faces in that image followed by the bounding box coordinates which are of the format Xmin, Ymin, width, and height, and finally the class.

So we want the annotations to be in the below format. We’ll first see how to get this format for a single image and then we’ll write a generalized function that will convert the annotations format for all of these images.

# custom annotation format idx=0 values = train_df.values[idx] print(values)

Source:- Author

# for dealing with images import cv2 # create annotation dict record = {} # image name filename = values[0] # height and width of an image height, width = cv2.imread(filename).shape[:2] # creating fields record["file_name"] = filename record["image_id"] = 0 record["height"] = height record["width"] = width # different ways to represent bounding box from detectron2.structures import BoxMode # create bbox list objs = [] # for every face in an image for i in range(len(values[2])): # fetch bbox coordinates annos = values[2][i] # unpack values x1, y1, w, h = annos[0], annos[1], annos[2], annos[3] # find bottom right corner x2, y2 = x1 + w, y1 + h # create bbox dict obj = { "bbox": [x1, y1, x2, y2], "bbox_mode": BoxMode.XYXY_ABS, "category_id": 0, "iscrowd": 0 } # append bbox dict to bbox list objs.append(obj) # assigning bbox list to annotation dict record["annotations"] = objs # standard annotation format record

The output is:-

Source:- Author

def create_annotation(df): # create list to store annotation dict dataset_dicts = [] # for each image for idx, v in enumerate(df.values): # create annotation dict record = {} # image name filename = v[0] # height and width of an image height, width = cv2.imread(filename).shape[:2] # assign values to fields record["file_name"] = filename record["image_id"] = idx record["height"] = height record["width"] = width # create list for bbox objs = [] for i in range(len(v[2])): # bounding box coordinates annos = v[2][i] # unpack values x1, y1, w, h = annos[0], annos[1], annos[2], annos[3] # find bottom right corner x2, y2 = x1 + w, y1 + h # create bbox dict obj = { "bbox": [x1, y1, x2, y2], "bbox_mode": BoxMode.XYXY_ABS, "category_id": 0, "iscrowd": 0 } # append bbox dict to a bbox list objs.append(obj) # assign bbox list to annotation dict record["annotations"] = objs # append annotation dict to list dataset_dicts.append(record) return dataset_dicts # create standard annotations for training and validation datasets train_annotation = create_annotation(train_df) val_annotation = create_annotation(val_df) # standard annotation of an image train_annotation[0]

Source:- Author

4. Register the dataset

To let detectron2 know how to obtain a dataset, we will implement a function that returns the items in your dataset and then tell detectron2 about this function. For this, we will follow these steps:

1. Register your dataset (i.e., tell detectron2 how to obtain your dataset).

2. Optionally, register metadata for your dataset.

We are going to register it on detectron2. So we are first importing the required functions from chúng tôi which are dataset catalog and meta catalog. We are registering the data and naming it as face_train.

from chúng tôi import DatasetCatalog, MetadataCatalog # register dataset DatasetCatalog.register("face_train", lambda d="train": create_annotation(train_df))

We are also registering the metadata where we are defining the class as a face since we want to detect faces.

# register metadata MetadataCatalog.get("face_train").set(thing_classes=["face"])

Next, let us visualize a few samples from the training set. So for that, we are first importing the visualizer function from detectoron2 and cv2_imshow. In order to visualize the images, we are going to import random. Since we are going to randomly pick the images. So here we are getting the names of the classes from the metadata catalog and we are printing the names.

# for drawing bounding boxes on images from detectron2.utils.visualizer import Visualizer # for displaying an image from google.colab.patches import cv2_imshow # for randomly selecting images import random # get the name of the classes face_metadata = MetadataCatalog.get("face_train") print(face_metadata)

So we are randomly picking five images from the training set and this is done using “random.sample” then we are reading the images using cv2.imread and using the visualizer function. We are going to visualize the bounding boxes using draw_dataset_dict. And also We are plotting the bounding boxes for the faces and finally using “cv2_imshow“. Now we are going to print the image along with these bounding boxes.

# randomly select images for d in random.sample(train_annotation, 5): # read an image img = cv2.imread(d["file_name"]) # create visualizer visualizer = Visualizer(img[:, :, ::-1], metadata=face_metadata, scale=0.5) # draw bounding box on image vis = visualizer.draw_dataset_dict(d) # display an image cv2_imshow(vis.get_image()[:, :, ::-1]

Till now the data preparation part is done in the next section we’ll train our face detection model.

5. Fine-Tuning the model

Now our dataset is ready. Let’s train the model. We will take a pre-trained model and fine-tune it as per our problem. So we’ll be using the retina net pre-trained model which is trained on the coco data set. Since our wider face data set is different from the coco data set will retrain the entire architecture of the pre-trained model as per our problem.

So here we are first importing a few helper functions which are model_zoo.

In order to load the pre-trained model default trainer will be used to train the model and get_cfg which will be used to get the configurations of the pre-trained model.

Next, we are defining the configuration instance and first of all, we are specifying the path of the pre-trained model in this configuration file,  and then we are loading the weight of the “retainnet ” pre-trained model which is trained on the coco detection data set.

# to obtain pretrained models from detectron2 import model_zoo # to train the model from detectron2.engine import DefaultTrainer # set up the config from detectron2.config import get_cfg # interact with os import os # define configure instance cfg = get_cfg() # Get a model specified by relative path under Detectron2’s official configs/ directory. cfg.merge_from_file(model_zoo.get_config_file("COCO-Detection/retinanet_R_50_FPN_1x.yaml")) # load pretrained weights cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCO-Detection/retinanet_R_50_FPN_1x.yaml")

Next, we have to define the name of the train and the test data set in the configuration file and these data sets must be registered in “detectron 2”. We have registered the training data set as face_train and hence we are giving the name here. Currently, we do not want any test data and hence we are keeping this blank.

# List of the dataset names for training. Must be registered in DatasetCatalog cfg.DATASETS.TRAIN = ("face_train",) cfg.DATASETS.TEST = ()

Now, we are defining a few hyperparameters for our model. Our model is defined, and we have changed the hyperparameters, it’s time to train the model.

# no. of images per batch cfg.SOLVER.IMS_PER_BATCH = 2 # set base learning rate cfg.SOLVER.BASE_LR = 0.001 # no. of iterations cfg.SOLVER.MAX_ITER = 1000 # only has one class (face) cfg.MODEL.ROI_HEADS.NUM_CLASSES = 1

So we are creating a directory, which will save the weights for the model, as the training progresses. Exist_score will tell us if the directory already exists or not. Setting this as true will mean that the directory already exists.

Now, using the modified configuration file we are creating a trainer using the default_trainer function, and using resume or load, we can set if we want to start the training from scratch, or resume using the pre-trained weights.

If the resume is set to true, and the last checkpoint exists, it will load the checkpoints, and start training on top of that.

If this is set to false, which is in our case, the training will start from the first iteration. So since we are training the model from the first iteration, we are going to set this resume is equal to false.

The summary of the model will be printed after every iteration, which will include the total loss, the classification loss, regression loss, the time taken to train for a few iterations, the learning rate after that particular iteration, and the maximum memory used.

# create directory to save weights os.makedirs(cfg.OUTPUT_DIR, exist_ok=True) # create a trainer with given config trainer = DefaultTrainer(cfg) # If resume==True, and last checkpoint exists, resume from it, load all checkpointables (eg. optimizer and scheduler) and update iteration counter. # Otherwise, load the model specified by the config (skip all checkpointables) and start from the first iteration. trainer.resume_or_load(resume=False) # train the model trainer.train()

The output of this training model is:-

Source:- Author

All right we can see that now the training is complete, as the training progresses, we can see that the loss keeps decreasing, for every subsequent iteration. If you observe both the classification loss and the regression loss, you will see a decreasing trend here.

To visualize the results of this training more efficiently, and in a better way, we’ll use the tensorboard. Tensorboard provides visualization needed for machine learning and deep learning. We can track and visualize the metrics like loss and accuracy. We can also visualize the parameters learned by the model and so on so.

# Look at training curves in tensorboard: %load_ext tensorboard %tensorboard --logdir output

We are going to use this visualization tool in order to visualize the training of our model.

So let us first visualize the total loss, and here we can see that we have plots for the regression loss and the classification loss. We can see that both for regression as well as for classification the loss keeps decreasing with the increasing number of iterations.

And if we check the plot for the learning rate and how it changes with the number of iterations, we can see that the learning rate keeps increasing with the increasing number of iterations. Similarly, we can visualize other factors as well.

6. Evaluating model performance

Finally, we’ll evaluate the performance of this model on the validation set. And in order to use any data set in detectron2, we must register it first. So we are going to register the validation data set similar to how we register the training data set.

we are going to name this as face_val we are also registering the metadata where we are defining the classes that are present in this data set which is the face.

# register validation dataset DatasetCatalog.register("face_val", lambda d="val": create_annotation(val_df)) # register metadata MetadataCatalog.get("face_val").set(thing_classes=["face"])

Next, we are loading the weights of the model which were saved as “model _final.pth” during the training. And we are keeping the score threshold as 0.8.

# load the final weights cfg.MODEL.WEIGHTS = os.path.join(cfg.OUTPUT_DIR, "model_final.pth") # set the testing threshold for this model cfg.MODEL.RETINANET.SCORE_THRESH_TEST = 0.8 # List of the dataset names for validation. Must be registered in DatasetCatalog cfg.DATASETS.TEST = ("face_val", )

predictor and pass the updated configuration file inside it.

# set up predictor from detectron2.engine import DefaultPredictor # Create a simple end-to-end predictor with the given config that runs on single device for a single input image. predictor = DefaultPredictor(cfg)

Let us now visualize the predictions on a few of the images, from the validation set. So we are going to randomly pick the images from the validation set, and here we are randomly picking three images. We are reading these images and taking the predictions from the predictor function, then creating the visualizations for these images.

So we are drawing the bounding boxes and printing these bounding boxes over the images and plotting the output using the cv2_imshow function. So here you can see that on this image the face is clearly detected with 95 % accuracy.

# create standard annotations for validation data dataset_dicts = create_annotation(val_df) # randomly select images for d in random.sample(dataset_dicts,3): # read an image im = cv2.imread(d["file_name"]) # make predictions outputs = predictor(im) # create visualizer v = Visualizer(im[:, :, ::-1], metadata=face_metadata, scale=0.5) # draw predictions on the image v = v.draw_instance_predictions(outputs["instances"].to("cpu")) # display image cv2_imshow(v.get_image()[:, :, ::-1])

Similarly on this image also we have the face detected with 98 %probability.

Source:- Author

let us now check the performance on the entire validation set. So for that, we are importing the coco evaluator and inference on the data set. These two functions are present within the detectorn2 evaluation module. In order to load the images from the validation set, we are going to use the build_detection_test_loader and we have imported that here as well.

So here we are using the coco evaluator that we imported, and creating the evaluators.

To this function we have to pass the validation data, the cfg file, and using this, the coco evaluator will be able to evaluate the performance on this validation set.

So val_loader is the loader for the validation set and the evaluator is used to evaluate the validation set. So here we have given in our val_loader and evaluator, to the function inference on the data set.

from detectron2.evaluation import COCOEvaluator, inference_on_dataset from chúng tôi import build_detection_test_loader # create a evaluator using COCO metrics evaluator = COCOEvaluator("face_val", cfg, False, output_dir="./output/") # create a loader for test data val_loader = build_detection_test_loader(cfg, "face_val") # runs the model on each image in the test data and produces the results inference_on_dataset(trainer.model, val_loader, evaluator)

Source:- Author

So we have the results here it will print the total inference time taken, and then the model performance. So we got an average precision value of approximately 93 %, for an iou of 0.5 and for iou of 0.75.

We have approximately 70%. So this is how we can build a face detection model that can detect single faces from the images.

Conclusion

In this article, we solved the problem of detecting a single face in the image. Fundamentals of face detection are critical for solving the business challenge and developing the necessary model. When it comes to working with image data, the most difficult task is figuring out how to detect faces from images that can be applied to the model. While working on image data you have to analyze a few tasks such as face detection, bounding box.

I hope the articles helped you understand how to deal with image data, how to detect faces from images, we are going to use this technique, and apply it in a few domains such as the medical, sports analysis domain.

In real-life scenarios, there can be situations when we have multiple faces in a single image so in the next article let us will work on building a model that can detect multiple faces from the images. Hope you enjoyed reading this article on the To Build Face detection Systems.

Thank you.

About the Author

Hi, I am Kajal Kumari. have completed my Master’s from IIT(ISM) Dhanbad in Computer Science & Engineering. As of now, I am working as Machine Learning Engineer in Hyderabad. Here is my Linkedin profile if you want to connect with me.

End Notes

Thanks for reading!

If you want to read my previous blogs, you can read Previous Data Science Blog posts from here.

The media shown in this article is not owned by Analytics Vidhya and are used at the Author’s discretion. 

Related

You're reading Learn How To Build Face Detection System

Learn How To Do Real

This article was published as a part of the Data Science Blogathon

Introduction

OpenCV is an open-source computer vision library that provides privileges to play with different images and video streams and also helps in end-to-end projects like object detection, face detection, object tracking, etc.

CVzone is a computer vision package that makes us easy to run like face detection, hand tracking, pose estimation, etc., and also image processing and other AI functions. At the core, it uses OpenCV and MediaPipe libraries. Check here for more information.

Why Real-Time Background Removal is required?

For many reasons, the background of the video needs to be modified as there are so many other interruptions in the background or the background colour doesn’t suit the person due to which background or the color needs to be modified. So, we use the real-time background replacement technique to substitute the backgrounds and add replace them with the desired content.

Popular background removal techniques

Image clipping path – This technique is used if the subject of the image has sharp edges. All those elements that fall outside the path will be eliminated.

Image cut-out – Here we cut the required region or subject in a frame and remove the background.

Image masking – If the images have frills or fine edges we can use image masking techniques.

Erasing the background – Erasing the background of an image using any different tools

Many famous applications use a background removal technique and replace it with a custom one. Here we are going to implement something similar, but using OpenCV and CVzone.

Let’s start the implementation

Install the required modules.

-- pip install OpenCV-python -- pip install cvzone -- pip install mediapipe

First, let us check if our webcam is working fine.

import cv2 cap = cv2.VideoCapture(0) cap.set(3, 640) cap.set(4, 480) while True: success, img = cap.read() cv2.imshow("Image", img) if cv2.waitKey(1) & 0xFF == ord('q'): break

The above code pops up a window if you have a webcam, Here the frame size is 640 X 480. So we need to take a note here because the background replacing images should be of the same size as the frame, that is 640 X 480.

Webcam output

Now create the folder inside the project directory here, I am creating a folder with the name ‘BackgroundImages’. You can download any images or any number of images and place them in this directory.

The project structure will look like the image given below:

Project structure

Let us write a small piece of code in a separate python file to resize all the images in the folder ‘BackgroundImages’ to 640 X 480.

import cv2 import os for root, subdirs, files in os.walk('D:/pycharmprojects/BackgroundRemover/BackgroundImages'): for f in files: if f.endswith('jpg'): # print(f) img = cv2.imread('D:/pycharmprojects/BackgroundRemover/BackgroundImages/' + f) img = cv2.resize(img, (640, 480)) cv2.imwrite('D:/pycharmprojects/BackgroundRemover/BackgroundImages/'+f, img) print(*["Image", f, "is resized to 640 X 480"])

The above code will read the image (jpg) files in the specified folder and resize all the images to 640 X480 at once.

Output after resizing all the images

Now we are all set to implement the background replacement technique.

Import the required modules

import cv2 import cvzone from cvzone.SelfiSegmentationModule import SelfiSegmentation import os

Here in the above module, ‘SelfiSegmentation’ is used to remove the background of the frame and replace it with our images in the directory.

cap = cv2.VideoCapture(0) cap.set(3, 640) cap.set(4, 480) # cap.set(cv2.CAP_PROP_FPS, 60) segmentor = SelfiSegmentation() fpsReader = cvzone.FPS() # imgBG = cv2.imread("BackgroundImages/3.jpg") listImg = os.listdir("BackgroundImages") imgList = [] for imgPath in listImg: img = cv2.imread(f'BackgroundImages/{imgPath}') imgList.append(img) indexImg = 0

In the above code, we take input from the webcam and also set the frame width to 640 X 480. Then we call SelfiSegmentation() and assign it to a variable called segmentor, and in order to display the frames per second(fps) in the output frames, we use cvzone.FPS() function.

Then we create a list of images present in the BackgroundImages folder and we loop through that list and read each and every image and append it to an empty list. The initial index is set to zero.

while True: success, img = cap.read() # imgOut = segmentor.removeBG(img, (255,0,255), threshold=0.83) imgOut = segmentor.removeBG(img, imgList[indexImg], threshold=0.8) imgStack = cvzone.stackImages([img, imgOut], 2,1) _, imgStack = fpsReader.update(imgStack) print(indexImg) cv2.imshow("image", imgStack) key = cv2.waitKey(1) if key == ord('a'): indexImg -=1 elif key == ord('d'): if indexImg<len(imgList)-1: indexImg +=1 elif key == ord('q'): break

Then we stack the images using cvzone.stackImages, here we will get the output of the background replaced image or frames. Then using a simple if statement we assign keys to change the background. For example, if we have 10 background images, as per the above code we can use key “a” or key “d” to change the background of the frames.

The entire code is given below.

import cv2 import cvzone from cvzone.SelfiSegmentationModule import SelfiSegmentation import os cap = cv2.VideoCapture(0) cap.set(3, 640) cap.set(4, 480) # cap.set(cv2.CAP_PROP_FPS, 60) segmentor = SelfiSegmentation() fpsReader = cvzone.FPS() # imgBG = cv2.imread("BackgroundImages/3.jpg") listImg = os.listdir("BackgroundImages") imgList = [] for imgPath in listImg: img = cv2.imread(f'BackgroundImages/{imgPath}') imgList.append(img) indexImg = 0 while True: success, img = cap.read() # imgOut = segmentor.removeBG(img, (255,0,255), threshold=0.83) imgOut = segmentor.removeBG(img, imgList[indexImg], threshold=0.8) imgStack = cvzone.stackImages([img, imgOut], 2,1) _, imgStack = fpsReader.update(imgStack) print(indexImg) cv2.imshow("image", imgStack) key = cv2.waitKey(1) if key == ord('a'): indexImg -=1 elif key == ord('d'): if indexImg<len(imgList)-1: indexImg +=1 elif key == ord('q'): break

The output screenshots are given below

Output 1 Output 2

The entire code is also available here:

Reference:

My LinkedIn

Thank you

The media shown in this article are not owned by Analytics Vidhya and are used at the Author’s discretion.

Related

How To Build Your Own Cotton

You don’t need to wait for a carnival to satisfy your craving for cotton candy. Instead, build this portable, pocket-size machine to turn granulated sugar into an airy treat.

A DIY cotton-candy machine consists of a small metal container, repurposed lighter parts to provide heat, and a switch-controlled motor to set everything spinning. Slowly pour granulated sugar into the container, and flames from the lighters will melt it. As the motor spins, the liquid sugar will fly out through little holes in the container’s sides, forming thin strands. A paper cylinder placed around the machine will capture them. Once they’ve built up, simply swirl a chopstick around the perimeter to gather the candy and taste your sweet success.

This article originally appeared in the September 2024 issue of Popular Science, under the title “Build Your Own Cotton-Candy Machine.”

Stats

Time: 2 hours

Cost: $26

Difficulty: Medium

Tools

Push pin

Power drill

Soldering iron

Materials

Long-nosed lighter

Torch lighter

Wire

Two-part epoxy

Superglue

Metal stand-off with a screw, washer, and nuts

A small cosmetic aluminum container (found in drugstores) or a metal drink cap

Small project box

DC motor

AA-battery holder

Clay epoxy

Paper, tape, rubber band, and a chopstick

Instructions

To build a system for heating the sugar, first open both lighters. Harvest the large fuel tank, igniter, and hose from the long-nosed lighter and the torch head from the torch lighter.

Use the long fuel hose to connect the fuel tank to the torch head.

For an ignition line, wrap a short length of wire around the metal base of the long- nosed lighter’s igniter and seal it with epoxy.

Push the igniter’s new wire through the torch head— where the torch lighter’s wire previously was. This is the main ignition line.

Connect the main ignition line to the brass part of the torch head. Seal with superglue.

Next, set up the spinning chamber. Epoxy the metal standoff to the shaft of the motor. (When joining two parts together with epoxy, sanding both sides will yield a stronger bond.)

With the push pin, punch holes all the way around the sides of the aluminum container, or drill tiny holes in the metal drink cap. Find the center of the container and drill through it. Add the screws, washers, and bolts to it, and screw it in place on the motor’s standoff.

Solder the battery pack’s terminals to the motor. Since the screw tightens clockwise, make the motor spin counterclockwise to prevent it from unscrewing.

To prepare the project box, plan where you will be placing the fuel valve, igniter, torch head, and spinning chamber. Mark each spot with a marker, and then drill the holes. You can use the photos as a guide.

Inside the cotton-candy machine Sophie Bushwick

Epoxy the motor in place in the box. Glue the battery pack to the outer side of the box. Seal the igniter in place—the end should stick out of the box—with clay epoxy.

Before sealing the torch system in place with the clay epoxy, measure the torch head and aim it at an angle so the flame will touch the near edge of the metal container.

To operate the cotton-candy machine, tape paper into a cylinder that fits around it. Then switch on the motor, squeeze the fuel valve (and hold it in position with a rubber band), and spark the igniter. Let the machine heat for 10 seconds, then place the paper cylinder around it and slowly add the sugar. Collect the candy with a chopstick.

Warning: Take care handling lighters and fuel. The sugar is molten when it comes out, so keep your hands out of the way of hot flying sugar. Also keep hands and paper clear of the open flame—or you might end up making jerky instead of candy.

Meta To Face ‘Accountability’ After 14

Execs from Instagram and Pinterest have been ordered to appear before an inquest into the death of a 14-year-old girl, Molly Russell. Russell took her own life after using the apps to view extensive material on self-harm, depression, and suicide.

It’s the latest development in growing concerns about the impact of social media platforms on the mental health of teenagers, girls especially …

Background

Concerns about the impact of social networks on the mental health of teenagers were crystalized last year, when an internal report carried out by Instagram concluded that it was harmful to as many as 20% of teenage girls using the app. Most worryingly of all, it was shown to increase the risk of suicide.

It can increase anxieties about physical attractiveness, social image, and money, and even increase suicide risk, according to Facebook’s own research […]

For the past three years, Facebook has been conducting studies into how its photo-sharing app affects its millions of young users. Repeatedly, the company’s researchers found that Instagram is harmful for a sizable percentage of them, most notably teenage girls.

“We make body image issues worse for one in three teen girls,” said one slide from 2023, summarizing research about teen girls who experience the issues.

“Teens blame Instagram for increases in the rate of anxiety and depression,” said another slide. “This reaction was unprompted and consistent across all groups.”

Among teens who reported suicidal thoughts, 13% of British users and 6% of American users traced the desire to kill themselves to Instagram, one presentation showed.

Instagram owner Meta said that the report only highlighted the worst-case scenarios, but the company subsequently “paused” its Instagram for Kids project and pledged to make the app healthier for teens.

Apple CEO Tim Cook is among those who have expressed concern about the potential harm technology can do to mental health.

Molly Russell inquest

BBC News reports on the latest development.

Almost five years after she took her own life, the inquest into the death of teenager Molly Russell is due to begin.

Molly, 14, killed herself in 2023 after viewing material about self-harm, suicide and depression, on social media sites such as Instagram and Pinterest […]

In the last six months of her life, Molly used her Instagram account up to 120 times a day, liking more than 11,000 pieces of content. She is thought to have used the image-sharing site Pinterest more than 15,000 times over the same period.

The coroner, Andrew Walker, has already been warned that some of the content is “pretty dreadful” and difficult even for adults to look at for extended periods of time […]

Meta, which owns Instagram, and Pinterest are officially taking part in the inquest, which is due to last two weeks. It will hear evidence from executives from both companies, after they were ordered by the coroner to appear in person.

Meta is likely to be questioned about a number of internal documents revealed by the former employee and whistleblower Frances Haugen. These include research carried out by the company into the impact of the platform on the mental health of young people.

While the inquest is taking place in the UK, many believe the case will receive close attention in the US and elsewhere. Matthew Bergman, a lawyer from the Social Media Victims Law Centre, says that Meta execs being questioned is an important development.

Regardless of the outcome, the fact that Meta senior personnel have been forced to testify in a proceeding like this one is a significant step toward accountability.

It’s not just Meta that is in the spotlight over this issue. An investigation last year revealed how TikTok’s algorithm can send people deeper and deeper into dark places.

Help is available

If you are considering self-harm, or would simply like someone to talk to, there are people ready to help. You do not need to be considering suicide to call.

The 988 Suicide and Crisis Lifeline is available 24 hours a day, seven days a week. You can either phone or text from anywhere in the US. You can also find mental health resources on the organization’s website.

In the UK, the Samaritans are also available 24/7. Call 116-123, or text SHOUT to 85258.

In other countries, Google “Suicide helpline” to find local help.

Photo: Max Bender/Unsplash

FTC: We use income earning auto affiliate links. More.

Learn How To Perform Statistics In Pandas?

Introduction to Pandas Statistics

Pandas Statistics incorporates an enormous number of strategies all in all register elucidating measurements and other related procedures on dataframe. The majority of these are accumulations like total(), mean(), yet some of them, as sumsum(), produce an object of a similar size. As a rule, these techniques take a pivot contention, much the same as ndarray, yet the name or whole number can determine the hub.

Start Your Free Software Development Course

Web development, programming languages, Software testing & others

statistics= dataframe.describe(Value)

where,

describe is used to define the specific row or column of the dataframe.

Value is the value assigned to the statistics on whichever operation has to be performed in that particular row or column.

How to perform statistics in Pandas?

Now we see various examples of how these statistics are performed in different ways in Pandas.

Example #1

To perform Pandas statistics only on the ‘Amount’ column

Code:

from pandas import DataFrame Vehicles = {'Company': ['Mercedes E','Honda City','Corolla Altis','Corolla Altis','BMW X'], 'Amount': [15000,16000,17000,18000,20000], 'Year': [2010,2011,2012,2013,2014] } df = DataFrame(Vehicles, columns= ['Company', 'Amount','Year']) statistics_numeric = df['Amount'].describe() print (statistics_numeric)

Output:

In the above program, we first import pandas, and then we import the dataframe from pandas. After importing the dataframe, we define the dataframe by assigning values to it. Now, we use the Pandas statistics function to describe the amount of each vehicle and give all the possible basic statistics for each vehicle. Finally, the program is implemented, and the result is as shown in the above snapshot.

Example #2

Adding astype(int) to get the integer values of Pandas statistics.

Code:

from pandas import DataFrame Vehicles = {'Company': ['Mercedes E','Honda City','Corolla Altis','Corolla Altis','BMW X'], 'Amount': [15000,16000,17000,18000,20000], 'Year': [2010,2011,2012,2013,2014] } df = DataFrame(Vehicles, columns= ['Company', 'Amount','Year']) statistics_numeric = df['Amount'].describe().astype (int) print (statistics_numeric)

Here, it is similar to the previous program except that we removed all the decimal values in the program. Instead, we produced the amount as integer values for the same column using statistics and described function.

Example #3

Including all the three categories in the Statistics

Code:

from pandas import DataFrame Vehicles = {'Company': ['Mercedes E','Honda City','Corolla Altis','Corolla Altis','BMW X'], 'Amount': [15000,16000,17000,18000,20000], 'Year': [2010,2011,2012,2013,2014] } df = DataFrame(Vehicles, columns= ['Company', 'Amount','Year']) statistics=df.describe(include='all') print (statistics)

Output:

In the above program, when you do not categorize data and include all the rows and columns of the dataframe, the program produces an output consisting of everything, as shown in the above snapshot. Hence, Pandas statistics can be used to figure out the different standards of descriptive statistics.

Example #4

Using different categorical data to produce statistics.

from pandas import DataFrame Vehicles = {'Company': ['Mercedes E','Honda City','Corolla Altis','Corolla Altis','BMW X'], 'Amount': [15000,16000,17000,18000,20000], 'Year': [2010,2011,2012,2013,2014] } df = DataFrame(Vehicles, columns= ['Company', 'Amount','Year']) statistics = df['Company'].describe() print(statistics)

Output:

In the above program, we first import the dataframe from pandas as usual and then define this dataframe and assign values to it. After that, we use the panda’s statistics to describe a different category of the dataframe, which is the company of the vehicle. So, the program is executed, and it shows the unique companies of vehicles, and it is implemented, and finally, the statistics are as shown in the above snapshot.

This capacity gives the mean, sexually transmitted disease, and IQR values. What is more, the work rejects the character segments and given a rundown about numeric sections. ‘incorporate’ is the contention that is utilized to pass fundamental data with respect to what sections should be considered for summing up. Next, takes the rundown of qualities; of course, ‘number’.

The object summarizes the String sections. The number summarizes Numeric sections and all Summarizes all sections together. This capacity gives you a few valuable things all simultaneously. For instance, you will get the three quartiles, mean, check, least, and most extreme qualities and the standard deviation. This is exceptionally valuable, particularly in exploratory information examination.

You can likewise pick explicit percentiles to be remembered for the portray strategy yield by including the percentiles contention and indicating. You can change the number of percentiles you request; however you see fit 4 percentiles are only a model. In the event that your article is non-mathematical, the synopsis measurements will be slightly unique. They will incorporate the tally, recurrence, quantity of kind qualities, and top worth. On the off chance that your article contains both mathematical and non-mathematical qualities, the portray strategy will just incorporate rundown insights into the mathematical qualities.

Conclusion

Hence I would like to conclude by stating that capacities like total(), cumsum() work with both numeric and character (or) string information components with no blunder. Despite the fact that is in practice, character totals are never utilized, by and large, these capacities do not toss any exemption. Also, capacities like abs(), cumprod() toss special cases when the Dataframe contains character or string information in light of the fact that such activities can’t be performed.

Recommended Articles

We hope that this EDUCBA information on “Pandas Statistics” was beneficial to you. You can view EDUCBA’s recommended articles for more information.

Learn How To Add Presets To Lightroom Cc?

Introduction to Lightroom CC

Lightroom CC is a creative image organizer as well as manipulation software that is part of the Adobe family and developed by Adobe Inc. You can find it in the Creative Cloud desktop app subscription of adobe. It has its compatibility with Microsoft Windows, Mac OS, iOS, Android, and tvOS, which makes it easy to use software and more handy. It has many good features such as saving, importing, viewing, tagging, editing, organizing, and many other features that work to enhance digital image properties. In addition, there are many functions that are used for performing a number of tasks, such as white balance, tone curve, HSL, Presence, color grading, and so on.

So let us go through the very interesting analysis of this software for understanding more things about it. Here I will tell you something about presets, which is an important part of their work.

Start Your Free Design Course

3D animation, modelling, simulation, game development & others

How to add presets to Lightroom CC?

Many times, we find a number of presets work effectively to enhance digital image quality and appearance, and for that, we want to use external presets with our edited image. So for that, we need to add preset to Lightroom, and I am going to tell you how you can do it in step by step manner.

I will first import an image here for which we are going to add preset here. So go to the Import Photos and Video option of a scroll-down list of File menu.

Now there are two options for installing preset into this software. One is ‘Lightroom Develop Presets .’ The next one is ‘Other Lightroom Presets.’ Develop Presets is for the LRTemplates file and Other Presets are for the XML file format. So, according to your preset installation file, you can go with anyone.

And it will take us to the Settings folder of Lightroom.

Open this folder, and you can paste your LRTemplate file here.

And it will take us to the Lightroom folder.

And then, open the User presets folder, and in this folder, you can paste your XML file.

And after installing the preset by any of the two methods, you have to restart Lightroom to use installed presets.

Advantages of Lightroom CC

The very first thing is you can have it on your mobile for doing a great job in photo editing with lots of effective presets, and the best thing is that the mobile version of Lightroom is for free to use.

If you go with the desktop version of it, you will benefit from online storage of your work, which is 1TB. In addition, you can have it in Cloud storage form.

Once you sync your photos with the cloud server, it allows you to edit them on your mobile devices, which is quite a good feature.

You can edit raw files with a smooth workflow environment that means you can directly import your raw photos from the camera and start editing them on your tablet or mobile devices.

Now let me tell you how you can create your own presets with our desired parametrical settings.

Create your own presets in Lightroom CC

Same as we use presets of external sources for getting our desired manipulation for our image, we can create our own preset that we can use with our other images. So let me tell you about how you can do this.

And make changes in the values of White Balance, Shadow, and Highlights, and so on.

And name it as per your choice. I will name it as White Balance.

So you can go with presets to try something new.

Conclusion

I am sure it was good for you to go through this article to understand what Lightroom CC is and how it uses presets to add more details to our photos. Now you also know that Lightroom CC is compatible with mobile devices, and you can sync your photos for editing them on both mobile and desktop.

Recommended Articles

This is a guide to Lightroom CC. Here we discuss Lightroom CC and how it uses presets to add more details to our photos. You may also have a look at the following articles to learn more –

Update the detailed information about Learn How To Build Face Detection System on the Moimoishop.com website. We hope the article's content will meet your needs, and we will regularly update the information to provide you with the fastest and most accurate information. Have a great day!