#originalfaces | Explore Tumblr posts and blogs

mbaljeetsingh · 7 years ago

Text

Overlay Glasses/Masks on Avatars with Vue.js and Cloudinary

Deep Learning, a subset of machine learning, helps break down tasks in ways that makes all kinds of machine assists seem possible. With deep learning, it is now possible to do image recognition by machines. Instead of hand-coding software programs with a specific set of instructions to accomplish a particular task, the machine is trained using large amounts of data and algorithms that give it the ability to learn how to perform the task.

Even though you might posses the technical know-how to train models to identify images, and perform some next level facial attribute detection, you now can leverage existing cognitive services to do your bidding.

What We'll Build

In this article, we’ll build a demo app with Cloudinary and Microsoft Cognitive service in which users can test different glasses and masks to see it looks on them before making a purchase.

Applying deep learning to enhance our business? Yes, we are!

What is Cloudinary?

Cloudinary is a cloud platform that provides solutions for image and video management, including server or client-side upload, a huge range of on-the-fly image and video manipulation options, quick content delivery network (CDN) delivery and powerful asset management options.

Cloudinary enables web and mobile developers to address all of their media management needs with simple bits of code in their favorite programming languages or frameworks, leaving them free to focus primarily on their own product's value proposition.

Let’s Get Started

Step 1: Create a Cloudinary Account

Once you are signed up, you will be redirected to the dashboard where you can get your credentials.

Take note of your Cloud name, API Key and API Secret

Step 2: Set Up A Node Server

Initialize a package.json file:

npm init

Install the following modules:

npm install express connect-multiparty cloudinary cors body-parser --save

express: We need this module for our API routes connect-multiparty: Needed for parsing http requests with content-type multipart/form-data cloudinary: Node SDK for Cloudinary body-parser: Needed for attaching the request body on express’s req object cors: Needed for enabling CORS

Step 3: Activate Advanced Facial Attributes Detection Add-on

Go to the dashboard add-ons section. Click on Rekognition Celebrity Detection Add-on and select the Free Plan.

Note: You can change to other plans as your usage increases.

Step 4: Identify Facial Attributes

Create a server.js file in your root directory. Require the dependencies we installed:

const express = require('express'); const app = express(); const multipart = require('connect-multiparty'); const cloudinary = require('cloudinary'); const cors = require('cors'); const bodyParser = require('body-parser'); app.use(bodyParser.json()); app.use(bodyParser.urlencoded({ extended: true })); app.use(cors()); const multipartMiddleware = multipart();

Next, configure Cloudinary:

cloudinary.config({ cloud_name: 'xxxxxxxx', api_key: 'xxxxxxxx', api_secret: 'xxxxxxx' });

Replace xxxxxx with the real values from your dashboard.

Add the route for uploading. Let’s make the route /upload.

app.post('/upload', multipartMiddleware, function(req, res) { // Upload to Cloudinary cloudinary.v2.uploader.upload(req.files.image.path, function(error, result) { res.json(result); }, // Specify Transformation & Facial Attributes Detection { transformation: [ { width: 700, radius: "max", crop: "scale" }, { flags: "region_relative", gravity: "adv_eyes", overlay: req.body.item, width: "1.7" } ] });

Quickly take a look at the overlay parameter. It takes in a value of req.body.item. In this app, the values are either glasses or harlequinmask.

Note: I uploaded two photos to my Cloudinary account and made sure they were renamed glasses and harlequinmask. These are the two images we will use as overlays in this app.

harlequinmask and glasses respectively. Go ahead and upload them to your account too.

The Advanced Facial Attribute Detection add-on detects specific facial attributes, including the exact position of the eyes of each face in a photo. Based on this information, Cloudinary can position overlays on top of all the detected eye pairs in an image.

To smartly overlay the glasses or harlequinmask on top of the detected eye pairs in the image, the user uploads, the overlay parameter is set to the ID of the harlequinmask or glasses image and the gravity parameter is set to adv_eyes. We also set the _regionrelative flag together with a 1.7 width to scale the overlay to 170 percent of the width of the detected eyes, and resize the image to an oval thumbnail with a width of 700 pixels.

Once a user makes a POST request to the /upload route, the route grabs the image file from the HTTP request, uploads to Cloudinary, identifies the pair of eyes and overlays them with whatever option the user chooses (either glasses or harlequinmask) and returns the right URL.

Note: The Advanced Facial Attribute Detection add-on is an integrated face detection solution that utilizes Microsoft Cognitive Services. Microsoft Cognitive Services provides high precision face location detection with state-of-the-art, cloud-based algorithms that can detect up to 64 human faces in an image. The detected faces are returned with rectangles (left, top, width and height) indicating the location of faces in the image in pixels, the exact position details of the eyes, mouth, eyebrows, nose and lips, as well as a series of face-related attributes from each face, such as pose, gender and age.

Test the functionality with Postman.

Step 5: Build the Frontend

We’ll use the progressive framework, [Vue.js] to quickly flesh out the frontend. Let’s get started by installing the CLI:

npm install -g vue-cli

Next, create a simple Vue project using the Vue CLI tool we installed:

vue init simple productshowcase

Inside the productshowcase directory, create an index.html file and add the following code to it:

<!DOCTYPE html> <html> <head> <title>Welcome to Vue</title> <script src="http://ift.tt/2nlicUC;></script> <script src="http://ift.tt/2oYI3XH;></script> <link rel="stylesheet" href="http://ift.tt/2cYLUyN; integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous"> </head> <body> <div id="app"> <div class="container" style="margin-top: 3%; margin-left: 2%;"> <div class="col-md-6"> <div> <div class="col-md-6"> <img src="http://res.cloudinary.com/unicodeveloper/image/upload/v1505797868/glasses.png" width="200" height="100" /> <span> Glasses </span> </div> <div class="col-md-6"> <img src="http://ift.tt/2y6KlrH; width="200" height="100" /> <span> Harlequin Mask </span> </div> </div> <hr /> <form enctype="multipart/form-data" @submit.prevent="onSubmit"> <div class="form-group"> <select class="form-control" name="item" v-model="model.item"> <option disabled value="">Choose an item</option> <option value="glasses"> Glasses </option> <option value="harlequinmask"> Harlequin Mask </option> </select> </div> <div class="form-group"> <label for="">File:</label> <input type="file" class="form-control" accept="image/*" name="image" v-on:change="upload($event.target.files)"> </div> <div class="form-group"> <button class="btn btn-primary" >Upload</button> </div> </form> </div> <div class="col-md-4"> <div class="col-md-6" style="margin-top: 20%;"> <img id="originalface" class="img-responsive" alt="" width="600" height="600"> </div> <div class="col-md-6" style="margin-top: 20%;"> <img :src="maskedface" class="img-responsive" alt="" width="400" height="400"> </div> </div> </div> </div> <script> new Vue({ el: '#app', data: function() { return { model: { text: '', image: null, item: '' }, maskedface: null, loading: '', } }, methods: { upload: function(files) { this.model.image = files[0] this.showPreview(files[0]); }, showPreview: function(file) { var reader = new FileReader(); reader.onload = function (e) { document.getElementById("originalface").src= e.target.result; }; // read the image file as a data URL. reader.readAsDataURL(file); }, onSubmit: function() { // Assemble form data const formData = new FormData() formData.append('image', this.model.image); formData.append('item', this.model.item); this.loading = "Processing....Please be patient." // Post to server axios.post('http://localhost:3333/upload', formData) .then(res => { // Update UI this.maskedface = res.data.url this.loading = '' }) } } }) </script> </body> </html>

Now, run the app.

What’s going on here? Don’t be scared, let’s step through the code. First we have a form for uploading of images.

<form enctype="multipart/form-data" @submit.prevent="onSubmit"> <div class="form-group"> <select class="form-control" name="item" v-model="model.item"> <option disabled value="">Choose an item</option> <option value="glasses"> Glasses </option> <option value="harlequinmask"> Harlequin Mask </option> </select> </div> <div class="form-group"> <label for="">File:</label> <input type="file" class="form-control" accept="image/*" name="image" v-on:change="upload($event.target.files)"> </div> <div class="form-group"> <button class="btn btn-primary" >Upload</button> </div> </form>

We bind the upload form to an upload event handler. There is a change event attached to the select file button. Once a user selects a file, the showPreview method called in the Vue instance below is invoked. This method shows a thumbnail preview of the image about to be uploaded.

Thumbnail preview of the image about to be uploaded.

Check out the methods, model and data properties on our Vue instance.

new Vue({ el: '#app', data: function() { return { model: { text: '', image: null, item: '' }, maskedface: null, loading: '', } }, methods: { upload: function(files) { this.model.image = files[0] this.showPreview(files[0]); }, showPreview: function(file) { var reader = new FileReader(); reader.onload = function (e) { document.getElementById("originalface").src= e.target.result; }; // read the image file as a data URL. reader.readAsDataURL(file); }, onSubmit: function() { // Assemble form data const formData = new FormData() formData.append('image', this.model.image); formData.append('item', this.model.item); this.loading = "Processing....Please be patient." // Post to server axios.post('http://localhost:3333/upload', formData) .then(res => { // Update UI this.maskedface = res.data.url this.loading = '' }) } } })

When the form is submitted, it calls the onSubmit function in our Vue method. The onSubmit method then makes a post request to the backend and returns data back to the frontend.

The data returned is the modified image with the overlay. And this reflects on the UI.

Harlequin Mask selected and an Image of Rihanna uploaded.

Glasses selected and an Image of Christian Nwamba, a.k.a codebeast, uploaded!

Feel free to check out the source code here.

Conclusion

We just performed a facial attribute detection together with an Image overlay transformation with Cloudinary. The options are limitless as to what you can do with the information in this tutorial.

Go forth and enhance your business with products that users will love. And Oh you don’t have to spend time building them, Cloudinary’s got you!

via Scotch.io http://ift.tt/2g9Rm43

#IFTTT #Feedly

0 notes