Interactive app shows the power of machine learning in the cloud

Paul Chin 25th May 2018
Machine Learning

One of our Cloudy Values is to “Be One Step Ahead”. That means staying on top of the latest tools, technologies, and architectures to make application development faster and more cost-effective. Last year, one of our customers tasked us with creating a technology demo to enhance their booth at a trade show. The demo needed to stop people in their tracks and show off the power of the latest cloud technologies.

 

Planning application development

We decided to take advantage of Azure Cognitive Services, Azure App Service, and Azure Cloud Functions to build a serverless web app. Azure Cognitive Services are a set of APIs that allow us to use various machine learning models trained by Microsoft. Azure App Services and Cloud Functions allow us to create a web app and API without worrying about managing servers.

Creating an engaging experience at a trade show requires a high level of interaction and a reason to pull other people in. Our application was able to do both of those things and allowed the presenters at the booth to have extended conversations with attendees.

The Azure Face API makes it easy to implement facial recognition. Instead of collecting training data and creating our own face recognition models from scratch, we let Azure do all the work. Offloading this work to Azure allows us to spend more time building the application and adding features. With these machine learning APIs, we we able to launch much faster than if we had tried to implement facial recognition ourselves.

The application we built is called Duckface, architected by Blaine Price, a Cloud Software Engineer.

 

What is Duckface?

Duckface is a fast-paced game where users are directed to make faces that display an emotion like ‘Happy’, ‘Anger’, or ‘Surprise’. Each participant’s face is scored based on the confidence levels reported by the Face API for that round. Since the Face API can detect up to 64 faces in a single frame, you can play the game with a group of people and see who is the best at making faces. When the game ends, it displays screenshots of everyone’s  attempts, which is always good for a laugh.

Behind the scenes the application takes photos of players using a webcam and sends them to an Azure Cloud Function to be processed by the Face API. All of this is done within the browser. Duckface is tons of fun. It never fails to get a crowd of people excited about facial recognition in the browser.

Give Duckface a try for yourself here.

 

The practical applications

While the Duckface demo is a lot of fun, the technology definitely has practical applications. We can quickly create a web application that runs in any web browser to visually authenticate visitors to your office, or recognize objects on an assembly line to detect manufacturing anomalies. Amazon also provides a similar API, called AWS Rekognition, so there’s no fear of vendor lock-in with these technologies. We are currently making another version of the backend with Lambda functions, so the front end code remains untouched.

The future of machine learning is here and it’s just an HTTP call away…