Cloudreach Blog

Catch the latest stuff

Eye of Horus

New Cloud Order 5th September 2017
eye of horus cloudreach - GCP Chatbot

Hackathon theme and rules

This hackathon was the third iteration of Cloudreach’s .create() hackathon. We were tasked with creating a GCP chatbot that could answer technical questions posed to it by the user.  The questions were then to be run against public GCP BigQuery datasets to search for answers. Our team was composed of Carlos Martell (Core Operations Analyst), Asser Hassan (Core Operations Engineer Lead), and Charles Gravlee (Cloud Systems Developer). Carlos and Asser work together in Vancouver and were able to meet up at Asser’s apartment to collaborate in person, while Charles was in Atlanta and connected with them as much as possible over Slack and Hangouts. The time zone differences actually worked in our favor in that it allowed for longer working hours for the team where some would work while the others were asleep, and vice versa. Our team actually came into the project with a head start since Asser, who is Google certified and a Chatops evangelist, and Charles had previous GCP work experience.

Eye of Horus – The inception

Once we reviewed the topic and details of the hackathon we set about doing research on the services offered in GCP that would help us accomplish our goal. We looked at App Engine and Cloud Functions to service the frontend requests and began investigating the layout of the Stack Overflow and GitHub public BigQuery datasets to determine which fields would benefit us most in our queries. Our conclusions at the end of this research were to use App Engine running python as the frontend and Stack Overflow’s dataset as the source. We chose App Engine over Cloud Functions because Cloud Functions only supports JavaScript and Python is the favourite Cloudreachers’ weapon of choice. We chose Stack Overflow only, instead of a hybrid with GitHub, because Stack Overflow provided additional fields which helped us rate the results and present the best answers to the user.

Creating a GCP Chatbot

User: @eye_of_horus Help!

Eye of Horus: Gotcha. I will try my best to assist. What do you need to know?

User: @eye_of_horus create IAM role

Eye of Horus: Here is what I found for “create IAM role”:

How can I create IAM Roles for Amazon EC2? (Score: 1)

AWS EC2 instance create via Ansible IAM Roles instance_profile_name UnauthorizedOperation: Error (Score: 8)

User: /stackoverflow create IAM role

Eye of Horus:  Here is what I found for “create IAM role”:

How can I create IAM Roles for Amazon EC2? (Score: 1)

AWS EC2 instance create via Ansible IAM Roles instance_profile_name UnauthorizedOperation: Error (Score: 8)

  • https://stackoverflow.com/questions/24401846

Gif of the bot in use: http://imgur.com/a/13xMj

Inner workings of creating a GCP Chatbot

Api.ai

This tool served as the conversation handler that would receive questions sent to @eye_of_horus. Different scenarios were created in order to not limit the user to specific syntax in order to receive a response. Starting the conversation was a simple as saying “help”.

GCP App Engine

We deployed an app in App Engine that handled the heavy lifting of the api.ai conversations.  Api.ai will send a formatted query to App Engine and it will return a response. App Engine can also answer questions directly from the user if /stackoverflow is used before the question. This method is effective using keywords, but does not handle natural language questions as effectively as the api.ai route.

GCP BigQuery

BigQuery is where the public Stack Overflow dataset is located and the source for all of our answers. We searched the title of Stack Overflow posts based on the query passed to the bot and ordered the results based on “accepted_answer_id IS NOT NULL DESC, score DESC“.  This produced the most accurate results.

What we learned

In creating this GCP ChatBot we learned that BigQuery datasets are extremely large and in order to use it effectively you need to research the dataset’s schema beforehand. This allows you to return useful results in a timely manner at the lowest cost.

Using Google’s owned API.AI provided us a lot more control and flexibility compared to running our bot directly on Google App Engine. The ability to create and customize intents, set up contexts, as well as the machine learning capabilities gave us a really solid starting block. Here’s to hoping our submission wins the final .create() hackathon.

 

 

For more hackathon related posts, click here

What will you create?