Together we will walk through a project that focuses on building a feedback loop with Keen as our data store and query engine, and Google Cloud’s Natural Language for pre-processing. This application will ingest a feedback response, send it through a serverless function which will enrich feedback data with a score. This score will consist of a magnitude and sentiment analysis done by Google’s machine learning “Natural Language Processing” API. This can prove to be colossally useful in cases where a business is aiming to understand if customers are enjoying their experience.
That experience could be the result of a cool new software application, maybe a healthcare app that enables patients to tell you about their latest office visit (the example we’ve built below). It’s possible to track a particular customer’s progress along some series of onboarding steps, or maybe you’re looking to onboard new hires and review the impact of training material…the possibilities are endless.
Building out analytics can be easy with something as simple as a ‘like counter’ but when you want to scale your application and need to deploy a server a host and update a database, you could quickly fill up the next 6 months of your calendar. By using Keen you can achieve both at once. Keen aims for “High Velocity” and automates the workflow for us and we’re able to launch this proof of concept in hours.
Getting Started
You will need an account for both Keen and Google Cloud Platform to accomplish this walkthrough but the image examples should provide a feel for what it takes to build this exact project or a similar one on your application. While this is a 10 minute read you’ll likely want to spend time with the signup process and becoming familiar with Google Cloud Console and Keen if it’s your first time. Contact sales@keen.io if you would like a live demonstration.
{
"keen": {
"timestamp": "2019-03-19T21:11:57.775Z",
"created_at": "2019-03-19T21:11:57.775Z",
"id": "5c915b1dfee46e0001ae176f"
},
"magnitude": 0.800000011920929,
"score": 0.4000000059604645,
"text": "Dr. Keen is the bees knees! I feel one-hundred percent better"
}
In Keen’s Explorer you can prototype a Dashboard based on the given sentiment score. Sentiment is an analysis by Google’s Natural Language API that attempts to determine the overall attitude (positive or negative) and is represented by numerical score and magnitude values. Positive scores are above zero, and a negative denotes those scores below zero. The scores range from 1 to -1.
What you will need before you start:
Request a Demo and sign up for Keen to create your first project. Complete the onboard process and then you’ll locate your Project ID and your Write Key in the “Access” tab. You’ll want to take note for customizing credentials for complete access.
Sign up for Google Cloud – cloud.google.com and be sure to enable the NLP API.
Getting Started with Google Cloud
We need to enable a couple of APIs in our Google Cloud Platform project: Cloud Functions API and Cloud Natural Language API. From the Cloud Console, click on the Main Menu in the Top Left corner and navigate to Menu → APIs and Services → Dashboard as shown below:
Click on ‘Enable APIs and Services’. This will bring up a screen as shown below, where you can enter Cloud Functions in the search field and enable it.
Repeat the same process i.e. enable the Google Cloud Natural Language API.
Basic html, css, javascript knowledge is expected, but it’s likely that if you follow these 3 steps verbatim you will have a sophisticated feedback loop that incorporates machine learning and Keen’s analytics API. I will do my best to keep this article updated but do submit questions to stack overflow and give back to the community whenever you arrive at a solution.
3 Steps To Build our NLP Feedback Form
- Fork the Github Repo Fork this Github Repo: https://github.com/JAC-Keen/keen-nlp-function
- Stand up a cloud function and update your credentials: console.cloud.google.com
- Submit feedback the login to your Keen project’s console: https://keen.io/projects/
Step 1
Fork the Github Repo https://github.com/JAC-Keen/keen-nlp-function and host it locally.
You can find a live feedback form here https://jac-keen.github.io/keen-nlp-function/
Have your editor of choice and open the project so that you can make edits in steps 2 and 3. You will need to replace the Keen credentials to see the final output and stream data to your project going forward. This will be found on lines 101 and 102 of your index.html file.
Remember to locate your Project ID and your Write Key in the “Access” tab of Keen.
Step 2
Next is setting up our ‘Cloud Function’ that adds the Sentiment Analysis to our string of feedback. But do locate your Function’s endpoint which you’ll find under the tab labeled ‘Trigger’.
Inside of the “Source” tab, you can inject and save both index.js and package.json files of code. Google cloud will invoke this function for us calling the .analyzeSentiment method every time a request is made to your cloud function endpoint.
**Remember to replace the project id and your write key specific to your Keen credentials other wise you will not be able to produce an output.
Index.js for your cloud function.
const Language = require('@google-cloud/language');
// Instantiates a client
const language = new Language.LanguageServiceClient();
// Post Message, Score, Magnitude to Keen
const KeenTracking = require('keen-tracking');
// Create your Client Credentials which can be found inside of your 'Access' tab
const client = new KeenTracking({
projectId: 'Replace with your project id',
writeKey: 'Replace with your write key'
});
exports.anthemcNLP = (req, res) => {
res.set('Access-Control-Allow-Origin', '*');
/* this handles CORS preflight check */
if (req.method === 'OPTIONS') {
// Send response to OPTIONS requests
res.set('Access-Control-Allow-Methods', 'GET');
res.set('Access-Control-Allow-Headers', 'Content-Type');
res.set('Access-Control-Max-Age', '3600');
res.status(204).send('');
return;
}
// this handles cors for the regular request
res.set('Access-Control-Allow-Origin', '*');
if (req.body.review_text == undefined) {
res.status(400).send("No review text provided");
return;
}
// Ingest the review_text and assign to the constant 'document'
const document = {
content: req.body.review_text,
type: 'PLAIN_TEXT',
};
// Google's NLP Language API request is analyzing the text and assigning Sentiment, Magnitude
language
.analyzeSentiment({
document: document
})
.then(results => {
const sentiment = results[0].documentSentiment;
console.log(`Document sentiment:`);
console.log(` Score: ${sentiment.score}`);
console.log(` Magnitude: ${sentiment.magnitude}`);
res.setHeader('Content-Type', 'application/json');
var resultObj = {};
resultObj.score = sentiment.score;
resultObj.magnitude = sentiment.magnitude;
// Record the event with sentiment and magnitude score to Keen
client
.recordEvent('FormResult', {
text: req.body.review_text,
score: sentiment.score,
magnitude: sentiment.magnitude
})
.then(results => {
res.setHeader('Content-Type', 'application/json');
res.end(JSON.stringify({ message: "ok" }));
return;
})
.catch(err => {
res.status(500).send(`Error in invoking Keen API: ${err}`);
return;
});
})
.catch(err => {
res.status(500).send(`Error in invoking NLP API: ${err}`);
return;
});
}
package.json will save the necessary dependencies
{
"name": "anthemcNLP",
"version": "0.0.1",
"dependencies": {
"@google-cloud/language": "^2.0.0",
"keen-tracking": "^4.0.15"
}
}
Now that we have the files set up we can test our cloud function.
You’ll be given a set of blank curly braces. You can follow the review text format and choose to pass and test any string you’d like:
Ex. {“review_text”: “Feedback forms are simple!”}
It will return an ok if it’s setup properly or an error:
Now that you’ve seen a successful output we’ll want to make sure that your front end event can be set up to hit the correct api endpoint.
In line 114 of your index.html file you will need to alter server endpoint url to match your cloud function url.
You can locate your cloud function endpoint within your cloud console. Select ‘Functions’ from the top left menu. (left side of the image)
The default view will inside the “General” tab. Next toggle to the “Trigger” option.
(pictured top right) and your URL is what you’ll need to input to your markup page.
**Remember to use your URL and not the one pictured.
Step 3
Now that you’ve made the necessary changes your entire feedback loop should be complete.
Enter a string into your front end feedback page, press submit. The final step is to head over to your Keen project https://keen.io/projects/YourKeenProjectID to view the results. Under the ‘Streams’ tab, you should see a similar JSON blob.
If you come across any errors don’t panic, just open up your console log and inspect tools in your browser. You can likely retrace the steps above and find an answer. If issues persist just type any error messages into stack overflow and begin a thread.
Now that you’ve seen how to implement this feedback loop you’ll likely want to use the sentiment analysis and magnitude as insight. You’ve preprocessed customer feedback and it would be great to separate this information by an over/under query for positive and negative reviews. If your software development team is interested in similar projects or seeking out analytics to showcase to customers get in touch https://keen.io/contact/
In our next tutorial, we will cover prototyping queries and charts inside of the Keen Explorer. We will take this as far as creating an embedding a visualization website for customers to access.
