How I integrated Keen IO and Ducksboard

Michelle here again. In my previous blog post, I described how I used Keen IO’s analysis query builder to answer questions about the health of our business. In fact there are about 15 queries I run throughout the day to monitor our progress. Queries such as the number of events posted in the past seven days and counts for each of the analysis methods being used.

As an example, here’s a select unique HTTPS query which gives me a list of users who logged in today:

https://api.keen.io/3.0/projects/<projID>/queries/select_unique?api_key=<key>&event_collection=login&target_property=user.email&timeframe=today

Because Keen IO’s analysis capabilities are exposed via a REST API like this, you can use them programmatically: in custom reports, workflows, dashboards, or a metrics tab in your mobile app.

My goal was simply to use the analysis API to display usage stats in our office. We already had a Ducksboard dashboard displaying our site traffic and social media growth, so I simply added a few widgets to display custom stats from our backend. I made a little ruby program that runs through a list of my Keen IO saved queries and pushes the results to the ducksboard widgets.

I’d thought I’d share some of my code here in case others find it useful. I also put it on github. Note: I just started learning Ruby and this is the first code I have ever shared… be nice!

My program runs through an array of about twenty different queries (the code sample below shows two of them).

require 'rubygems'
require 'net/http'
require 'uri'
require 'json'
require 'date'


#==========================================
# This program takes Keen IO query URLS, fetches results from Keen IO, then pushes them to existing Ducksboard widgets. 
# It also outputs results to Terminal.
# Each query requires the following information
# 1. Description of the Query.
# 2. Query URL
# 3. Required Ducksboard Formatting (number, series, list)
# 4. Ducksboard Widget ID. You need to log into Ducksboard to get this for each widget. 
#==========================================



#Ducksboard API Key. Get this from ducksboard. The program assumes you are only sending data to a single ducksboard account.
dbkey = "abcdefghijklmnopqrtsuv"


#List of Queries. Add new ones to the bottom. This is the only place you need to make updates for new queries (besides creating the ducksboard widget). 
queries = [{
          "description" => "Count of Unique Orgs with Analysis Queries",
          "keen_query_url" => "https://api.keen.io/3.0/projects/projID/saved_queries/engaged_devs/result?api_key=key",
          "format" => "numeric",
          "db_widget_ID" => 84409
          },
          {
          "description" => "Count of unique logins (all time)",
          "keen_query_url" => "https://api.keen.io/3.0/projects/projID/saved_queries/unique_logins_alltime/result?api_key=key",
          "format" => "numeric",
          "db_widget_ID" => 68658
          }]

Here is a function I made which performs a simple HTTP request to get the query result from api.keen.io.

def get_keen_value(keen_query_url)

   uri = URI.parse(keen_query_url)
   http = Net::HTTP.new(uri.host, uri.port)
   http.use_ssl = true
   http.verify_mode = OpenSSL::SSL::VERIFY_NONE

   response = http.start() {|http|
     response = http.get(uri.request_uri)
   }

   keenresult = JSON.parse(response.body)
   result = keenresult['result']

  return result
end

This second function formats the result data based on query type, then pushes it to ducksboard.

def send_to_ducksboard(db_widget_ID, dbkey, value, format)

   uri = URI.parse("https://push.ducksboard.com/v/#{db_widget_ID}")
   http = Net::HTTP.new(uri.host, uri.port)
   http.use_ssl = true
   request = Net::HTTP::Post.new(uri.request_uri)
   request.basic_auth(dbkey, "password")

   if format == "numeric"

     db_value = {"value" => value}

   elsif format == "series"
     series_array = value
     final_array =[]

     series_array.each {|x| 
           keentimestamp = DateTime.parse x['timeframe']['start']
           unixtime = keentimestamp.to_time.to_i
           keenvalue = x['value']

           newhash = {:timestamp => unixtime,
                      :value => keenvalue}

           final_array << newhash
           }

     db_value = final_array

   elsif format == "list"
     list_array = value
     leader_array = []

     list_array.each {|i| 
           leaderboard_row = {:name => i,:values => [],}
           leader_array << leaderboard_row
           }

     db_value = {:value => {:board => leader_array}}

   else
     print "error - specify a valid format"

   end

   request.body = db_value.to_json
   response = http.request(request)

 return response
end

This code loops through all the queries, gets results, prints them to terminal, and sends them to ducksboard.

queries.each do |query|
 puts "========================================"
 puts query['description']
 puts keen_result = get_keen_value(query['keen_query_url'])
 unless query['db_widget_ID'] == "none"
 send_to_ducksboard(query['db_widget_ID'], dbkey, keen_result, query['format'])
 end
end

Final result? Our dashboard gives us a great snapshot of recent user activity. Note: this screenshot is a bit outdated. Want to see goofy pics of our team and make that facebook fan number go up? Here you go.

====

This post is Part 3 in Michelle’s Analytics Showcase. Here are some links to other posts in the series:

Part 1: Logins

Part 2: Filtering & Saved Queries