Announcing New Docs for Keen IO

We’re excited to announce the release of our new Keen API Documentation. We’ve updated both the content and design of our documentation to make it even easier for you to use the Keen API for collecting, querying, and visualizing your data.

Our new documentation includes:

API Reference: Look up all API resources here in our three-pane API Reference, complete with code samples in cURL, Javascript, Ruby, Python, PHP, Java, and .NET.    

Data Collection, Data Analysis, and Data Visualization: These newly designed overview pages give a snapshot of each of these areas, with quick links to take you to the right resources.    

Guides: Find how-to guides, recipes, and deep-dives into areas such as building funnels, conversion analysis, and user behavior metrics. We’ll be adding lots more guides here to help you make the most of using Keen and to get the maximum value from your data. Stay tuned!    

Quick-Start Guide: If you’re not already a user of Keen, you can get started here. You can also select an SDK from the options outlined on our SDK page, sorted by collection, analysis, and visualization.    

Integrations: Our many integrations with partners such as Stripe, SendGrid, and Runscope are featured here, with step-by-step instructions.

We can’t wait for you to get started using our new documentation and we’d love to get your feedback! Please send your comments to team@keen.io or chat with us on Slack!

Nahid Samsami

Product at Keen. Cat Advocate. Also known as Hidi.

Introducing Analytics Tracking for Arduino

We’ve heard the clamoring and finally we’re proud to announce that we have an Arduino library to send events to Keen IO! If you want to check out the code, its all open sourced here.

To get the creative ideas flowing, I started a sample project using this library to create a dashboard that tracks motion detection from a PIR sensor. The full code for the dashboard and Arduino snippet live here.

Activity Tracker

What are we building?

Have you ever wondered how active you are throughout the day, or if your cats are running around all over your desk at night? I have! What we will build here is a motion sensor hooked up to an Arduino Yún board that sends event data to Keen so we can display it on a nice dashboard.

Components Used

Setting up the Arduino Example

So, Keen IO requires SSL to work, and currently, the Yún is the only Arduino board that supports it. And, to make things even more fun, you have to do a little work with security certs to make it work. There’s a nice write-up on how to do that here.

Once the Yún is configured with the new certificate, it’s time to run the example code to make sure you can send events to Keen IO. One small caveat to the built-in example, since I am programming the board over wifi, I had to use Console instead of Serial to see debug output.

#include {Bridge.h}
#include {ApiClient.h}
#include {KeenClient.h}
#include {Console.h}

KeenClient keen;

void setup() {
  pinMode(13, OUTPUT);
  digitalWrite(13, LOW);
  Console.begin();
  digitalWrite(13, HIGH);

  Serial.begin();

  while (!Console);
  Console.println("Started!");
}

void loop() {
  keen.setApiVersion(F("3.0"));
  keen.setProjectId(F("YOUR_PROJECT_ID"));
  keen.setWriteKey(F("YOUR_WRITE_KEY"));

  keen.addEvent("motion_detections", "{\"cat\": 1}");
  keen.printRequest();

  while (keen.available()) {
    char c = keen.read();
    Console.print(c);
  }

  Console.println();
  Console.flush();

  delay(2000);
}

This code will boot up on the Yún, and then send an event to the motion_detections collection associated to your Keen account. If you’re programming it through the USB cable, the Serial object will be what you want to see debug output.

Tracking Motion

Before we write more code, we have to hook up the PIR sensor to the Arduino.

Now, what we really want is to track motion, but there are a few things we have to figure out to do that. First, we have to be able to parse the date and time, which isn’t very straightforward. I found this helpful example, which I then modified to parse out the pieces of a date and time that I would need for my data model.

//if there's a result from the date process, parse it:
while (date.available() > 0) {
  // get the result of the date process (should be day:month:year:day of week:hour:minute:second):
  String timeString = date.readString();

  // find the colons:
  int dayColon = timeString.indexOf(":");
  int monthColon = timeString.indexOf(":", dayColon + 1);
  int yearColon = timeString.indexOf(":", monthColon + 1);
  int dayOfWeekColon = timeString.indexOf(":", yearColon + 1);
  int hourColon = timeString.indexOf(":", dayOfWeekColon + 1);
  int minuteColon = timeString.indexOf(":", hourColon + 1);
  int secondColon = timeString.indexOf(":", minuteColon + 1);
  int nanoColon = timeString.lastIndexOf(":");

  // get the substrings for hour, minute second:
  String dayString = timeString.substring(0, dayColon); 
  String monthString = timeString.substring(dayColon+1, monthColon);
  String dayOfWeekString = timeString.substring(yearColon+1, dayOfWeekColon);
  String hourString = timeString.substring(dayOfWeekColon+1, hourColon);
  String minuteString = timeString.substring(hourColon+1, minuteColon);
  String secondString = timeString.substring(minuteColon+1, nanoColon);
  String nanoString = timeString.substring(nanoColon+1);

  // convert to ints, saving the previous second:
  // int year, month, month_day, day_of_week, hours, minutes, seconds;
  month_day = dayString.toInt();
  month = monthString.toInt();
  day_of_week = dayOfWeekString.toInt();
  hours = hourString.toInt();
  minutes = minuteString.toInt();
  lastSecond = seconds;
  seconds = secondString.toInt();
  nano = nanoString.toInt();

  // Need to make sure we don't send an erroneous first motion event.
  if (lastHour == -1) {
    lastHour = hours;
  }
}

There’s a lot of nasty boilerplate code in that snippet, but this lets us track the different numbers we need to look at things like active seconds per day, hour, month, etc.

Next, we want to add some logic to the main loop to detect when the PIR sensor picks up motion:

void loop() { 
  if (pirVal == HIGH) { 
    if (pirState == LOW) {
      digitalWrite(13, HIGH); // LED ON to show we see motion.
      Console.println("Motion detected!");
      pirState = HIGH;
      lastActivity = nano;

      keen.addEvent("motion_detections", "{\"motion_state\": \"start\"}");
      keen.printRequest();

      while (keen.available()) {
        char c = keen.read();
        Console.print(String(c));
      }

      Console.println();
    }
  } else {
    if (pirState == HIGH) {
      Console.println("Motion stopped!");
      pirState = LOW;
      digitalWrite(13, LOW);
      keen.addEvent("motion_detections", "{\"motion_state\": \"stop\"}");

      while (keen.available()) {
        char c = keen.read();
        Console.print(String(c));
      }

      Console.println();
    }
  }

  Console.flush();

  // poll every second
  delay(1000);
}

Setting up the Dashboard

I wanted to set up a quick dashboard to track motion, so I took our hero-thirds dashboard starter, and loaded that into an emberjs project (I wanted to learn ember as well). You can see a live demo here.

I played around in the Data Explorer until I found the visualizations I wanted, then added them to the page. The final version of the Arduino code is also available to view.

So with a few simple lines of code and a quick dashboard, you can start tracking some interesting data with your Arduinos on Keen IO!

Have ideas for a project or want to hack around on your Arduinos? Come chat with me in our Slack channel!

Alex Kleissner

Software engineer by day, Partybot by night

How to write a round-closing pitch deck

Last July, we hit a huge milestone when we closed $11.3 million in Series A funding led by Sequoia. This raise was preceded by a few seed rounds. Of course, this was amazing for Keen, but the road to closing these rounds weren’t easy. It took several tries to get to the pitch deck that worked.

Our co-founder Dan Kador recently gave a talk at 500 Startups about the process Keen went through to get to the winning deck (for one of our seed rouds). We hope it will be helpful for anyone going through fundraising mode now, or looking ahead to a future fundraise!

The slides

Do you have any other tips for writing pitch decks? We’d love to hear them on twitter or in the comments below.

Alexa Meyer

Brand and behavioral design. Learner + Activator. Cheese consumer.

How I Visualize My Time Spent Programming

This is a guest post by Alan Hamlett, CTO of WakaTime.*

As a developer and CTO, I’m always looking for new ways to visualize my effectiveness as a programmer. I want to track things like “What was my most productive day of the week?” “What programming languages have I used the most?” “What files do I spend the most time in?”

I use the WakaTime dashboard to see my projects, files, and branch level visualizations over various time ranges, but it doesn’t allow me to create custom queries of my data. I could use the WakaTime api, but I found an easier way…

Introducing wakadump!

I created wakadump to easily export my WakaTime data to Keen IO and take advantage of their powerful data explorer features. For example, I queried all my WakaTime data from the last year and was quickly able to create these custom visualizations:

Most productive day of the week

Top files worked in over the last year

Editor usage over the last year

Programming languages used over the last year

Create your own visualizations

To create your own custom queries using the Keen IO data explorer, just follow these steps:

  1. Install wakadump with:

    sudo pip install wakadump 
    
  2. Export your logged time from your settings page.

  3. Sign up for a free Keen IO account account and grab your Project ID and Write Key from your project page.

  4. Upload your WakaTime data to Keen IO by running this command:

    wakadump --input wakatime-user.json --output keen.io
    keen.io Project ID: XXXX
    keen.io project Write Key: XXXX
    Preparing keen.io events...
    Uploading events to keen.io...
    Complete.
    
  5. Create custom queries using Keen IO’s Data Explorer

If you find something interesting from your WakaTime data, please do share your insight in the comments or send me an email!

Alan Hamlett

CTO, WakaTime

An Introvert Confesses

I recently went on a couple dates with a super friendly, happy, sociable guy I’d known for a few years. On our second date, he said he had a confession to make: he was actually an introvert.

The confession surprised me, partly because I had a different image in my head of what an introvert was like, but mostly because it made me sad that he thought of it as a confession.

I consider myself an extrovert, drawing energy from the people around me, and I hadn’t really thought about the barriers in our society both personally and professionally for people who just want a little more space to reflect inward.

Why did he feel like he had to apologize?

Based on a recommendation from an introverted co-worker, I looked up Susan Cain, a self-described Introvert. In both her book, Quiet: The Power of Introverts in a World That Can’t Stop Talking and her TED Talk, she describes how American society became increasingly extroverted in the 20th Century due to the rapid growth of cities where people needed to constantly network and prove their worth to strangers.

This trend has only intensified in the 21st Century.

Our modern workspaces favor open seating to foster better communication. Promotions are often received based not just on the quality of work, but also on how well you can present to large audiences. In Lean In, Sheryl Sandberg specifically advises women to raise their hands and make themselves heard in order to become leaders.

Meetings fill up each day of the work week. Brainstorming is frequently a group exercise. And companies often require employees to be in the office during core hours to demonstrate productivity. In 2012, the CEO of Yahoo, Marissa Mayer, created a policy banning telecommuting.

Those constant extroverted activities can drain the energy out of an introvert, and it’s debatable whether they improve productivity for everyone. Thinking and working alone can help people enhance skills, discover new ideas, and create something from nothing.

As an extrovert myself, I know that working alone can sometimes be draining, but I also know that it’s the quiet moments when I’m left alone that I can be most productive (writing this piece, learning to code, choreographing dances).

The world needs introverts too.

Is your work environment good for both extroverts and introverts?

At Keen, a lot of people work remotely, so we try to make it easy to work from home (using Slack to communicate, Google Hangouts for video calls, and GitHub for pulling and pushing requests).

Right now, we can’t control our physical environment completely because our office is in a co-working space, but we will soon be moving into a new space that will have different rooms with varying degrees of isolation and noise levels.

Our organizational structure is flat, which means that being comfortable presenting to large audiences doesn’t enable you to move up; it just puts you in a role that’s best suited for your comfort level.

We also have regular book clubs and writers workshops to bridge the divide between quiet reflection and outward communication, which can both be important parts of ourselves.

What can extroverts learn from introverts?

Susan Cain talks a lot about how an introvert often has to act like an extrovert in order to interact within our society. Perhaps extroverts should also practice acting like introverts.

Keen encourages everyone to work remotely for at least one solid week (#remotematters challenge), but I haven’t done that yet. It may be something to try, not only to better understand what it’s like for remote folks, but also to isolate myself from the buzz in the office.

I once saw a TED Talk by Jason Fried on productivity that encouraged people to have at least one day a week in which talking was not allowed. That would also enable introverts to relax, knowing there wouldn’t be any forced group interaction that day, and enable extroverts to turn inwards on projects best done in solitude.

Interruptions can stress out introverts and decrease productvity

Perhaps I should bring this to my personal life, too. I could disconnect from the internet each night for a week. I could periodically practice Shavasana or some other form of meditation. And probably most importantly, I could make sure to listen to the quiet introverted people in the room.

My extrovert resolution

I want to be more inclusive of introverts because I value what different perspectives can bring and genuinely believe that you need both introverts and extroverts to be able to create and communicate new ideas and projects.

But also, I really like that guy. I don’t want him or others like him to feel that they are confessing a secret. They shouldn’t feel the need to apologize for being introverts just because our culture is one that can’t stop talking.

Maria Dumanis

Good news everyone!

Introducing Pushpop for Product Hunt

We love Product Hunt. It has been a super useful platform for launching new products and gathering initial feedback. We wanted a way to easily gather and filter information from Product Hunt activity that we find interesting, so we built Pushpop for Product Hunt using Product Hunt’s API.

With Pushpop for Product Hunt you can trigger anything. For example:

  • Send an email to your team when you pass 100 upvotes
  • Obsessed with people who make productivity tools? Watch any collection that matters to you and collect the Twitter handles of all the makers in a collection
  • Pull a list of makers for any product
  • Receive a notification when your products show up on PH

These are just a few examples! It’s entirely open source. Check out how it works and let us know what you think!

p.s. Pushpop is an open source tool for scheduled integrations between popular services. We’re starting to build a number of popular extentions for it, including Pushpop for Slack. Contributions and PRs are always welcome!

Joe Wegner

Open source something

Join Keen Community Discussions Every Friday

The other day I was poking around the Keen Community Slack, and got really excited when I saw this:

WOW ALL THE PEOPLE

For some reason, it appears, people have been tricked into thinking we’re cool enough that they should idle in our community Slack. THIS IS SUPER DUPER CRAZY AWESOME.

Right now, however, the community in that chat kinda looks like this:

cute, but sleeping

My hope is that we can get it to look more like this:

loooooovvveee

In order to accomplish that, we are going to host Keen Community Discussions! These will be modeled after some stuff we’ve seen in the Ghost dev community - it’s essentially an open discussion time for the community to come around, talk about what they’re building, ask questions about data, analytics and APIs, and generally get to know each other.

This Friday is our first one. Assuming people are interested, we will be hosting these every Friday at 11:00 AM PT.

BONUS: Real Life Keenies

I think I’m pretty cool, and I’m sure all of you are excited to chat with me, but I’m also gonna have some other real life Keen employees available for you to gab with. I’ll be choosing a few specific Keenies every week to join us - if you’ve got any specific topic you want to discuss, let me know and I’ll make sure the right person is there.

I’m really excited about this. I spend a lot of time with my head down trying to build cool things, and it’s easy to lose touch with the community. This is a reminder and stellar opportunity each week to really dig in and meet the people who are building amazing things.

To join us, just hop into our community Slack channel. See you on Friday!

Joe Wegner

Open source something

Finding a Job That Fit My Life

Alone in Athlone

This time a year I ago I was in Athlone, Ireland, slightly miserable and trying to figure out why (because it probably wasn’t just the weather).

When I had planned the trip it seemed like I was going to be living the dream, being paid well to spend time in Europe, yet somehow that isn’t quite how it worked out. As a result I spent a lot of time thinking about where I wanted my life to go and what might actually make me happy.

When it came down to it, I realized that because I love my work I had been letting what I did dictate the rest of my life, rather than just contributing to it. I would like to share some of my process and results with the humble hope that someone out there might learn and benefit from my experiences.

I was here.

How did I end up in Ireland anyway?

Here is the whirlwind version. After college I was lucky enough to land a job at a small DVD by mail company called Netflix. I spent the next seven years working on amazing things like launching a streaming service and moving a large scale business to the cloud, while interacting with and learning from great people in an awesome environment.

As a Bay Area native I then decided it was time to explore the land of startups and proceeded to work at a couple of those and then start my own. I left the most recent one with a couple of colleagues to do some consulting and start our own thing because we really wanted to build something that was ours.

As we were leaving I was also offered an opportunity to consult for a large company in Ireland, which seemed like a great way to get some money in the bank and visit some friends.

Three lessons I learned about myself

My last day was a Friday, and Saturday morning I hopped on a plane to Dublin. And this taught me lesson #1; going straight from one job to another is a high stress, high energy operation and therefore has an enormous cost.

This leads me to lesson #2: it’s great to be excited, but if you get too caught up in your work, the rest of your life can suffer without you realizing it at the time.

Finally, lesson #3; After an amazing SF summer and a trip to Bali it became really clear to me that warmth and sunshine really do make me happier.

Figuring out what I needed

I spent months doing just that, both at home and abroad, getting feedback from my friends and mentors. I had decided that the next time around I would do things differently and select a job that went with my life rather than being my life.

In the end I made a list of what was important for my balance and happiness:
* Time for introspection
* Spending time outside
* Flexibility, in both what I work on and where
* Doing new things

Once I had these core needs, I decided there were a few parameters I could use to help me select a company that would support them. One of them was company size: are they big enough that I wouldn’t have to live on the other end of the batphone?

I love to firefight and had done it for years, but it just wasn’t conducive to a good work/life balance for me. Size also gave a sense of whether the company could support a cross-functional role, which is where I am at my best and most engaged, but just isn’t appropriate until a bit later in a company’s life.

This is not what I was looking for.

Finding the right culture

Finally, it was really important that the company have a culture that would support me in my desire to live a healthier lifestyle.

One thing I found useful was asking the most senior people I could talk to (founders, CEOs, etc.) how they balanced their own lives. What kind of hours do they work? How much vacation do they take?

One of the best answers came from Kyle at Keen: “When it’s sunny out, I will take part of the day to go for a bike ride" (this was reinforced by spandex and FiveFingers for an impending bike ride) “and I will probably take 8 weeks of vacation this year.”

He also said, “I would never expect someone to do something they have done before if they don’t want to. That just sounds boring!”

These were the kinds of things that made me feel like people were practicing what they preached and not just telling me what I want to hear.

Kyle in the wild

What about compensation?

After narrowing the field I was still faced with an interesting conundrum of how to evaluate compensation from the companies I liked most. This is always a challenge because the mix of cash and equity can be notoriously hard to value.

But then there was the added dimension of how to value what my life would look like at each company. If a company pays a bit more but might be more stressful, how does that fit into the equation? What about future career growth? Some companies have clearer tracks than others.

In the end I chose to make the monetary and career aspects secondary and select first for what would best support the life I wanted to have. This was made a bit easier by the fact that in the end they were all going to pay more than enough to have a comfortable life. (#firstworldproblems)

So what’s the moral of the story?

For me it was just how valuable it was to have some time away from work to think about what I really wanted to have come next, and the best way to go about achieving that.

It’s easy to get caught up in the excitement of new things and experiences, but the result was that I lost sight of the bigger picture. So I would encourage everyone to be patient and thoughtful before you jump into the next big thing.

Super-Quick Guide to Query Caching

Did you know we peeled off the “Beta” label on our Query Caching feature?

We wanted to make sure you had an easy way to take your most important repeated queries significantly faster. If you have query results you want many people to see, caching will make their experience so much better. By simply adding the “max_age” property to your query, you can specify how long you’re willing to quickly serve an existing result before we recalculate it. 


Check out the detailed blog post and docs if you’d like to learn more! If you have any questions please don’t hesitate to let us know.

Peter Nachbaur

Passionate about throwing and catching flying discs and data

Speed up dashboards with Query Caching

After ad-hoc analysis, many of you put your most valuable queries to repeated work in internal dashboards or front and center in your product. As your data volume and query complexity grow, you might find yourself asking “can I get access to these insights even faster? Can I improve my product’s experience by loading my customer’s metrics at the blink of an eye?” The answer is yes.

Data sent to Keen is available for querying almost immediately. We strongly believe that quick access empowers you to make decisions and build features that accurately reflect the state of your businesses world.

Query Caching allows you to trade this instant accuracy (or ‘freshness’) in exchange for increased query speed. The primary use case of caching is for queries that don’t need to have up-to-the-second answers, but that need to be answered as quickly as possible. If you’ve got a monthly, weekly, or even daily dashboard you are probably less concerned with the data from the last five minutes. But you certainly don’t want to wait over a minute to look at your historical trends!

Does your day ever feel like this?

Check out your product dashboard. Look at those charts. Up and to the right. Look at your Stephen Curry bobblehead. Up and down. Aw yeah, everything looks great. Boss emails a question. Back to your dashboard. Time to refresh. Loading. Loading. Loading. Are you still waiting for the {“result”: “valuable information”} to load, or are you already getting up to get your fourth cup of coffee and maybe play some foosball with Mike?

How many minutes are wasted due to slow loading dashboards? How many of your users leave your site before their visualizations finish loading? If you get frustrated waiting 111 seconds for a big query to finish, you can’t expect them to stick around. And I can’t speak for your code, but it’s not going to be happy blocking and waiting for an answer to “most popular cat shirts” to determine what recommendations to show.


Alright, better alt-tab back to your dashboard. You notice your acquisition funnel looks a little low in the last 30 seconds! Look to your right. Look to your left. Do you have an emergency lever you can pull to enhance your product immediately? Of course not. As a human, you probably never manually make drastic changes based on the last 30 seconds of behavior. You try to steer your product based on longer-term trends.


What you want, what your customers value, what your product needs is not always up-to-the-second accuracy but reliably rapid results. What your slow dashboard or product needs to leverage is Query Caching.

How to add Query Caching

Adding max_age to a Javascript query is as simple as adding a max_age parameter to your Keen queries:

var count = new Keen.Query("count", {
  eventCollection: "pageviews",
  groupBy: "property",
  timeframe: "this_7_days", 
  maxAge: 300
});

So go ahead, add the Caching max_age parameter into any query you wish was snappier! If you have any questions about query caching, building dashboards, or data analysis, don’t hesitate to reach out to us or ping us on Slack!

Peter Nachbaur

Passionate about throwing and catching flying discs and data