Integrate Eventbrite and Keen IO in 60 seconds

We were getting ready for an event recently and wanted an easy way to visualize all of our Eventbrite registrations. Eventbrite has handy webhooks that enabled us to quickly start sending our registration data to Keen IO. We used this data to build a dashboard to share with our team and promotional partners:

130 people will be at Open Source Show and Tell! woohoo

Here’s what you do:

  1. Create a free Keen IO account if you haven’t already
  2. Grab your Keen Project ID and Write Key
  3. Head over to your Eventbrite Account page
  4. Scroll down and click on ‘Webhooks’
  5. Add a webhook with your Keen URL: https://api.keen.io/3.0/projects/<KeenProjectId> /events/Eventbrite_Events?api_key=<KeenWriteKey>
  6. Watch your registration events start flowing into Keen!

Once your Eventbrite data is flowing in, you can use the Keen IO data explorer to start querying and visualizing your Eventbrite registrations.

A view of our daily eventbrite registrations

Or you can use our JavaScript library to create your very own custom dashboard to share with your team! We got our event dashboard live on the interwebs super quickly using Divshot.

Let us know if you have any questions. Have a great event!

Searching for a Better Way to Do On-Call Rotations

The other day, Cory (one of our platform infrastructure engineers) sent out a company-wide email about how Keen’s Platform and Middleware Teams were trying to make on-call more manageable. It was a really interesting glimpse into the challenges of ensuring round-the-clock reliability, while also maintaining healthy personal relationships and some degree of sanity.

I thought this might be helpful to other people working on-call and asked Cory if he’d be okay with sharing his original email on the blog. Always eager to help, he said yes, so here it is! Kevin

Hello party people!

Recently I was chatting with some folks and realized we’ve not talked much outside of the on-call group as to what’s been going on with on-call. I wanted to take some time to conduct some information out as to what we’ve been doing!

Recap

As many of you know things were pretty rough in February and part of March. A lot of long nights got pulled and we had to resort to swapping people out of on-call a few times to rest folks. We learned a lot. Primarily we learned how to band together to fix problems and help each other out. It was a time of sacrifice that many of us (and our families) are still recovering from.

While we’re here, we want to thank everyone for being so understanding and willing to help. We know many of you wanted to do anything you could to help. You didn’t have knowledge needed (yet!) to sit at a keyboard and fix a busted thing, but you all contributed in your own way and we appreciate it!

First, Current State

We have made huge strides in the last few weeks to improve the on-call situation. The most important metric is people’s attitudes and our rested state. This is hard to measure. From my seat the team is in a much happier place. Many of us have taken small vacations to help shore up our moods and repair some of our relationships.

What is measurable is the number of pages:

The spikes represents The Troubles™ and we’ve made a huge improvement. The recent uptick is not representative of problems. It’s representative of improvements we’ve made that took some tuning and got sorta noise for a week or so.

I can relay that Jay, who just came off primary on-call last week, called it one of the “lightest” on-call weeks in recent memory. Yay! Congrats to everyone spending so much time on these improvements.

Second, Mechanics

We’ve been meeting regularly to review how on-call works so we can optimize things for everyone. The first thing we decided was that adding new people to the rotation would not immediately help. In fact, as Brooke’s Law describes, it would’ve hurt us as we raced to recover from our problems. We made this clear to some of the new team members. This is not a permanent thing, just a short-term plan to mitigate the blast radius.

Breadth Is Hard

It’s tough to know your way around Keen’s entire stack. Our desire to be polyglot and leverage OSS tools means that we have a lot of stuff for people to learn. So much so that no single person knows how everything works. To that end we’ve begun to specialize our on-call rotation into three categories:

  • Stormshield: Cassandra, Storm, some Kafka bits
  • Middleware: Pine, Myrrh, Service, LBs, some Kafka bits
  • Triage: General overview of everything, meant to help mitigate simple failures and escalate harder ones

Thanks to Kevin, as of last week we officially have two on-call rotations and our alerts are divvied up between 3 escalations. We’re beginning to leverage both on-calls depending on the nature of the failure. This has some great side effects:

  • You have a domain expert on hand to help deal with a problem
  • You aren’t alone

We’re not done with these mechanical improvements. We’re still meeting every two weeks to iterate toward an on-call that is more approachable. We’re now discussing how to integrate new people and bring down the OMG ON-CALL IS HARD AND LONELY problem. Luckily we have a lot of on-call experience, smarts and compassion.

Third, Infrastructure Improvements

There has been a ton of work in the area of maintenance, bugfixes, upgrades and other contributions from nearly everyone in PLAT and MID. Here are some of the big items:

  • Complete overhaul of Zookeeper machines, which coordinate both our Storm and Kafka machines. (Thanks to Brad for keeping this going, which was really scary!)
  • Ongoing repair and improvements to our Cassandra data. (Shout out to Brad for stewarding all of the repairs and to Manu and Kevin for working with our Cassandra consultants!)
  • Revamp of our fleet of Storm machines to have gobs of memory and not run supervisor instances on our nimbus nodes. (Thanks for Shu for provisioning, overseeing upgrades and making all the changes for this.)
  • Overhaul of our “chat ops” deployment system to homogenize the deploy commands for all our stuff. Every Keen-created service is now consistently deployable from @robot! (Thanks to Alan for the revamp and to Shu for continued care and feeding of the bot!)
  • Continued improvement of a “query tracing” feature for diagnosing where slowdowns occur and where we can optimize execution of queries. (Thanks to Kevin for introducing this feature and to Manu for his amazing efforts at producing measurable analysis of query execution so that we can compare efforts going forward.)
  • Improvements in the efficiency of the compaction path, causing fewer pages and operation issues around compaction, as well as reducing overall load on Cassandra (Amazing effort by Kevin!)
  • Pine has evolved and developed a considerable number of protections to keep the service healthy. Some have been bumpy but overall we both stay out of trouble more often, and recover from trouble much faster under it’s supervision of query scheduling.
  • Keen-Service has seen dozens of bug fixes and improvements to logging, query tracking, error handling and general maintenance over the last few months. The most recent improvement fixed an oversight where a large number of queries were not being load balanced! (Shout out to Jay and Stephanie for their continued diligence and ingenuity in improving Keen-Service!)
  • Our observability and monitoring has been repeatedly improved and rethought across every service within Keen. We have considerably more fine-grained visibility in to how things are behaving from per-queue query durations to visibility in to specific wait times in storm bolts. (Amazing work by Stephanie in testing metrics in Service, Manu in creating Turmeric and every person who handles on call for continually improving our monitoring.)

I’m probably leaving out contributions by a bunch of people. Sorry, I did this from memory and tried to iterate through every major component I could think of.

The Future

Note that we’re not just focused on short term fixes. PLAT is actively working on query performance improvements, data storage/compaction improvements and a bunch of other stuff. MID is working on caching and continued improvements to Keen-Service and it’s future incarnations. There are also 3 new folks that have joined (or will be joining soon) to contribute their considerable experience to the mix. Yay!

Thanks!

On-call shouldn’t dominate our lives. It’s also a necessary and important part of how we maintain the trust our customers place in us every day. We’re lucky enough to work in a company where the power to control this major part of our job is in our hands. To that end we’re working weekly to make on-call an experience that as many people as possible can contribute to. It’s worth nothing that this point in Keen’s history is hard. We’re just big enough to need to specialize, just small enough to not have all the people (yet) that we need to specialize, and all present in a period of growth wherein this transition is hard and messy. Thanks to everyone for working every day to make this a supportive experience.

Henceforth we’ll try and collect information about this every month or so to conduct things out to everyone at Keen. If you’ve got any questions, let me know!

Cory Watson

Bigger than a breadbox.

OSSAT?

Keen IO is proud to partner with Rainforest QA and Segment to support the open source community by putting together the next SF Open Source Show and Tell. “What is Open Source Show and Tell?” you may ask. Well, I’m glad you did!

TL;DR: Open Source Show and Tell is a series of events for anyone and everyone interested in open source projects. They are inclusive events that are all about sharing, learning, and getting involved in open source projects (“OSS”).

We will be featuring a presentation by Beth from Microsoft about their experiences open sourcing the .NET platform.

Click here to sign up and join us

Want to talk about your project? :D You can submit talks by posting a Github Issue™

In the past we’ve had many indie developers present their own projects. A couple notable ones are:

  • Alan Schreve on ngrok
  • Alex Gaynor on organizing the python community and doing proper code reviews in a distributed collaborative (and hopefully friendly) environment.

We have also had presentations about internal OSS projects from a wide variety of companies including Google, Airbnb, Uber, Rackspace, Plivo, Balanced, Keen IO, Twitter, and others.

Hope to see you at Open Source Show and Tell on April 24th!

PS. If you’re interested in organizing a #OSSAT in your city here’s a playbook for more information. It’s open source, so pull requests welcome ;)

Justin Johnson

community guy, hacker, music nut. i like to help people build stuff.

Lessons from a failed YC pitch with Paul Graham

Last week, Kyle Wild and I sat down to talk about one of our favorite topics: failure.

Our friend Alan had given us the idea. He said it could be interesting to hear people with some degree of objective success talk about the times they had failed. He thought it could help new people coming into tech, or any field, cope with impostor syndrome and other fears.

I asked Kyle if any failure story came to mind, and he started laughing immediately: “Oh man, this is bad. This is real bad.”

“Excellent!” I said, and started up the recorder.

Paul Graham experiencing brain failure as Kyle Wild pitches Keen IO (then-named Schmetrics)

Kevin: Tell me about your biggest failure.

Kyle: Okay, when we were starting Keen IO, I went to this thing called Startup School, which is Y Combinator’s weekend of talks for aspiring entrepreneurs. They sent this thing that said, “Hey we’re going to have a couple people come on stage do live office hours with Paul Graham, the founder of our company. He’s going to demonstrate what YC office hours are like.”

YC office hours are like, there’s somebody on the pulpit who’s brilliant, and the lowly entrepreneur is going to sit down with them for an hour and talk. I was hoping to get an interview for Y Combinator anyway, so I said I’d love to do it.

Anyway, I’ve always had stage fright but for some reason I thought I’d be okay for this. When they announced my name I got text messages from friends. They were watching live, which made me nervous. So I go on, I don’t know what happened, it was a blur. It was bad.

Kevin: What was so bad about it?

Kyle: I was abstract, I was mean, I was futurist, I was a little too certain, I didn’t have any evidence, I was reasoning from first principles.

Kevin: How do you know it was a failure?

Kyle: Two things. Number one, we didn’t get into YC. That was painful. This is the biggest class ever. 80-something companies got into YC and we weren’t one of them, which feels shitty.

Then I jumped on Hacker News and there’s a post on “How Not to Talk to Investors” and it had two videos, and I was one of the examples of how not to do this, basically being publicly humiliated.

So obviously I didn’t go on stage for demo day when we were in Techstars the next year, and I didn’t speak at all again until two years later! Every time I’d think about public speaking I’d clam up and I’d think about how the internet is everywhere and if I fuck up people are going to refer back to it as a failure.

Kevin: What are your biggest regrets about it?

Kyle: Honestly one of my regrets is I stopped talking, I stopped going out and being me. I was afraid of being me. I’m a pariah in the industry. I’m on the do-not list of fundraising so how can I fundraise? On the do-not list of public speaking, how can I public speak?

I’ve become a fairly effective public speaker now and I’m getting better. It sucks to lose two years that I could have been working on that. Could’ve helped the company.

And I didn’t even feel like an impostor until somebody called me one and then I was just like, well, I guess I’m an impostor. It’s not because I think he’s right. I now know he wasn’t right. It’s because he said the stuff I was afraid of. That puts you on a bad path.

Kevin: What’s it like to have success after the worst thing you could imagine happened to you?

Kyle: Oh it’s fucking sweet. Like at KeenCon, I did the opening talk. I’m like, what’s the worst that could happen? Someone could record it and then write a post about how stupid I sound and it wouldn’t even be news! It’d be like, well it happened again.

Kevin: What does it make you think about failure in general?

Kyle: Failing at that investor talk in 2011, being shit on a little for that, once I came around, it made me feel pretty bulletproof. Well I failed, now what?

There’s a big difference between zero failures and N failures. When you have an N greater than zero, you’re fine. You know what it’s like. It’s not that bad.

This post was on Hacker News for a second and everything on Hacker News is gone in 15 seconds. I’m sure 1,800 people saw it and maybe 90 of them remembered it.

I did have somebody approach me who said they remembered it. Somebody was like, “I think I saw you on stage in an article about flubs or something.” I was like, “Oh was it an article like, look at these idiots?” He’s like, “Yeah.” This is somebody who approached me after a talk specifically because he liked my talk.

This is why I feel like fear of failure is one of those dangerous things. We should all push ourselves so hard we fail once in a while. I am a little disappointed in myself for not just getting right back on the horse. I had to process in my own way.

Kevin: You’ve talked to people who are going through various startup programs now. Have you given advice that relates to this experience?

Kyle: To some extent. This thing happened to me, maybe October 2011, and then October 2012 I went on a trip, Geeks on a Plane, and I met a coach who is an amazing person. Ed Nussbaum. He noticed I was clenching and freaking out because I was about to go on stage just to say, “Hi I’m Kyle, I’m one of the geeks. I work at this company Keen IO.” This wasn’t even a talk but I was freaking out.

He said something great. He said, “Kyle, you’re an improviser. You’re not a politician. You’re not a stump speech guy. You’re an improviser so what you need to do is give yourself permission to improvise.” Those are the exact words.

So what I started doing was before I go on stage I would just try to meditate. I’m not going to think about anything. I’m just going to stand here. Find a dot on the far wall and just look at it and be like, I wonder how big that dot is.

Just focus thoughts on shit that doesn’t matter and then they’d call my name and I’d be like, oh shit okay. “I’m Kyle, I work at Keen IO. We make an API that lets developers build analytics into their apps. Whatever kind of analytics they want. If you want to talk to me about it, find me after the show.” Really simple little sentence. I could never write that but I just made it up. Even just now, I just made it up. And that’s what I’m good at.

So basically, fuck the haters. It’s easy to find people who are going to call you an impostor. It’s harder to find people who are going to help you find your path.

You’ve just got to find people who build your confidence, listen to them, and find the people who shrink your confidence and just crush them out of your life. They suck.

image
Kyle improvising for the crowd at KeenCon

Post-script: At several times during our conversation, Kyle said I should try to find the article from Hacker News. But after half an hour of Googling, I gave up. It seems to have vanished.

I did find the video of the talk, though. Get ready to cringe a little…

Kevin Wofsy

Teacher, traveler, storyteller

Using HTML5 attributes to clean up dashboard JavaScript code

For anyone who’s built one or more dashboards, there is a common practice I have termed “copy pasta” code. This is where you copy and paste a single code block multiple times and then tweak one or two parameters here and there. This can happen when using the Keen API to build a single page with a bunch of charts (AKA a dashboard).

There has to be a better way, though! All that copy and pasting means that making changes and updates is error-prone and tedious. In a recent revamp of one of our open source projects, Pingpong, I decided to tackle this very problem.

HTML data attributes are your friends

HTML5 added data attributes you can use to store custom data for a given element. What if we moved all of the customized information for a given chart into data attributes and had a single JavaScript function to render each one?

In the context of the Pingpong project, I wanted to have a status page for checks with a variety of charts, but I wanted to make it easy to play around with the different display styles of the charts and deal with “global” filters for all of my queries. Updating each of the charts every time was tedious, though, so I took the old HTML that looked like this:

<div class="row">
  <div class="col-sm-8">
    <div class="chart-wrapper">
      <div class="chart-stage">
        <div id="grid-1-1"></div>
      </div>
    </div>
</div>
  <div class="col-sm-4">
    <div class="chart-wrapper">
      <div class="chart-stage">
        <div id="grid-1-2"></div>
      </div>
    </div>
  </div>
</div>

and turned it into this:

<div class="row">
  <div class="col-sm-8">
    <div class="chart-wrapper">
      <div class="chart-stage">
        <div id="grid-1-1" class="chart-container" 
          data-querytype="average"                        
          data-charttype="line"              
          data-groupby="check.name"              
          data-timeframe="last_48_hours"            
          data-interval="hourly"            
          data-targetproperty="request.duration"             
          data-chartoptions='{"legend": {"show":false}}'>

        </div>
      </div>
    </div>
  </div>
  <div class="col-sm-4">
    <div class="chart-wrapper">
      <div class="chart-stage">
        <div id="grid-1-2" class="chart-container" 
          data-querytype="count" 
          data-charttype="donut" 
          data-groupby="response.status" 
          data-timeframe="last_48_hours" 
          data-chartoptions="">

        </div>
      </div>
    </div>
  </div>
</div>

All of the chart-specific information is now part of the DOM, and if I want to change something on a single chart, I can just edit that div element.

One JavaScript function to rule them all

The “copy pasta” method of rendering charts with the Keen JS SDK is straightforward, but it means that often your code turns into:

This code works, but there’s quite a bit of repeated code in there, and we could pull all that information from the DOM elements if we used HTML data attributes.

So instead, let’s add a single JavaScript function that handles the JavaScript elements of rendering a chart by pulling information from the data attributes (I’m using a lot of jQuery helpers)!

So, what does this do? Now any element with a class of chart-container will get a fancy Keen chart attached to it based on the metadata in the data attributes.

If I want to apply a filter to all charts, I can do that in this function! If I want to enable per-chart filters, that’s easy to do, too. If I want to play with the c3 library or chartjs, it’s easy to swap out all the charts on a given page.

Hopefully this is helpful in making dashboard pages manageable to maintain and modify! If you have any questions or additional ideas about how to use this technique, feel free to let me know.

Alex Kleissner

Software engineer by day, Partybot by night

An update on query durations

We wanted to provide an update on the state of Keen’s query performance. After some rough patches in February and March, we’ve made significant progress in stabilizing queries.

However, query durations are still not as fast as they were, say, three or four months ago. We understand this continues to be frustrating for customers who built solutions that relied on those faster query times. We want queries to be faster too, and hold ourselves to a very high standard when it comes to reliability & performance. It pains us to limit your experience. As part of our commitment to transparent communication, we wanted to increase your awareness of what we’re doing to address the situation.

Why are my queries slow?

There is no single reason for these query duration issues, but they are generally related to the challenges of rapidly scaling our service. To be perfectly transparent, many of you are running fast-growing companies, and your data & query volumes are growing with you. On top of being fantastic, committed, and growing customers, many of you have also recommended Keen to new developers, too. As a result Keen usage has consistently grown (and continues to grow) 20% month over month. Scaling to support you is the challenge we signed up for, and we’re happy to do it, but you wouldn’t be paying us if it weren’t indeed a challenge.

Although platform companies like ours would love to say it isn’t the case, of course another factor that leads to spikes in query duration are individual users dramatically exceeding the standard query load (aka, noisy neighbors). We have already drastically improved, and continue to improve, our ability to detect and protect the platform from these types of use cases, and to work with these customers to find the right solution for their needs. It’s our job to ensure noisy neighbors don’t impact your experience, and we’re committed to that. We don’t want to pretend like that isn’t a challenge either, though.

For those interested in the technical challenges (and triumphs!) of building distributed systems, we plan to write more to explain individual bottlenecks we have encountered with various pieces of the pipeline infrastructure.

What are you doing to resolve this?

Currently we are significantly strengthening nearly every major internal system we rely on. To get a bit more technical, enhancements to our Zookeeper installation are wrapping up. Capacity expansion to our Storm cluster is underway. Our Cassandra data model is being reworked to address costly hot spots. And we’ve further rationalized our internal DNS which will ease deployment and maintenance.

In addition, we now have even more powerful internal tools for performance profiling and benchmarking. We will also be rolling out better service protection in the coming weeks. Structurally, we are looking to significantly expand the size of the platform engineering team (there were only 6 of us until recently; now we have 8 and our team is still growing).

Finally, we didn’t set out to build a company just to see how fast we could grow it. There is no point in scaling Keen bigger and faster if it comes at your expense. The trust of our customers is our most precious asset. So, as another protection to our customers (and our team), we’ve decided to put on hold several new, very large potential customers. Longevity, stability, and sustainability are far more important to us than fast growth.

How can I stay up to date?

You can check our status page for regular updates on performance metrics and query durations.

We also suggest checking out our Query Performance Guide. The guide contains some great tips on how to optimize your queries. In addition, our beta Query Caching feature is now ready for general availability. If you’re interested in significantly increasing performance and consistency for queries that are used repeatedly please reach out to us and we can enable this new feature for you.

What’s the timeline?

While we continue to work through this rough patch, queries will not improve in one fell swoop. All queries will continue to be slower than normal over the next few weeks. Please be aware that Extractions and Percentiles are particularly slow at present.

Achieving our performance standards–in speed, dependability, and scalability–is our top priority. We believe that the investments we are making over the next few weeks will pay off and your query performance will improve, not just in the near term but well into the future as our volume continues to grow.

As always, your patience and understanding are greatly appreciated. We can’t imagine building for a better community. Once again, our deepest apologies for providing you with less than stellar service. We will improve, and we are committed to transparent communication along the way.

Peter Nachbaur

Passionate about throwing and catching flying discs and data

Don’t Let Anyone Tell You That You Can’t Be a Developer

My hands were shaking…I could barely breathe

I had just finished the first one-one-one coding assessment in my six-month coding bootcamp and it had not gone as well as I had wanted.

Honestly, I felt like I bombed it.

Slowly, I withdrew my hands from the keyboard. My mind was racing. It was going to be ok, right? Surely there would be a point where I didn’t feel so lost? When I wouldn’t feel like I didn’t have a clue, right?

Did I just make a huge mistake by putting my life on hold and taking out a loan to start this coding school when I would never be able to become a developer?

I needed a little reassurance. In a moment of self-doubt, insecurity, and vulnerability I turned to my instructor and said, “I know I didn’t do so well but I’ll get better, I’ll be able to learn and become a developer, right?” He threw his hands up and said, “I can’t say…this isn’t for everybody. Not everyone can learn it.”

I was crushed.

After going to the bar down the street for a nice pour of whiskey, I returned to class and just happened to run into another TA in the hall. She asked how things went and I told her my fear that maybe I wasn’t cut out for this. Without missing a beat she said, “You can do this. Don’t let anyone tell you that you can’t be a developer!”

She was so sure, so confident, I was taken aback. “Are you sure?” I timidly asked, hoping against all hope that she was. She smiled, “Aubrey, this isn’t easy. It’s going to take a lot of hard work, but if you want it, you can do it.”

That night I had an existential crisis

I asked myself why I wanted to become a developer. I had always had a deep love for tech, the arts, and for helping others. When I was younger I had trouble deciding which direction to follow, first going to school for teaching, then spending some time in Nashville writing music, then doing humanitarian work in Central America, and finally finding myself working in an Apple Store in Boulder.

While I saw parts of myself in each new career turn, I never found a way to merge my strengths until I discovered software development.

Why had I not started sooner? Well, I remember being told as a kid that I “wasn’t very good at math” or I was more of a “creative type” - great with music and the arts. These assessments molded how I felt about myself and to some extent created internal boundaries I felt that I could not cross.

Unfortunately, messages like these are all too common. According to the National Center for Women & Information Technology, 56% percent of technical women leave tech companies within 10 years – more than double the dropout rate for men. And a Harvard Business Review study found that 50% of women in these fields leave because of hostile work environments.

Reflecting that night after my assessment I realized that in the past I had taken discouragements to heart more often than I had encouragements. It was at that moment I decided I would forge ahead despite how hopeless I felt and throw myself completely into learning as much as I could.

I would ask questions in and out of class, make connections with speakers who came in, and stay in touch with the people I met along the way.

Ten months later…

Less than a year after my anxiety-inducing moment of extreme self-doubt, I am happy to report that I am indeed a developer. I am three months into my dream job at Keen IO. It’s a running joke at work - every now and then a coworker pinches me just to prove that it’s real life, I’m not dreaming.

Not only am I working and learning more about code, but I am creating curriculum for an apprenticeship-style program that will allow people right out of bootcamps and college to rotate through our teams, learn more about new technologies, and continue to grow as developers.

The Learner Program will have cohorts of apprentices, multiples of two so that they can work together on projects and talk about how the program is going together. By working as pairs, they are also “not the only one” and are more apt to ask questions and seek out answers.

They will be rotating through the company spending 4-6 weeks embedded in each of our teams. Throughout the program, we will encourage Learners to share their learns and experiences with their team, the company, and the community.

As I am creating this program, I am also the first Learner and this is my first post. I’ll be sharing more along the way.

I’m sharing my story for a couple of reasons

If you are trying to become a developer, just know that along the way you will hear a lot of other people’s thoughts. Some will be positive and some will be negative. You can learn from both, but try to hold on to the positives and use them as encouragement while learning from the negatives without letting them hold you back.

If you are already a developer, to you I say choose your words carefully. Choose to speak words of encouragement to those junior devs at your company or those you come across at conferences or MeetUps. With a few minutes of kind words and attention, you can change their life. You can give them the boost they need to persevere through the struggle and pain of learning in this great, but sometimes terrifying world of software development.


Sharing my gSchool to Keen IO story

Aubrey Howell

Developer of software and people.

Introducing: Data Explorer

We’re excited to release Data Explorer, a brand new and improved version of the Keen IO workbench for querying and visualizing your data.

Many of our customers use the workbench to run ad-hoc queries, create quick charts, and extract data. We’ve made that even easier and more enjoyable with the new Explorer.

To check out the new Explorer, go to your project page in Keen IO and then click on the Explorer tab! Let’s walk through building a query, and some of the new things you’ll see.

If you haven’t sent data to Keen yet, and want to play around with Data Explorer, please check out our getting started guide!

First, build your query:

  • View your event collections and schema without leaving the page, using the new preview button that gives you a quick glance at your schema and recent events

  • Easily select the right collection and parameters for your query, using our dropdown menus. Just start typing part of the word you’re looking for and it will autocomplete!
  • Build a filter for your query, using the event type as a base for your filter - choose from string, number, null, list, Boolean, or datetime
  • Try out the new geo-filter, which enables you to to filter events by latitude/longitude
  • Pick a date and time range for your query using our calendar selector

Next, beautifully visualize the results of your query:

  • Toggle between different visualizations of your data, choosing from chart types including area, line, or pie. You can also view your results in a metric or JSON format. (Javascript will also be coming soon)
  • Select the table output to view your data as a simple table

Alternatively, extract your data by email or preview an extraction in the browser:

  • View up to 100 events as a preview table in the browser
  • Send a full extraction to your email, with an optional limit on number of events to extract

Our first couple years at Keen, we focused primarily on building the API and backend tools. While that remains our top priority, we now have a team of engineers focused on building out our front-end and visualization offerings, and Explorer is our first product release. We’re excited about growing this team to better serve your needs.

We’ve also worked closely with a set of customers to test out the Explorer in beta, and we’d love to give a shout-out to them here for their patient feedback and suggestions. We would also like to get your feedback on how you like the new functionality. Email explorer@keen.io with any comments, feedback, or suggestions!

Happy exploring!

Nahid Samsami

Product at Keen. Cat Advocate. Also known as Hidi.

How to scale your company culture

Your company culture is what you collectively believe and — in practice — what you collectively do. It shapes how people work together, how you deal with problems that arise, how people feel when they meet someone from your company. Derived from a company culture is typically a set of values and beliefs, such as transparency, collaboration, or trust.

What do these values actually mean, though? If I’m a new employee coming into an organization, am I expected to know how to convey these values? No. Of course not. That’s why companies instill their cultural values in all kinds of different ways. For example, at Facebook the value “moving fast and breaking things” is introduced during onboarding. But instead of just telling people to “move fast and break things,” Facebook sends all new employees off for 6 weeks to literally code new things, break things, and learn from seasoned “fast movers and breakers”.

At Keen, we value introspection: the examining of one’s own mental and emotional processes. We believe this is important for sanity, harmony, and productivity in the workplace.

One way we support introspection is through a weekly activity called Anxious/Excited, where we all get together to share the things we’re anxious about (work-related or otherwise), along with the things we’re excited about. Anxious/Excited (A&E) is such an integral part of our culture that we frequently invite people to participate if they’re thinking about joining the company. It gives them a chance to see what we’re like in our most reflective moments and get a sense of what it would feel like to work here.

I often tell people outside of Keen about this activity and the reaction I typically get is, “That’s great, but how do you scale that?” The answer: you don’t.

The commonly referenced “do things that don’t scale” for startup growth can apply to the expression of cultural values, too. What’s key to cultural scalability and success is a shared understanding of the company’s values and a commitment to revising and evolving how they’re expressed as the organization grows.

Sounds straightforward, but oftentimes organizations that should be working toward the same goals and values end up fighting over tactics, resulting in a toxic and unproductive environment.

Values > Tactics

When Keen was small (as in 6 employees working out of the founders’ living room), A&E was an excellent tactic for introspection. Everyone was working together day in and day out. Some days were stressful, some days were happy. Everyone was close. Taking time to share and reflect at the end of the day was not only valuable, it was easy.

We are now at 40 people, and as you can imagine, A&E is not as effective as it once was. Debates have emerged on “how do we scale A&E?” How do we recreate the feeling of safe space and intimacy that allows people to open up and be introspective?

In times like this it’s helpful to think about why we did A&E in the first place. If we look back to our value of introspection we can see this “scaling problem” from a different, more open lens. There may be a totally different and better way to support introspection at 40 people than 6, and yet another way to do it at 100, 200, and 1000. The key is stay committed to our values, and to evolving the way they’re expressed.

We’ve already evolved A&E to make it work better for more people:

  • Two time slots for A&E
  • Remote A&E (for our remote employees) + In-Person A&E

We’re also starting to think about other ways to support an introspective culture, such as:

  • Team-based A&E vs. the entire company
  • Company reflection time for personal journaling, or shared wiki/internal blog
  • Bringing in an onsite psychologist and coach to help talk through problems
  • Having a writer’s workshop to explore the issues and processes on our minds

The bottom line is: Don’t be afraid to let go of your cultural tactics. In fact, you should be constantly evaluating them. Are they still working? Do they still represent the value you intended? Be committed to evolving. Remember why you decided to implement an activity in the first place, even if that leads you to an entirely new approach.

How have you scaled your company’s culture and values? We’re still figuring this stuff out, and would love to hear your ideas. Feel free to tweet at me or share your thoughts in the comments below.

Alexa Meyer

Brand and behavioral design. Learner + Activator. Cheese consumer.

SxSW Field Report, Day 3: Farewell Transmission

Hi. Hello. Hi. We have departed Austin, so this will likely be my last post, unless something particularly wacky and hilarious happens on the dynamic 3-hour drive between here and Dallas. Which, um, can we all pray it doesn’t?

SCHMIDT HANGOVER INDEX

Do you all know about the Schmidt Sting Pain Index? This entomologist named Justin Schmidt invented a pain scale to measure how bad various insect stings hurt, but the descriptions are all written in really weird, irreverent, evocative language. Like: “Caustic and burning. Distinctly bitter aftertaste. Like spilling a beaker of hydrochloric acid on a paper cut.”

Anyway, there are some days when I try to describe my hangovers along those lines. Today is, hmm… like a poisonous fuzzy caterpillar worming its way along the folds of my brain? How’s that? Pretty good?

OK, SO THE R.V. WAS PROBABLY A MISSTEP

When Justin told me he was thinking about getting an R.V. for Sx way back when, it seemed like an absolutely amazing idea. We’d rock around the streets of Austin like royalty, doing weird little pop-up dance parties, mobile opium dens, etc., etc., etc. But, as it turns out, you need a sober driver for most of those things, and no wanted to deal with that trash, clearly, so mostly the RV just sat around this whole trip and provided reasonably comfortable sleeping quarters for any overflow Keenies who couldn’t fit in the AirBnb. (The RV was parked kind of tilty, though, so I found myself slowly rolling off the bed over the course of each evening.)

Favor has shifted even more against the R.V., however, as we are driving back now, and something is wrong with the waste tank or something, because everything smells like poop. There is a distinct poopy smell. It is… not great. I can’t really explain to you how not great it is, to be trapped with this smell, hungover and exhausted and emotionally drained. I do not super recommend it.

That said, it is a little funny, still, in that sort of masochistic gallows humor way I enjoy. The internet has not been super helpful with responses to my search for “poop rv smell,” so we mostly just have all the windows open and have stuffed toilet paper or ear plugs in our noses. This all seems kind of fitting to our collective mood, I guess is what I’m saying.

image

Not the happiest bear in the woods, perhaps.

UPDATE: We stopped at a gas station, and I bought a handful of car air fresheners to strew about the place. They may not help with the smell, but they will certainly help us with Courage. 

image

OUTLET MALL PURGATORY

Tim Falls needed coffee, so he won’t nod off and steer us into a traffic embankment, so we stopped for Starbucks and a grim food court lunch at an outlet mall 30 miles outside of Austin. 

I’m sure it’s more our collective mood than anything, but this place just seemed buh-leak. Like, I am sure it is a fine place if you’re in the mood to wander about and get bargain store prices on brand name goods™, but in the state we’re in, it just felt like sheer depression. We sat there, miserably eating our Subway sandwiches and kind of avoiding eye contact with each other, and I was like, “I imagine this is what Purgatory will be like.”

image

Seriously, hooray for everything.

They also had a sad, lonely Easter Bunny on-hand you could have your kid’s portrait taken with. I really, really wanted to get a morose group shot of us – “Happy SxSW Aftermath!” – but I don’t think anyone else wanted to stay there any longer than they could help it. 

Purgatory is gonna be a looooooong wait, huh?

PROP-UP AFTERMATH DAY

This might actually be a pretty short post (haha no it won’t), as yesterday wasn’t nearly as eventful as Saturday. If Saturday was our superhero day, Sunday was more like our kryptonite. We were all kind of beaten and broken and burned out, and it took us a looooong time to rally. (Some of us never actually managed, I’m pretty sure.) Justin’s stomach was a 5th-grade Science Fair volcano, for example, and I spent a disproportionate amount of time doing dead stares into the middle distance all day. WOO LET’S PARTY, is my point.

It took me twice as long (and like 3 times as many substances) to get through yesterday’s field report – although it’s been getting pretty good feedback from people, so that’s nice! 

(Although, it needs to be said, John and Andrew, I am so sorry I apparently amalgamated you into “some guy” who quoted my field reports back at me. I am normally pretty good at names, because I write little mnemonic poems about everyone I meet – e.g., “Steve with the rolled-up sleeves” – but I clearly dropped the ball on this one! Ahh, sorry!) 

That’s the tricky balance about this sort of writing, though – to actually sit down and write all this shit out, you’re probably going to have to miss out on some things. I DO THIS FOR YOU, INTERNET. 

image

Pictured: Me, Substances, Writing.

Anyway, I finally got out the door around 4 pm, just in time for our impromptu pop-up party at Weather Up, which was a brilliant move on Justin’s part because a) that is an amazing bar, with a lovely porch and friendly dogs and delicious drinks and hand-cut artisanal ice (which is apparently a thing humans can care about?) and b) because we still got to meet people without actually having to move or do things. 

image

You could really see just how rough of a state we were in for the first hour or so, all kind of just quietly sitting in a circle and trying to choke back enough alcohol to prop ourselves up and start interacting with other humans again. Taylor came by with a bunch of her school friends, and they were all so bright-eyed and bushy-tailed. I’m sure they saw us lurching around and were like, “What kind of dead-eyed ancients work at this company, anyway?”

I hate to say it, but the time may have come for some younger Keenies.

After a couple hours and a couple shots of house-recipe Fireball, though (I imagine it’s just Hot Tamales soaked in whiskey?), we started to get a little more sociable. Actually, truth be told, I kind of overshot my mark a bit and ended up quite trashed there for a while, handing out sloppy tarot card readings that kept predicting, over and over, that our night would end in disaster. (Which…)

Lots of great people showed up, though – Andrew and Cobi from Taplytics, Ryan from Galvanize, Matt and Simon from Adnostic came all the way from England! Tony from Context.IO was there (Hope your call went well!), Kerry of Night Mode fame came along for the evening. (He’s actually with SolarCity, but I believe popping the lenses out of sunglasses at night will be his true claim to fame). Basically, we were super thrilled and flattered y’all made the hike out to East Austin to hang with us!

THE WALL

After sunset, a bunch of us grabbed dinner at that little food truck area on 6th for the third time in as many days (The people at the Philly cheesesteak truck remembered me from my drunken Nic Cage tirade. Hooray! I have notoriety!), and then we kind of aimlessly wandered for a bit. It’s nice having gotten to know people enough that you start to have random run-ins with them on the street – additional shout-outs to Jo and Crystal and Constantine and Daryna (Sorry I kept calling you Serena apparently? I have no recollection of this.)

We popped into Latitude 30 for a minute, because the line at something else looked horrible, and I stood enthralled with some folk duo called Paper Aeroplanes for like 40 straight minutes. It’s pretty great how you can just walk into any random bar in Austin, and like the best band you’ve ever heard just happens to be playing there, no big. 

After, we met up with Justin’s roommate, Sarah from Sprinklr, who took us all up to the top floor of the Bank of America building to see their offices… then just as quickly took us back out again when it became clear the people up there didn’t super want like 15 rowdy Keen monsters stomping about.

image

*I* wanted to go to this place, but apparently I was in the minority there. (P.S. I like how their icon for food is a beer.)

But THEN we hit this basement party at a wine bar, and we all just kind of hit a wall. Well, not Tim and Justin and Dustin, bless their hearts, who gamely kept up the good fight and mingled the night away, but the rest of us were just kind of done

Days upon days of talking and drinking and partying and wandering finally caught up to us, and even the extroverts among us were pretty much ready to call it. We all found a quiet room off to the side of the main party and formed a foreboding-looking circle of desperate exhaustion and protracted silences. A few kind souls tried to come over and chat, and I’m pretty sure we just made dementor moans at them. 

Like I said before, we love talking to strangers at Keen, but we were just plumb out of extrovert gas, and if you can’t give it your all, you shouldn’t be out there doing it, so we decided it was time to bail. 

ON PARTIES

OK, parties are kind of a passion project of mine, so forgive me for going on the teensy tiniest little bit of a rant here for a second.

The one thing that sort of annoyed me about SxSW as a whole has been a lot of people’s attitude towards the parties here – this weird mix of FOMO and perpetual dissatisfaction and status flashing that kind of makes us come off as entitled and gross.

The entire time we’ve been here, it’s always seemed like, whatever we were doing, some people were always holding out for something better. No one ever liked where they were at, unless they were still in line for it. As soon as you got to a place, you’d start planning for the thing you’d go to next. And I understand that there is something really fun about the thrill of the chase – the seeking matters more than the finding and all that. Rumors flying around, sharing secret passwords and passes with friends, the common struggle of standing in a boring line together – that stuff is totally fun in its own right. But it made me a little sad how unappreciative and impatient some people were with the parties they were already at.

It sort of seemed like, underneath it all, everyone was always in search of this one mythical Perfect Party – the platonic ideal of a party – which I found pretty funny, because every event we showed up to usually ended up being pretty much the exact same as the last: A buncha people standing around, sipping drinks, and chatting. And not that that’s bad by any stretch of the imagination, but what exactly is everyone hunting around for, anyway?

(Admittedly, there might have been a bunch of crazypants parties that I just totally missed out on – raves and drug orgies and costume balls. Maybe I just don’t hear about the cool shit?)

And then, even with the events that did seem kind of big and fun and crazy, it seemed like lots of people enjoyed them more for the bragging rights than because they actually had a good time? Like, I was talking with this one guy, and he was like:

Guy: “Yeah, so I managed to get VIP passes to see Spoon at such and such a thing…”

Me: “Oh, awesome! Do you like Spoon?”

Guy: “Eh, they’re OK.”

Me: >_<

I guess what I’m gradually working my way around to saying is that maybe there is no Perfect Party out there – that instead (and this is totally cornball, I realize), maybe the Perfect Party is inside all of us. Rather than scouring around, waiting to be handed some sort of magical, perfect event that has been custom-tailored to give you everything you’ve ever wanted, why not appreciate all of the stuff we have been given and then try and turn it into something even more magical yourself?

I have helped host events before – I spent no small amount of time helping put KeenCon together, for instance – so I have at least some small idea of the absolutely insane amount of time and energy and resources that have gone into creating every single one of these parties, parties that most people seem to start writing off as soon as they get into them.

I think the least we can do is take a second to actually appreciate all that effort that’s been done for our benefit, and try to return the favor by being awesome, amazing party guests in turn. You want a crazy dance party? Grab a bunch of people and get them moving! You want to get into some hijinks? Start thinking some up! A party is a symbiotic beast – if we want them to be great, we have to give back as much as the party gives us.

OK, sorry for getting preachy. I hate to generalize like this – I met tons and tons of people who were having a great time wherever they were at – but I think there is a lesson here we could all maybe benefit from a bit.

DANCE MESS

OK, back to business. Heading out, we knew we didn’t have the right energy to really get into deep conversations any longer, but we did still have enough energy to bond through the connective power of dance!

So, we asked Taylor, our resident Austinite, to take us to all the hot spots where the youths go to dance. (Taylor is 22, by the way, and seems to derive a sick pleasure in reminding the rest of us how grotesquely old we all are. “Oh yeah, I listened to that band…. in MIDDLE SCHOOL.” AUGH, WHY, TAYLOR.) 

Unfortunately, even during Sx, a Sunday night is still a Sunday night, and the dancing prospects were pretty, pretty dire. We began a comically depressing march down 6th St., poking our heads in any place that looked remotely promising and then pretty much immediately fleeing in terror at the sad, deserted dance floors. We also began mainlining tequila, it should be said, to try and increase our odds of finding this whole endeavor acceptable.

image

Yeah, that face pretty much sums it up.

Eventually, we’d surveyed pretty much the entire lineup of bars, with no luck. So, with grim resolve, we made our way back to a place we’d seen early on but had instantly dismissed out-of-hand because it was so, so not our scene. 

KRAVE. 

KRAVE. Whose name I mention only so I’ll remember to never go there again.

KRAVE. Which was definitely 100% an 18-and-over bar and therefore completely mortifying as a 30-year-old.

KRAVE. Whose DJs seemed more concerned with yelling catchphrases than playing actual music. (”Y’ALL READY TO DANCE? MAKE SOME NOISE! WHO’S READY, AUSTIN! HERE COMES THE BEAT! HERE IT COMES! HERE IT COMES! adhfuiahsuhq834&09r3augh just play the song you fuckers)

KRAVE. Which featured a crowd of immobile, incredibly angry-looking women in the center of the dance floor, who seemed to be there mostly to glare at us for existing.

KRAVE. Which ate my lovely blue cardigan, never to be seen again.

KRAVE. 5 stars. Find us on Yelp.

OK, actually, it was pretty fun, in a buswrecky shitshow sort of way. It’s certainly the most I laughed the entire trip, just at the pure molten horribleness of it. I don’t think there has ever been a place on earth that was less our scene.

We held on for about 45 minutes, and then, our dance task complete, we headed home and all piled in the hammock for a while, while Tim regaled us with the tale of the 2 kinds of Domino’s Pizza he had eaten that night. (Thin crust and regular!) 

And, yeah. That’s pretty much it. Mischief managed. I am now on a plane, heading back to SF, and already feeling the initial onslaught of the inevitable death illness that allows follows trips like this. South By SARSWest, I think someone called it. 

I GUESS THIS IS IT?

Hey. Thanks. 

I mean it.

Thanks for reading, for sharing, and for being part of these adventures. Thanks for talking to us and hanging out with us and hosting us and buying us booze and food and teaching us new things and becoming new friends and just being rad in general. Thank you, Austin, for being a wonderful host and a perfectly lovely city I am trying to con my parents into moving to. Thank you, Keen, for letting me come along on this trip and document it in whatever weird way I wanted to (although, admittedly, I didn’t really “ask” so much as “just started posting shit”).

I’m not sure these field reports were useful in any particular way, or even all that representative of the SxSW experience, but I do hope they were at least sort of entertaining. It’s weird. I feel like I did so much over the past several days, and yet there was so much that I missed, too. But this was just my random experience. I would totally love to hear about yours, if you ever wanna drop me a line.

If you liked these field reports, let us know, and I imagine we’ll keep doing them for future events. If you didn’t like them, let us know as well, and I will begin self-flagellating IMMEDIATELY. I guess I also have a blog and twitter and stuff, if you are just super enamored with my cool brain and thoughts or whatever, but otherwise, I hope you had as great and weird and exhausting of a South By as all of us at Keen did!

This was a blast; let’s never do it again. (’til next year.)

<3<3
Nate

Nate Walsh

Writer. Party Planner. A Third Thing.