Introducing the Keen IO Community Code of Conduct

A few weeks ago we sent an email to the whole company introducing the Keen Community Code of Conduct. This blog post includes most of that email with a few more things added that we wanted to share with our community.

A few months ago the work began on the Keen IO Community Code of Conduct. We’re very excited to announce v1.0 of the Keen IO Community Code of Conduct is now public. 🎉

This Code of Conduct applies to all Keen IO Community spaces, such as the Community Slack group, open source projects, Keen IO meetups, Happy Data Hours, and more! It will be added over the next few weeks to different projects and other community spaces.

It is the product of many meaningful conversations and advice from many Keenies and other humans from outside of Keen IO. To anyone that contributed to this Code of Conduct, thank you. The process of creating a document like this isn’t easy, and we have so much respect for anyone who has done it before.

The Code of Conduct is a living document. This is only v1.0. It will grow and change with Keen IO and its community. This is why it is on Github. Issues can be created to help with revisions and updates. There is also a feedback form, which can be filled out anonymously. Feedback is always appreciated. It will also help guide training and more internal procedures for the Community Code of Conduct.

Lastly, we’re looking forward to making it even clearer to our community that we are dedicated to providing a safe, inclusive, welcoming, and harassment-free space and experience for all community participants, which will help grow our community in amazing ways. We hope this Code of Conduct clearly states what behavior is expected and not tolerated as well as establishes a path for community members to report possible incidents and seek help.

Please feel free to ask me any questions! I would be more than happy to have a larger conversation about it and its existence. 😀

Taylor Barnett

developer, community builder, and huge fan of tacos

Keen IO Secures 14.7M in Series B Financing

Press release originally posted to Business Wire

Keen IO Secures $14.7M in Series B Financing to Accelerate the Adoption of its Leading Cloud Analytics Platform and Intelligence API

Financing Led by Pelion Venture Partners with Continued Participation by Sequoia

Keen IO, the leader in cloud analytics, today announced it has closed $14.7 million in a Series B financing round. This funding brings Keen’s total capital raised to date to approximately $30 million and supports the company’s accelerating global growth in the enterprise.

Keen now provides service to more than 3,000 customers and recently crossed the 50,000 developer milestone. The Keen platform has been integrated across a wide range of categories, including mobile, gaming, e-commerce, media, IoT and retail. The company also recently launched Native Analytics, a comprehensive product offering enabling customers to white-label real-time analytics and custom visualizations at Keen IO scale.

“We are thrilled to announce this financing and welcome these visionary investors as partners to our team,” said Kyle Wild, CEO and co-founder of Keen IO. “Our original vision of making the world’s best data science tools available to all developers is continuing to bear fruit. We now see our platform taking hold as an ‘Intelligence API’. In our fast-moving, hyper-connected world, everything that connects to Keen’s API becomes more intelligent through the power of data science.”

Investors Align to Make Keen IO the Standard for Cloud Analytics

Keen IO’s Series B attracted a prolific group of investors. The round was led by Pelion Venture Partners, with continued participation by Sequoia Capital, who led the Series A financing. Several other institutional investors, including Rincon Venture Partners, Amplify Partners, and Rothenberg Ventures, also participated, joining original seed investors Techstars, 500 Startups, Heavybit, and Galvanize Ventures.

“Keen IO is providing the best of breed data and analytics platform for the API economy,” said Ben Dahl, partner at Pelion Venture Partners who will join the board of directors as part of this financing. “We see expansive potential for the impact Keen can have as it continues to scale its footprint from the mid-market to the enterprise, empowering its customers to have analytics as a core competency.”

Hewlett Packard Pathfinder joined the Series B as a strategic investor and partner.

“As part of Hewlett Packard Pathfinder’s mission, we invest and partner with innovative startups like Keen to help them scale while meeting the needs of our enterprise customers,” said Lak Ananth, managing director of Hewlett Packard Pathfinder. “Keen uniquely addresses the need for our enterprise customers to embed analytics effortlessly and ubiquitously into their applications.”

The Next Phase of Growth for Keen IO and the Intelligence API

The company hosts a highly scalable cloud API that allows companies to collect, enrich, pipe, store, and analyze custom event data. Customers integrate Keen into all parts of their internal business infrastructure, marketing pipeline, and customer-facing products.

“We are seeing accelerated adoption of Keen’s Intelligence API,” said Will O’Brien, COO of Keen IO. “We are experiencing rapid growth of new accounts and new industries, while at the same time our existing customers are finding new use cases and integrating Keen in every part of their digital ecosystem. We are honored to be a core partner to their amazing and innovative journeys, and this financing will enable us to meet the increasing market demand even faster.” Keen employs 50 people and is headquartered in San Francisco.

// About Keen IO

Founded in 2011, Keen IO (https://keen.io/) is the leading cloud analytics platform used by more than 3,000 customers to collect, enrich, pipe, store, and analyze custom event data at a truly massive scale. The company has raised approximately $30 million in venture capital from Sequoia Capital, Pelion Venture Partners, Hewlett Packard Enterprise, Rothenberg Ventures, and other institutional investors. Keen IO is a core platform to customers in a variety of industries, including media, e-commerce, adTech, gaming, IoT and retail. The company’s cloud analytics API queries trillions of events daily, with operational performance required by the enterprise. Keen hosts an open-source visualization library that is the largest of any private company with over 900 forks and 8,500 stars on GitHub.

To get started with Keen IO, visit keen.io or email us at team@keen.io

We are hiring! Read about our job openings: https://keen.io/careers/

Press contact:

Joe Volat

press@keen.io

(408) 645-0448

Will O'Brien

COO @ Keen IO

Announcing our Series B

Hi! I’m Dan, one of the co-founders of Keen IO. If you’re reading this, you might already have seen our official press release announcing the closing of our Series B round of financing.

My co-workers and I have been building this company for four and a half years. Like most entrepreneurs, we’ve poured our fair share of sweat and tears into this company. And, as our press release says, we’ve accomplished some amazing things. Through a combination of hard work and dumb luck, we’ve got over 3,000 customers. Our developer community has passed 50,000 members. We have some of the best investors in the world. And we’ve just raised another 14.7 million dollars.

These are all true things. Which is wonderful, and grand, and lovely.

But.

We have plenty of opportunities to trumpet our successes. I’d rather not contribute yet another “we raised money so we’re the best company that’s ever existed” blog post to the Valley echo-chamber. And right now I’m asking myself: “What do I want you, my kind reader, to know about Keen IO?” It’s not an easy question for me to answer. There are lots of facts that I want you to know.

If you’re our customer, I want you to know how appreciative I am. We wouldn’t exist without you. Full stop. You’re the reason we wake up in the morning (and sometimes the middle of the night). If you’re happy with us, I hope we continue whatever it is that makes you happy. And if you’ve got some frustrations, I hope you let us prove to you that we can fix what you’re frustrated about.

If you’re a potential customer, I want you to know what problems we can solve for you. I want you to make an educated buying decision in this increasingly noisy and complicated analytics landscape. I want you to solve the analytics or data problems you’re thinking about. And, in the best cases, I want you to use Keen and get promoted for doing so.

If you’re an existing investor, I want you to know how much you’ve taught us. We’ve come a long way since three best friends with a vague idea that building custom analytics solutions should be a lot easier. You’ve been an instrumental part of that. And, I hope that we’ve taught you a thing or two as well.

If you’re a potential investor, I want you to know that you’re not going to hear much from us for a while. Let’s talk in a bit. I’m looking forward to pausing VC conversations and focusing entirely on building our business. ;)

If you’re a curious reader, I want you to take something useful away from this. I hope I’ll make that happen for you!

Finally, if you’re one of my co-workers, I want you to know that you’re the special sauce. You’re the reason I love working at Keen. I want you to know how important you are to me, even if I don’t show it all the time. I want you to know I’m a big old introvert and me not hanging out at happy hours doesn’t mean I don’t think you’re amazing. I want you to know how grateful I am to work with you.

And that, I think, might be the key to all of this. For me, at least. Gratitude. It’s easy to lose sight of how grateful I should be that I can do the kind of work I do, with the people I do, and for the customers we have.

So let’s end on that note.

I’m grateful that we raised $14.7 million in our Series B from world-class investors.

I’m grateful that we will use that money to continue serving our amazing customers.

I’m grateful that we will continue to serve our customers by building innovative analytics products backed by really cool technology.

I’m grateful that we will continue to build a workplace that invites its members to bring their full selves to the office.

And now I’m grateful to gently close one chapter of Keen IO’s history and begin the next. I can’t wait to see what it brings.

Daniel Kador

Software engineer, entrepreneur, geek, all around okay guy.

Follow Stream to Cabin & a Native Analytics wonderland...

Yesterday we told the world about Native Analytics - the latest offering from Keen IO, which is helping developers and businesses embed analytics and deliver insights to their audience (customers, partners, etc) via custom visualizations within their user-facing products.

Today, we’re announcing a project in collaboration with our friends at Stream.

The project is called Cabin - cute, huh? Cabin offers developers a series of tutorials to teach you how to build an application using React and Redux. Each installment in the series is based on a unique technology (APIs, cloud-services, etc) that makes it easy for a developer to add meaningful functionality to her app. With the powers of each “component” combined, one can create a full-featured application with relative ease.

Where Keen fits into Cabin

One important aspect of Cabin is its analytics page, which displays stats like how many people have viewed your profile and the number of views for each image you’ve submitted to the Cabin community. They used features from Keen’s Native Analytics to power the stats page of the Cabin React example app. Here’s a screenshot of what it looks like:

How did this come to be?

Stream - the API for scalable feeds - asked us to join them and several other developer-tools companies to create Cabin, which is, at its heart, an tutorial-based educational experience to help developers learn how to build a feature-rich, scalable social network application.

We’re quite stoked for this opportunity to work with other great companies to support the growth and advancement of software development, while illustrating the powerful possibilities that open up with the combination of some of the world’s most advanced APIs and platforms.

If you’re stoked, too, head on over to Stream’s blog for the full tutorial. And go here to learn more about the Cabin project, other participating companies, and further tutorials for learning fun!

Tim Falls

community crafter and scarf enthusiast

Why We Built Native Analytics

Four years ago, my friends and I embarked on a quest to build a business. We were going to build an analytics API. We dreamed big. Our platform was going to be so elegant and so powerful that, basically, anyone building anything connected to the internet would want to send data there. It would help people understand. It would help them automate. It would help them explore. It would help them discover. It would be the nervous system for the next great version of the internet.

In those four years, we’ve made some progress :). We built a database in the cloud. We built an API that allows you to query it, in real time, from anywhere. We built charting libraries so that developers could paint their apps with pies and bars.

And the whole time, we watched, delighted and amazed, to see what developers would create with these building blocks we had fashioned for them. We beamed when they showed us their dashboards. We cried when a real life rocket scientist showed us data from the Mars rovers. We watched, and we listened, as we learned, as thousands of developers integrated Keen into their products and built their businesses.

One of the main patterns we noticed was that many people were using our platform not only to analyze their business data, but to display data in the applications they were building for their users.

I’ve always thought of our business as a very organic thing, a jungle with wide-reaching and thickening roots, spreading and collecting data from all over the earth (and space!). But there is so much more to it than collecting data. The most exciting part has been seeing how people have taken that data and reflected it back out into the world. To see those roots grow into trees, and branches, and a striking variety of leaves and flowers.

It was this common usage pattern, alongside the growing demand for analytics views in nearly every modern SaaS application, that inspired us to build a number of core features that make building analytics natively into your application even faster and more powerful.

Today we’re announcing Native Analytics, a core product on the Keen platform. I hope you’ll take a peek at our new landing page, our product overview, and possibly even drop a note (see chat button, bottom right!) to talk about building analytics into your product. We’ve learned a lot over the years and look forward to building and sharing with you as we continue to grow.

Michelle Wetzler

Chief Data Scientist

Introducing Native Analytics by Keen IO

Press release originally posted to Business Wire

Cloud Analytics Leader Keen IO Announces Industry-First Solution for Native Analytics

Comprehensive Product Offering Enables Customers to White-Label Real-Time Analytics and Custom Visualizations at Keen IO Scale

June 22, 2016 – SAN FRANCISCO, CA – Keen IO, the leader in cloud analytics, today announced the availability of Native Analytics, a complete solution for customers to white-label the real-time analytics and advanced visualizations of Keen into their own products and services. A milestone for the analytics industry, Keen’s Native Analytics leapfrogs the conventional approaches by point solutions and walled-garden data warehouses by opening up the full power of its platform API and unparalleled visualization library to the capable hands of developers.

“With Native Analytics, we are answering the demand from the market for analytics solutions that are fully customizable and pushing ahead with our core vision of making the world’s best data science tools available to all developers,” said Kyle Wild, CEO and co-founder of Keen IO. “While other products arbitrarily constrain you to a set of rules for your data models, queries, and visualizations, Keen’s Native Analytics lets you delight your end users by seamlessly embedding the magic of Keen into your app, website or service.”

Founded in 2011, Keen IO now serves over 3,000 customers across a variety of categories, including media, e-commerce, adTech, gaming, IoT and retail. The company’s cloud analytics API queries trillions of events daily, with operational performance required by the enterprise. Keen hosts an open-source visualization library that is the largest of any private company with over 900 forks and 8,000 stars on GitHub.

Native Analytics is the latest in a series of product offerings designed to address the rising demand for real-time, scalable, and extensible analytics that can be deeply integrated into end-user experiences.

How Customers Are Using Native Analytics

Customers are rapidly adopting Keen’s Native Analytics into their core products, saving them time and money, and helping them separate from the pack in their respective industries.

Bluecore, an e-commerce automation platform used by Newegg, Staples, and BCBG, was an early adopter of Keen’s Native Analytics. Bluecore’s customers are presented with real-time dashboards and critical data for shopping cart abandonment and email recapture campaigns; all white-label Keen.

*Bluecore Powers Campaigns for Large Brands with Native Analytics from Keen IO*

“From day one, we needed to be able to show customers how their e-mail campaigns were performing,” said Mahmoud Arram, co-founder and CTO of Bluecore. “Keen is core to our product.”

Pixlee, a visual marketing platform used by Levi’s Jeans, Kimpton Hotels, and Converse, embraced Keen’s Native Analytics early to power its core product. Pixlee uses Keen to provide a highly customized analytics dashboard with real-time, beautiful representations of end user engagement and conversion. The Pixlee dashboard presents custom metrics like “lightbox engagement” alongside traditional metrics like pageviews and conversions.

Keen’s cloud analytics platform helps Pixlee optimize the content they display on their clients’ sites. With this approach, Pixlee delivers an 8-to-20 percent increase in engagement over a purely randomized approach.

*Pixlee Delivers Real-Time Visual Marketing Analytics with Native Analytics from Keen IO*

“With Keen’s Native Analytics, our customers can see their data and say, ‘Wow, this is actually working.’ It builds trust with our customers and really sets us apart from competitors,” said Jeff Chen, co-founder and director of engineering at Pixlee. “Keen saved us months of work building real-time analytics into our product. Our clients love seeing actionable insights as a native part of the user experience.”

Keen has seen more than 40 percent of its customers adopt Native Analytics ahead of the commercial launch. The company expects a groundswell of new business from the general availability of this industry-first offering.

Moving Data Science from a Cost Center to a Growth Driver

Before Keen’s launch of Native Analytics, companies have broadly viewed data science as a necessary cost to try to understand their business and marketing activities. Now, with Keen IO, data science has transformed to a proven ingredient for customer and revenue growth, as analytics becomes promoted to a core product feature.

“We are very excited by the business innovations coming from our customers adopting Native Analytics,” said Will O’Brien, COO of Keen IO. “Some customers are turning advanced analytics into a paid feature, while others are bundling analytics into their core product to improve retention. We all recognize that we live in a ‘moneyball’ society where users today expect stats and charts in each and every one of their favorite apps and websites. Keen’s Native Analytics lets every company compete on data instead of on price. It’s a new game.”

Getting Started with Native Analytics

To get started with Native Analytics, visit http://www.keen.io/native-analytics or email a Keen IO team member. Existing Keen Pro customers will get priority access to these advanced capabilities of the platform.

// About Keen IO

Founded in 2011, Keen IO is the leading cloud analytics platform used by more than 3,000 customers to collect, enrich, pipe, store, and analyze custom event data at a truly massive scale. The company has raised more than $15 million in venture capital and its last round of funding was led by Sequoia Capital. Keen IO is a core platform to customers in a variety of industries, including media, e-commerce, adTech, gaming, IoT and retail. The company’s cloud analytics API records billions of events daily and runs queries on the trillions-of-events scale, with operational performance required by the enterprise.

Learn more about Native Analytics from Keen IO: keen.io/native-analytics

We are hiring! Read about our job openings: https://keen.io/careers/

Press contact:

Joe Volat

press@keen.io

(408) 645-0448

Will O'Brien

COO @ Keen IO

Announcing: Keen Dashboards

We have some big news to share today!

We’re excited to announce Dashboards Beta release, a brand new tool that gives anyone the power to create custom dashboards in just a few clicks (or taps!)

Example of a custom product metrics dashboard for a customer success team

Keen Dashboards can be used to:

  • Quickly create custom dashboards with a point-and-click interface
  • Share sets of metrics around KPIs across every team, project, and department
  • Give anyone on your team from analysts, marketers, to product managers the ability to query, create, and edit their own set of metrics (no need to know SQL!)

For the past few years, we’ve focused on making it super easy for developers to explore and discover the data that matters to your business with Keen’s powerful analytics platform and API. Many of you have used our Data Explorer, open source dashboard templates and visualization libraries to create beautiful and responsive displays of your most important metrics wherever you need them.

With the release of Keen Dashboards, we’ve combined the power and flexibility of a fully custom analytics solution with a simple interface for exploring and visualizing your most critical data. Now anyone across your organization can create and share dashboards, eliminating bottlenecks and making it easier than ever to follow the metrics that matter to you and your team.

How does it work?

  1. Log in to your Keen IO account and save a few of your favorite queries!
  2. Click on the “Dashboards” tab
  3. Click the “Edit” button to activate the Editor
  4. Add rows and configure each row’s tile layout
  5. Select a saved query for each tile
  6. Save and close the Editor to see your awesome new dashboard!

Ready to create yourself a beautiful, custom dashboard unique to your business? Log-in or sign up for Keen today and start putting your most important business metrics in one place or schedule a demo!

ps. Want to help us design and build awesome tools for our customers like Keen Dashboards and Data Explorer? Join us!

hack.guides() Tutorial Contest!

image

Are you obsessed with building Dashboards? We are excited to sponsor a $500 prize for the best guide using Keen IO to power dashboards in their apps.

Over next six weeks, you can submit tutorials and collaborate with hack.guides() developer community on the best practices, hacks, and tricks using keen, RethinkDB, and other partners in production.

http://tutorials.pluralsight.com/contest/

The hack.guides() tutorial contest is unique and brings the developer community together to create and curate collaborative guides. With the Github API on backend, hack.guides() tutorials can be forked, improved, and merged just as simple as a pull request.

Submit your post here and the share your posts with us on Twitter to spread the word.

Justin Johnson

community guy, hacker, music nut. i like to help people build stuff.

How Net-a-Porter scaled developer culture for their massive e-commerce platform

Here’s the scene: your company was one of the pioneers in bringing retail to the internet. Your store has millions of monthly visitors. During your massively promoted sales, you see HUGE spikes in traffic. Every sale counts. And everything is running on one giant Java application.

This story isn’t fiction, it’s Robin Glen’s job. In this interview, he tells us about how his dev team re-architected the massive designer fashion site Net-a-Porter, and the organizational changes their technology teams made to do it.


Net-a-Porter: Sometimes you need a $6,000 leather skirt!

What’s the focus of the tech team at Net-a-Porter?

So, the tech team is split into cross-functional customer focused sub teams; my team is made up of full-stack developers, test specialists, product owners, delivery managers. We’re responsible for the listing pages, product pages and site search - basically the entire product catalog. Right now our team is replatforming our architecture into micro-services and working on this concept of “headless commerce”. This allows for the entire e-commerce platform to be front-end agnostic.

How did the decision to move to “headless commerce” come about?

When Net-a-Porter was started, our e-commerce site was one big giant Java application, which is probably quite a common situation. Our delivery rate was too slow and we wanted to move towards continuous integration, but we had this big application that didn’t have great testing around. We needed to improve and evolve.

Was there a specific problem you were trying to solve?

We get a huge amount of web traffic whenever we launch a sale. There’s a massive amount of physical hardware you need to have for a short-term sale, which results in a lot of redundancy during normal times of traffic. This was obviously inefficient and a pretty big problem. We decided to break out the sales part of the site into a listing application in the cloud so we could have horizontally scalable applications to handle sale traffic.

This was a huge success and kicked off the concept of dev ops at Net-a-Porter, so developers were doing the ops and we were building full stack applications ourselves. This really opened the floodgates. As soon as the sale application was out, we began to look at every part of the site to see how we could make it scale horizontally, we are still on that journey.

How’s it going so far?

We’re still pushing towards Continuous Integration. We can do multiple releases a day. We don’t need regression testing, so we can actually move forward a lot faster. At a company of our size, innovation obviously gets a lot more expensive because you have to stay so far ahead, so all these things are helping us to keep moving forward.

Another interesting thing is the cultural shift that has occurred as a result of this change. It’s promoted a lot of ownership. Everyone in our team is able to quickly iterate and find out what’s been successful and not successful. The developers who write the applications are now also responsible for supporting them and for setting up monitoring alerts to make sure things are running.

It sounds like monitoring and alerting are pretty critical to ensure things are running. What role do performance metrics and analytics play in that process?

My colleague Matthew Green and I were tasked with answering the question “How can we make the website more performant?”. But in order to do that, we first had to answer the question “How can we measure our current performance?”. You can’t improve what you can’t measure.

So we started tracking browser performance metrics and reporting that data to Keen IO. We even extended this to collect performance metrics on client-side API calls. So now if I want to know how long it’s taking to see what’s in someone’s basket, or how long it takes to add something to the basket, we can do that.

As our experience and understanding grew, we started to add more granular metrics to help us identify and diagnose issues quickly

For example, when the website throws a 5xx error we can dive into real customer errors and I can say, “This page errors the most. In this country, that is where the problem is happening.” We use Keen to track any type of error into complete granularity. It enables us to identify errors in a customer’s experience across all parts of the site. Of course, with an e-commerce site, errors can be very expensive if you don’t diagnose and fix them quickly.

What makes my job easier is knowing: Are things working? Are they being used? If they are not, we can alert on them. We can do out-of-hours calls like, “Okay, the 500 errors have gone up. Send a text to the team,” or “The website is running slow. Send a text.”

Why did you choose Keen IO as your monitoring solution?

Keen gave me the granularity I wanted, complete control. How we use the data, visualize the data, it’s all in our control. We were doing a lot of work to get the metrics we needed, but the monitoring tools we had were not granular enough to tell us if our improvements were working. The data was not democratized. I wanted to be able to monitor anything and make it readily available.

What we’re using Keen for now is monitoring granular performance and availability, and for that it’s been perfect. We have a lot of ideas of what we want to do in the future and because of Keen’s openness it gives us is endless potential.

As an example, we tag all of our events with the build number of each application. We know how long it takes for a ticket to move through our workflow, we know how long it takes to run our build and test scripts, and now we know with how long it takes for customers to start interacting with new features. This data can give accurate development estimates to our business stakeholders.

So we can now accurately say how long a story or feature has taken from inception to real customer interaction.

What are you most excited about for the coming few months at Net-a-Porter tech?

We are market leaders in luxury fashion e-commerce and we’re on the path to making our customer experience even better. All of this re-platforming is going to unlock innovation. I want our customer experiences to feel as slick as a native app.

I’m a big believer in the web as a platform. The reason that it took off originally is because it’s frictionless. You don’t need to find a website in a store, buy it, download it, install it and launch it before you use it. I totally understand why native apps took the lead, they felt more responsive, they work offline, give you access to the device’s hardware, and overall they gave users a better mobile customer experience. This even lead some people to proclaim “the web is dead” and for a while maybe it was. Browser technology was in flux, no one could agree where mobile web was going and it stagnated, we fell behind.

This however seems to be changing. New web APIs are coming through rapidly, and thanks in part to Google’s great work, there’s no reason that we shouldn’t be able to create native-feeling apps on the web right now.

My goal is to show it’s possible to build, test, deploy, and monitor “progressive web apps” in production, at scale, for a large e-commerce websites.


Robin will be joining us in San Francisco on May 17th to dive deeper into how Net-a-Porter scaled their developer culture. If you’re interested in hearing more about their team, the infrastructure they’ve built, and their experience at Google I/O, be sure to register for our event on May 17th. Hope to see you there!

Alexa Meyer

Growth and UX. Cheese chaser. Aspiring behavioral economist.

How should deadlines be used in software engineering?

wocintech (microsoft) - 42
Photo by #WOCinTech

How many conversations have you had about deadlines where at least one person scoffed at the concept? Pfft!

I have heard this a number of times - including from myself - and I want to fix this.

The world of software is very different from the world of print media, where our current concept of deadlines is derived. However, just because there can be no universal definition of deadlines across all software projects doesn’t mean we shouldn’t use them or that they aren’t valuable.

Deadlines are extremely important for planning and parallelizing work. Without expectations about when dependent work will be completed, all teams would have to work in serial. This would drastically slow down delivery times.

At the same time, deadlines can be frustrating and counter-productive if people disagree about what they actually mean.

Problems and solutions

In my experience, these are the most pertinent problems with deadlines in engineering organizations, and the most promising ideas for how to solve them.

1) They are interpreted differently from person to person.

Person A: “The deadline is next week so I’m pulling a ton of extra hours.”

Person B: “Why? It’s a fake deadline and doesn’t matter.”

Person A: “But I don’t want to get fired!”

This shows that a deadline means vastly different things to Person A and Person B. This can lead to confusion and frustration on the team working towards that deadline.

Instead, deadlines need to be a rallying point. Everyone needs to understand why this deadline is important. They need to know what impact missing the deadline has on their world (that can be other teams, customers, or the business as a whole).

More importantly, deadlines that are met need to be celebrated. Wildly. This is often overlooked. Building a culture around celebration for meeting deadlines is a much better practice than berating people for missing them.

2) They are used too early in a project life cycle.

Person A: “Hey we need to accomplish [insert_really_hard_unknown_thing]. When can we have that done by?”

Person B: [quickly wikipedia'ing what that thing even is] “uhhhhh, I don’t know.”

Person A: “I need a timeline!”

Person B: “Thre–FOUR! mmmmo-wee-months. Four. Months.”

Person A: “Great, see you in four months.”

Asking for a deadline when EVERY aspect of a project is unknown is a recipe for disaster. It’s stressful for everyone involved and sets the project up for failure. So let’s take a deep breath. Wait a few days for some exploration to be done. We will be spending time to buy information, but then we can come up with a meaningful estimation of the work. This information will help us set deadlines that have a better chance of being accurate.

3) They aren’t updated often enough.

Person A: “Hey, deadline is in 5 days. Are we still on track?”

Person B: “We’re a little behind but yeah, we can make it.”

Person A: “Great!”

[ 4 days, 23 hours later ]

Person A: “Just checking back on that project. We good to go?”

Person B: “Uhm, no we’re not gonna make it. Something came up. It looks like another week of work.”

Person A: “$%@!*”

In this case, the deadline wasn’t moved or re-evaluated when new issues became known. Instead of raising that flag immediately, Person B waited until the deadline itself to inform others. Now Person A is affected as well AND the team gets to feel the stress of coming up on yet another deadline in the future.

Deadlines shouldn’t be set to force humans to do inhuman amounts of work. They should be used to set expectations externally and enable a sense of predictability for planning. They need to reflect reality as accurately as possible or trust is lost and they can no longer deliver that predictability. Now, I’m not advocating for changing deadlines hourly or daily. But perhaps weekly, or at the very least within a standard planning cadence.

Updating a deadline isn’t limited to extending the date. Scope reduction is also an option. Choosing which action to take (or a combination) is a conversation the engineering and product teams need to have.

4) All the “knowns” aren’t taken into account… just the fun ones.

Person A: “How long to ship this feature?”

Person B: “Two weeks.”

[ two weeks later ]

Person A: “Why isn’t this done?”

Person B: “Well, it technically IS done. Now we’re working on testing it, building a new deployment mechanism for it, and we’re gonna roll out a beta first. Also, I was on vacation last week.”

This deadline was made without a complete understanding of the work to be done, and the time to dedicate to it. (not to mention Person B also threw problem #3 into the mix.)

We need to make sure we take all known challenges into consideration when determining a deadline. Will we be losing person-hours to a known cause? Vacation? Company offsite? Birthday party which will cause missed mornings due to hangovers?

Also, what un-sexy tasks are we potentially forgetting about? How many tests are we gonna write for this? How do we get this sucker into a production environment? Slow down and think thoughtfully about your process and the availability of resources. It’ll make your deadlines much easier to deliver against AND it will make deadlines easier to defend against scrutiny.

On estimation: uncomfortable but necessary

Deadlines that are set by engineering teams will largely be informed by estimating work. That means that everyone on the team is going to have to get comfortable with being wrong. A lot. Saying anything that you know to be wrong and have little confidence in can be a very difficult thing to do.

We need to have a collective understanding that it’s our best guess and that we’ll get better at it over time. Estimation is a skill that gets better with repeated use. In the early stages, it’s going to feel uncomfortable, but we need to do it.

Estimating tasks

Before we can define the delivery date of a large project, we should break the project down into tasks, where tasks are defined as things we believe we can accomplish in roughly 5 workdays or less.

Some helpful questions to ask when estimating a task:

  • Is the project green field or in a pre-existing area?
  • What is the quality of code in that area?
  • How well do I know the area of code?
  • How familiar am I with the programming languages involved?
  • What touchpoints or integration points with other pieces of code are there?
  • How good is existing test coverage?
  • Does this work touch any business-critical areas? (write path, billing, load balancers, signups)
  • Has anyone done this work before? What are their thoughts?
  • What are the tradeoffs I am making?
  • What is the objective of this task?
  • Does this task need to be done at all?

Estimating projects

Projects are typically defined as a larger body of work that multiple people can work on in parallel.

Some helpful questions to ask when estimating a project:

  • How much time will we actually be dedicating to working on it?
  • What is the objective of the project?
  • Do we have any known days off?
  • What are ALL the tasks required to be done?
  • Do we have any blocking dependencies on other teams?
  • Are any tasks blocking other tasks in the project?
  • Is any new infrastructure/hardware required for this project?
  • What are the Doneness Criteria for this project?

Doneness criteria

Even knowing when something is done is difficult. Different roles within the team can have different perspectives on “done” so we need to have specific criteria that can determine what that means for the project.

Some examples of typical Doneness Criteria:

  • Deployed in production
  • Tests fully automated
  • Communicated internally or externally
  • Spent a certain amount of time in an internal or external pilot
  • Documentation in production
  • Education of sales or marketing team complete
  • Landing page launched
  • Analytics and tracking
  • Operational runbook and observability

Conclusion

Deliverability is essential to any company as it grows and matures. Deadlines are a major tool in the toolbelt. When used properly, they are incredibly useful. But it takes time and practice to get better at using deadlines. So, I suggest that engineering organizations treat them as alive and breathing, continue to learn about them, and document shared experiences internally and with the engineering community.

If you have any tips or suggestions on using deadlines effectively, we’d love to hear them. You can drop me a line at ryan@keen.io or ping us on Slack

Subscribe to stay up to date on future software engineering posts (no spam, ever):

Ryan Spraetz

Problem Solver — in business, coding, building, and bouldering.