In July I joined Keen IO, transitioning from working with organizations focused on the software development lifecycle (SDLC) to data analytics. Keen’s analytics SaaS is an end-to-end platform that allows organizations to approach data from an analytics-as-code perspective. To speed my learning process I began creating comparisons across the two industries to better understand the value and benefits that customers glean from using an API-centric analytics platform. (We call it the Intelligence API.)
In this post I’ll be sharing my thought process as I developed a deeper understanding of analytics and how it relates to SDLC concepts: specifically agile, continuous integration, continuous deployment, DevOps, and infrastructure-as-code. For a bit more background on each of these terms, feel free to read my previous blog post.
Applying Agile to Analytics
Agile is no longer just a software development term. It is now ubiquitous across marketing, sales, and organizational design. The primary principles of agile revolve around delivering what the customer needs through short sprints involving collaborative teams.
As I started contemplating analytics-as-a-service, I discovered that organizations want to be able to start collecting data on web, mobile, or IoT devices with ease and almost immediately, rather than having a drawn-out process of building an entire in-house analytics infrastructure from scratch.
The goal is that the heavy lifting should be taken care of on the backend by a service that can be responsive to peaks and valleys and the projected growth of the data collected. It doesn’t take an organization long to get started with an AWS server, and analytics should be no different, where you should be able to start making data-driven decisions to guide product direction, customer acquisition, and resource allocation strategy within days.
Achieving Continuous Integration
Continuous integration grew out of a need to add automation to agile practices, first starting with the concept of accomplishing automated unit tests on the host machine of a developer. This quick feedback allowed developers to fix their code as it was still fresh in their minds and get instant feedback regarding quality.
By using an analytics-as-code approach via API, developers can automatically gather data from cached queries, allowing these organizations to quickly ask questions of large datasets and return near-instant results. This ability to run complex queries without a lag-time allows analysts and business leaders to build momentum on ideas and begin executing on their insights in the time it takes to run a CI build.
Continuous Delivery and Deployment of Data
Continuous delivery and deployment started to become ubiquitous as top web organizations began promoting their ability to roll out code to subsets of their user bases for A/B tests in production, with the ability to quickly deploy fixes and updates while production was still running.
I found this process similar to the way analytics-as-a-service provides flexibility with data models, allowing users to collect and store unstructured data that can be analyzed in the future alongside yet-to-be-collected data from emerging technologies such as Internet of Things, Virtual Reality (VR) and Augmented Reality (AR). In this way, organizations never have to limit their future analytics capabilities by being forced into an overly rigid data model today.
Analytics Through the Lens of DevOps
While all of the unit and production testing grew in importance, organizations realized that they needed to adapt their IT operations teams to be more flexible and able to respond to internal teams’ requests. This led to the prominence of DevOps, where traditionally siloed operation teams became part of agile development teams, with the goal of delivering configured infrastructure on demand.
With analytics-as-a-service, organizations don’t need to rely on specialized data scientists, data engineers, database architects, and datacenter architects to deliver powerful analytics capabilities. Instead, the ability to ask and answer complex questions via code allows every team, from sales to marketing and product to engineering to begin discovering value.
If all of these teams can easily access, query, and customize their data views to match their specific needs then their day-to-day operations will be optimized and they essentially have a data scientist on each of their teams. One great thing I learned about Keen was that companies can even provide this ability to their own customers by integrating data visualizations directly into their products as a native part of the user experience.
From infrastructure-as-code to analytics-as-code
The most exciting evolution that unlocks all of the above capabilities for the SDLC is the adoption of infrastructure-as-code. Initially popularized by Amazon, infrastructure-as-code is now a core tenet of many IaaS or PaaS offerings, where you’re able to modify, setup, or tear down your infrastructure by utilizing an API or CLI.
Admittedly, not every analytics-as-a-service provider follows this model, but Keen has always maintained the API as the backbone of the service, with all analytics capabilities built on top of it: collection, storage, query, visualization, access, account provisioning, and more being added all the time. Since our customers are able to use our services in a programmatic fashion (a la: analytics-as-code), they are able to scale from 1 to thousands of unique projects, collections or analysis by writing a few lines of code. They don’t need to build up a huge in-house infrastructure or assemble a specialized IT and data science team to run the system. A developer is all that’s required.
Closing Thoughts
This post is by no means trying to make a comprehensive comparison of every aspect of these technologies. My goal was to walk through my own thought process in case it helps others with an SDLC background make sense of the comparatively new field of analytics-as-a-service.
If you’d like to discuss these ideas in more detail or if you have any follow-up questions about Keen, analytics-as-a-service, or analytics-as-code, I’d be happy to chat. Feel free to reach out to me at vladi@keen.io
