Roles & Responsibilities of a Successful Analytics Team

After years of working with hundreds of companies, I’ve learned a thing or two about what makes analytics projects successful. I’ve also watched many projects fail.

The most common reason for failure might surprise you. It’s not a lack of data expertise or an integration mistake. It’s simply that the organization forgot to make it anyone’s responsibility to make use of their data. Far too many companies collect months and months of interesting data only to have it sitting in the corner collecting dust. That’s what compelled me to write this guide and hopefully inspire you to add a little bit of structure to your next project. Your analytics implementation should be a wellspring of business value that you tap again and again.

If you can find at least one person who is excited to take on each of the roles I’ve outlined below, I promise your project will be 1000x* more likely to succeed. Not all of these roles need to be fulltime, and one person could certainly play more than one part (or even all parts)! More detailed role descriptions & tips following.

*not scientifically proven!

Roles & Their Outputs

Project Lead → Project plan with scope & timeline

Data Architect → Data model and queries

Product Developer → Implementation of tracking

Analyst(s) → Generation of new business questions

Reporting Developer → Reports for your business

Project Lead

A single individual should be responsible for the delivery of your initial analytics implementation. You probably already know what an effective project manager does:

• Identifies the project’s stakeholders and figures out what they need. They ask, “What are the specific business questions we want to answer?”
• Sets & communicates the goals, scope, and timeline for the project to everyone involved.
• Manages dependencies and identifies roadblocks to delivery.
• Ensures that the project is actually delivered and achieves its goals (e.g. that the data answers the questions that are important to your business).
• Makes sure everyone involved, from the engineers to the product managers, is in sync and understands what will (and won’t) be delivered as a part of the project. This part is important as it’s also very common for people to under and over-estimate what your data will be able to do.

Tips for the Project Lead:

• You’ll get the most immediate payback on your analytics project if you focus on questions whose answers will lead to a direct change in your product or business strategy. An example question might be: are customers from our new campaigns converting to paid (should we keep investing in this channel)? -or- We’re thinking about nixing this feature — can you find out if any paying customers use it?
• Keep the scope of your project as small as possible. Start with tracking only a few key actions that are important to your business so you can quickly answer the most pressing business questions (e.g. what is our retention like for customers who use this feature?). Your immediate, helpful results will get your organization hooked and they’ll soon come up with more questions that you can tackle in the next sprint. In other words: analytics work should be done in an agile fashion, becoming a little bit more in-depth with each iteration. If the scope of your analytics project gets too big (e.g. taking two weeks of engineer time), you risk the entire thing getting put on hold in favor of a pressing product feature or some other business priority.

Data Architect

This is a fancy title but it just means that your team needs someone somewhat technical to create your data model and understand how querying the data works. The data model can be as simple as an email that lists what key actions you’ll track and what properties you’ll include on them. The model helps define and communicate the scope of your project. The data architect helps the team assess what business questions are possible to answer vs versus those that are not. This person is typically not a data science PhD. This role is most commonly filled by one of your app developers or someone adept at modeling things in spreadsheets.

Tips for the Data Architect:

• Take the time to have your data model reviewed by someone who has built one before using the same toolset. For example, if you are using Keen, talk to a developer who has used Keen before. You can also ask your analytics provider to review your data model with you. No matter what toolset you are using, there will be tradeoffs and parts of the solution that don’t work quite like you’d expect. Save yourself some time and talk through your plans with someone who has enough experience to see around corners for you.
• When modeling the data, use the vocabulary of the users and the business instead of the language of the application architecture. For example, you wouldn’t track an event like “state change” since this doesn’t have any meaning to the user of the product, and won’t have any meaning to other people in your company. If you keep the language of your data business-oriented, it helps other people in the organization understand how to query and make use of it.
• Have at least one other person in your organization review your data model and confirm it makes sense to them too. You might find that some labels that make sense to you are unclear to others. For example, something like “uuid” might mean something different to different folks in your organization.
• Don’t reinvent the wheel. If you’re building on Keen, please check out our Data Modeling Guide and Best Practices to learn some tricks and avoid the common pitfalls. If you’re starting a business-critical implementation, pair with one of our data architects.

Product Developer

At least one of your product developers has some pretty straightforward responsibilities at the beginning of your project: get tracking set up. They will add bits of code here and there so that every time a login, purchase, upload, or other action happens that you are able to collect data. If you have many sources of events, say your mobile app and your website, then this work might be done by multiple developers (e.g. a web dev and a mobile dev). In a small organization, the developer who sets up tracking often also plays the role of Data Architect. In larger groups, your developers should work closely with a Data Architect to make sure they model data optimally and that things are tracked and labeled in a consistent format (e.g. “user.id” = “23cv42343jk88” not “user.id” = “fran@cooldomain.com”). Setting up tracking is a relatively straightforward process as most analytics services provide drop-in client libraries to greatly simplify the effort, but your team still has to do the work of deciding what to track and how to name things.

Tips for the developer implementing tracking:

• Make sure you’re implementing tracking according to a data model that makes sense to your organization. If your team doesn’t have a data architect, take on that role yourself and sketch out a model before you get started. This will clarify your thinking and make it easier to communicate your plan with others.
• Implement separate repositories, with separate keys, for dev, test, and prod, so that you don’t get testing and production data mixed up.
• Once you have tracking set up in development, get someone to peer review the incoming data before you go live with it. Your analytics implementation should go through a QA process just like any other feature of your product. It’s easy to make mistakes like sending numbers as strings, naming something in a confusing way, improperly formatting your JSON, or having typos in your labels.
• Here is an inventory of tracking SDKs that work with Keen IO.

Analyst(s)

You’re going to be collecting lots of interesting data, but it won’t be very valuable unless someone uses it! You’ll need at least one person on your team who is very curious about what that data might reveal. I call these people analysts. Very often the analyst is a developer, product manager, or someone on the product or marketing team. Not only will these folks be dying to see the results of the business questions they set out to answer, they will be continuously thinking up new questions. Analysts love digging into the data you collected in the first phase of the project and will have a lot of ideas of what new things you can collect in the next phase. In other words, you need people on your team who enjoy the practice of analytics. Don’t worry, there are lots of people out there who do :). Having a technical background will be a huge asset for this person as they will quickly learn how to build queries to get the results they need. This role is absolutely critical for your success because if you don’t have people who want to learn from your data, you won’t be able to extract any value from it.

Tips for the Analyst:

• The results of your analysis may be super meaningful and obvious to you, but they won’t be to anyone else. That’s because you know what questions you were looking to answer when you set out to do the analysis in the first place. You know exactly what data the dataset includes and excludes. Plus you wrote the queries that ultimately produced the visualization or report you’re looking at. That’s a lot of context that you need to share in order for other people to understand what the numbers mean.
• When sharing results of your analysis, write out the conclusions you are drawing from the data and what business actions you think should be taken as a result of the analysis (e.g. our conversion decreased with this latest release and we should roll back). Not only do other folks perhaps not have the context to interpret the data correctly, they probably don’t find it as fascinating as you do and may not have the time to derive meaning from the data.
• Not to hammer on it too much, but communication skills are so important for this role. Around half of the analyst’s time needs to be spent on communications. It takes quite a bit of time to explain and summarize the results and conclusions you’ll draw from your data. If the results of your analysis are sleeping in people’s inboxes, you’re not doing it right. Sometimes you may be the only person in the organization who knows about a problem or opportunity, and it’s your responsibility to make sure the organization is responding appropriately to what you’ve learned. Sometimes you gotta be the squeaky wheel. Don’t underestimate the value of your work.
• If analysis work is something you repeatedly run out of time to do, try getting it added to your official job description and dedicating a certain number of hours per week or per month to it. Block it off on your calendar.

Reporting Developer

This role is optional, but you may want to build out some reports that can be easily accessed across the team or by other stakeholders. You can greatly increase the utility of your data by incorporating it more closely into your business workflows, rather than leaving it trapped in a database where people have to remember to login to check it. A front-end developer will be able to turn queries into a reports for product managers and others across the business. As an example, you may find your results are most useful:

• emailed weekly in a summary report (example)
• on a page on your internal website (example)
• in a customer-facing app (example)
• published to a google sheet
• pushed to a slack channel (example)
• displayed on a dashboard (examples)
• pushed to salesforce (example)

Tips for the developer creating reports:

• Get the most value out of your work by making sure the consumers of your reports understand the data. One way to do this is to ask them “When you see that conversion is 5.2%, what does that mean to you? How would you guess it is calculated?”.
• Another way to increase report literacy is to put a guide (e.g. tooltip or footnote) that explains where the data comes from and how it is calculated. For example, does the data include users from the website and your mobile app, or just one of those? Does it include test users and internal users from your company, or have those been filtered out?
• Have fun! Watching someone’s eyes light up when they discover something new is the best part of the whole analytics project, and you’re often the one helping bring that realization to life.

Starting an important new implementation and need some help? Our team of data architects has done this lots of times and can answer any questions you have! Just drop us a note using Keen’s support channels.