A metrics store is a centralized and governed place for organizations to store key metrics. It is a way of centralizing organizational knowledge, creating a repository for stakeholders to access key metrics in a repeatable way, regardless of how they plan to consume them downstream.
Until recently, metrics stores were built in-house. Companies with large data teams (like Airbnb) built internal metrics stores to standardize metrics definitions across data sets and data tools. This helped streamline experimentation and at a larger scale, it helped get everyone on the same page when it comes to defining important indicators like “revenue,” “new users,” or “customer growth.” Today, data teams can purchase this functionality from off-the-shelf SaaS tools.
In this article, we’ll cover:
Metrics stores are starting to gain traction as part of the modern data stack. That's because having a metrics store can provide some major benefits. With a metrics store, you can:
To understand the value of a metrics store, you need to understand the context. Most organizations have indicators to track success and progress towards goals. Some organizations call them key performance indicators (KPIs) or leading indicators, while others call them metrics. The premise is the same—organizations need to understand if their decisions (or actions) are helping them achieve their objectives. Usually, data teams are responsible for building accurate data sets for the business to track these metrics in downstream business intelligence (BI) tools like Tableau, Looker, or Mode. But not every team uses the same tool, and the underlying data used to define a metric like revenue might live in a few different tables in the warehouse. That means that reporting “revenue” can become a battle of questions like...
“Which definition is right?” Why are there multiple definitions?” “Can I even trust this data?” Even simple questions like “How many sales did we make last week?” can lead to confusion.
By building metrics in code (in a metrics store), data teams have a centralized location to build, measure, and identify business metrics. This also makes your data teams more efficient and strategic because they don’t have to build data sets over and over. Instead, they become enablers, helping business leaders define, share, and champion their metrics across the organization.
Companies design their data infrastructure based upon how they expect to drive decision-making through analytical systems. Typically, these companies manage vast amounts of data, flowing from various different applications or third-party source systems. Data is captured and stored in a data warehouse or lake, depending on the type of analytical or operational workflows that the data is intended for. Some transformation may occur before the data flows to the warehouse (think ETL), but more often than not companies are opting for an ELT approach, where datasets are loaded into the warehouse, then prepared for analytical application consumption downstream. Another approach is when denormalization is performed at the application layer itself, sequestering the metric logic within those bespoke tools (see diagram below).
A metrics store sits conveniently in between an organization’s data warehouse and other downstream tools. Essentially, it acts as a proxy to the warehouse, translating metric requests into SQL queries to the warehouse. In this way, transformation occurs at the 'metric level' —consistently defined in code, accessible to all downstream tools, and centrally governed to maximize insights.
Although metrics consistency is one critical piece of an organization’s success, data teams also use a metrics store for a variety of other important tasks. As Nick Handel explained on the Data Engineering Podcast, teams use a metrics store to:
A metrics store has direct benefits for data and analytics teams, but more importantly, it has a direct impact on the organization as a whole. When everyone is speaking the same language of data, you have less tension over what a metric means or which number is accurate. Everyone has a single source of truth.
The primary people who use a metrics store are:
Data producers (Data engineers, analytics engineers, data analysts) - People who build normalized data sets in the warehouse and who build and test hypotheses. These producers are responsible for providing clean data sets for business users.
Data consumers (Line-of-business owners) - People who like to consume and explore data. They want everyone to use the same definitions for metrics and they want to see metrics sliced by different dimensions.
Data is only helpful when it’s accessible and accurate. Up until recently, there was no centralized tool for this purpose. Analysts used a combination of BI tools and documentation tools to record metrics definitions and lineage. Adopting a metrics store puts all of your metrics information in one place. It becomes a knowledge base for metrics for the entire organization. This means end users have more context and detail so they get more insights out of your data. It also exposes metrics to a broader range of applications and a larger group of people, making your data team more efficient and strategic since they don’t have to build and rebuild data sets for analyses. We see a future where metrics stores are available to everyone, regardless of your organization’s size or industry.
Here are some amazing companies in the Metrics Store.