here is an update on the workstream around setting up the business analytics (=Key Performance Indicators, or Metrics) and processes & tools to measure them.
Objective
The objective is to provide a simple way to keep on track of important KPIs for everyone, compare (+ visualize) them over time and eventually being able to aggregate them on a global level and compare between instances.
Metabase KPI Dashboard “Test”
here is the link to the first version for a “Global KPI Dashboard” in Metabase:
The idea was to create a dashboard for France as a “test instance”, starting to implement & track some of the sales & user (enterprise + shopper) metrics listed here in this Analytics Overview Page in Notion: https://www.notion.so/openfoodnetwork/Analytics-b94ec8beccb349e69f939353b23a92e0
collect feedback and then replicate this dashboard for other instances.
That Analytics Overview Page lists the metrics of interest +
respective research questions
user types
tool/data source etc.
that are (planned to be) implemented and shall serve as well as wiki/dictionary for defining important metrics that turned out to be not extracted that straightforward from the database (e.g. “Active Shoppers”).
These metrics include
standard business metrics (sales, # of customers, acquisition, retention…),
user engagement data (e.g. # of active shoppers, purchase frequency)
impact (e.g.seller concentration, number of regenerative ag producers %…)
and other research questions that need to be supported by quantifiable data.
Tracking this data will help us gaining insights about our users (both enterprise & shoppers) and understand the business value and impact of OFN, per instance and eventually on a global base.
Next Steps
Collect Feedback to the current dashboard (who does not have access Metabase yet and would like to have a look at the Dashboard please DM me on Slack)
Replicate Dashboard for other instances
Collect Feedback on the List of KPIs - are important ones missing? Which other research questions are of interest?
Further investigate (& document) database definitions for Metrics Wiki (e.g. unclarities like “Active Shoppers”, Distributors vs. Non-Distributors etc.)
Prioritize KPIs as part of the product development process (as implementation of queries and dashboards requires dev & product ressources)
Create tasks for setting up prioritized KPIs in Metabase & Matomo
From the perspective of a hub/shop manager, these are the pieces of information (stats) which you get from a platform such as Etsy and the things I think they might want to know to help them grow their customer base:
Number of shopper visits, orders and conversion rate : I think these are already in the Notion list
Baskets which have not been checked out (although you need to pay Etsy extra to view this info.
How people land on your shop front (which social media link, website referral, email referral, if searched through the OFN platform or if from a google search etc)
Search terms customers use most frequently on your shop front
Top visited products
Top products which actually sell
What the same customers search for on the platform in other shops
How this information changes with time (ie compared to last year/month)
Love that OFN is working on providing more metrics for their hubs to use (as well as the instance to use). I think this will be a very powerful tool
@lbwright22 's feedback made me think maybe I misunderstood this. So the KPIs are not necessarily at an aggregated instance level only? They could be for a given shop. Is that right?
Could they be for a group of shops? ie - could we enter the names of 20 shops that are in a market togther, and run indicators for them collectively?
One more question - (sorry) - do we have policies (yet) or best practices… about who gets access to metabase? Presumably, we would not ever give users access - but I’m just checking. Access would mean seeing pretty much all the data - right?
and one addition to the list – sales of particular products - where the user can define the products.
No actually you understood correctly @tschumilas: the purpose of the dashboard is to provide data on an aggregated instance level.
It is (currently) not supposed to be on a (group of) shop level or to provide anyalytics as a service to shops.
Not because this would not be of value, but it would require much more ressources to set up & manage which is currently not priority/ feasible.
Thank you still for your feedback @lbwright22 - some of the stats are also interesting on an aggregated level, like top selling products / categories (via Metabase) and traffic split (via Matomo).
so just further on this (very helpful - thanks) - we’d like to begin offering analytics and a kind of annual dashboard to users as part of the service we offer (and likely for an additional fee). In doing so, we should not consider metabase as a source of enterprise-specific metrics for this. Correct? We’ll need to come up with something else to use.
I don’t know about that @tschumilas - some of the things specific users would want could very likely be set up in dashboards for them (and quite easily). What I have no idea about is the access control i.e. whether we can have a user that only can see one Dashboard- the one specifically for them. Does anyone know this?
This is interesting - and definately a way of delivering even better services to our users (and perhaps another revenue stream). Even if a user cannot do this themselves (because they would see other data besides their own), it would still be a nice service that the instance could offer to users.
I think @Matt-Yorkley has studied this. Matt do you remember your findings?
In anycase we should focus first on getting instances used to the tool before getting hubs onboard.
I’m not sure also hubs need a high frequency of refresh. Starting with an extract of Metabase/matomo for some of them might give us more info on what they need?
For now I think Metabase will just be for instance managers / admins, but in the future we could look at options for giving certain enterprise users access to dashboards for their data. It would need additional dev work and looking carefully at access control. It might be quite complex.
There’s another option here as a middle-ground in relation to enterprise users, which is adding some simple in-app charts in admin for a couple of key bits of useful data. I think it wold be incredibly easy to do, using something like this: https://chartkick.com/ and some simple queries which we could optimise and heavily cache for minimal impact. It could probably be done in a few hours of dev time.
So:
complex KPI dashboards that involve big queries on lots of data at instance-level in Metabase
a couple of really simple KPI charts in the app with small queries at enterprise-level
I was actually assuming that mostly, an instance manager would prepare a report for a particular user (likely a larger hub user) as a fee for service. I’m less worried about users having access to set up their own dashboards. I just want to make sure that its possible for the instance to create a report/dashboard that pulls data for a specific enterprise(s).
Your feedback is welcome before Jana moves forward with the current set up.