Interesting!!!
Metabase
I see this as a BI challenge, that’s why I can only think of Metabase, we really need to leverage it’s power now that we have the setup basically done. This is something inevitable, we just need to decide how much we invest in it vs investing in a custom reports solution like the one we have.
The main decision is: what users will be using metabase: instance managers and admins will obviously be using it. Will enterprise managers have access to metabase?
One of the major challenges with reporting (and metabase is not different) is data level authorization (show users only the data they have permissions to see). This is already for us the main challenge for performance optimization in the existing reports (we did some good progress recently).
In metabase, we can embed this authorization layer in the SQL (like we do in the current reports) BUT I have seen many different approaches to this problem.
This is just an example but I can think of one case where it was done in the ETL/data streaming layer with incredible success and simplicity. Basically, instead of having one DB from where metabase reads, we can create enterprise specific DBs and make kafka write live data to these databases according to what enterprise the data refers to (same data can be written to different DBs if that’s something needed). This can sound a bit hard to do initially but the power and flexibility we get from it is amazing: with these enterprise specific DBs we would have no data authorization to handle and we could use metabase to build whatever reports/dashboards the users need. We would need to think about a way to simplify the authorization models so that this is feasible. We will need to do this simplification anyway otherwise reports will always be an authorization nightmare (that class we have OrderCyclePermissions is just nuts).
Well, this idea is not yet very mature in my head but I think it definitely deserves to be considered. I dont think these enterprise specific BI DBs is the only way to go, we can find alternative approaches.
I have seen a lot of reporting and BI tools in action and I know how powerful for the business it can be to have something like metabase available to the managers.
Custom reports
I am strongly tending towards solving this problem with metabase/BI and not with more troublesome ruby code that renders specific reports and always needs a dev to fix problems or improve things. But if we decide metabase is only for admins, we need such a solution.
I am not sure we need/should rely on the API for these reports, the datasets needed for these reports will be specific and large, I am not sure what’s the advantage of having reports as part of the API.
I am not sure but I think it’s better to separate reporting and API requirements and solutions. The reports can be exported to json but I see reports as particular representations of aggregated data while the API has a standard representation of basic entities (products, orders, enterprises, etc). In that respect, external systems do not need reports to integrate with OFN, they need good API endpoints with all the data they require.
@Matt-Yorkley I am not sure about the gems approach but I think that’s because I dont understand what the gems could do for us. I think the gems will differ considerably in approach? Maybe a spike would clarify that