This post is to summarise some of the discussions on high level API implementation. What follows is essentially a proposal for the wider delivery team, summarising a lot of content discussed in other places alongside some of the discussions we had in person.
1. Documentation
Documentation will be automatically generated by RSwag gem. You can find an example of what this will look like here. Documentation takes info in specs and turns it into human readable Swagger docs. This documentation will be automatically updated when the code is updated, which is obviously excellent. There was some discussion that we might want to add a little additional detail to aid useability, like more descriptive details on attributes. We might also want to add a wiki somewhere that outlines the different ways filtering can be applied.
2. Pagination Strategy
We propose that pagination is implemented on every v1 endpoint. We discussed a few different options and realised that perhaps we just want to start conservative and increase numbers when we have more data about API use. Our proposed starting points are:
Set the max number of rows returned per request at 200
Set the default number of rows returned (when no number of rows is specified in the request) to 50. We recognise is low but setting the default to a low number seems sensible.
3. Authentication Strategy
Essentially weâre all in agreement that the authentication strategy remains unchanged between v0 and v1. This means:
Basic auth via login when API is accessed via browser
API Key as param
API Key in header
4. Implementation
The technical framework has been decided through conversations with techie folks in Slack and Github. We propose that new API controllers should be implemented with full CRUD functions from the start. @Matt-Yorkley suggests that there is no point implementing only specific actions on API controllers, that we might as well define and scope all actions upon creation of each API controller.
5. Bug Severity
We discussed how we assign API endpoint severity, and essentially propose that the same fundamental severity ranking will apply as now in v1. If endpoints are suddenly down these will be s2 at least. For performance issues we agree performance is a feature, as always. In designing endpoints the proposal is that we keep calculated fields (eg customer balance) to an absolute minimum and never show field like this on high level index. Endpoints should be nested. Attributes that require heavy calculations might be hidden by default and might exist only on :show. Datadog already monitors endpoints currently, it gives detailed information so we should be able to stay ahead or at least have good visibility on performance. This is important as for the foreseeable future the API will be served over production servers so performance issues could impact all users.
6. V0 Maintenance
The proposal is that v0 endpoints are not maintained going forward. Obviously v0 endpoints that are used by the app will need to be maintained specifically for app use as the app will continue to use v0 endpoints until we have fully migrated to ReactiveRails. To be clear, We wonât transition the app to use v1 endpoints at all. V1 of the API is an entirely separate product to the app. That being said, v1 endpoints might be used in modularisation processes eg /shops page might be turned into an independent and modular app when we work on Discovery, and this app would use v1 (or v2 or vwhatever). The same would be true of modular Shopfronts implemented via the API.
We propose that officially v0 endpoints are now scheduled for deletion and deprecated. This means that by default v0 bugs are not fixed (unless directly needed by the app). Enhancements on v0 not prioritised and rejected by default. V0 documentation wonât be maintained. We understand that a number of instances are using v0 but we propose that if changes are needed any instance using the API should support the development of v1 endpoints if they require changes (via the funded features process). The idea is that once a v1 endpoint is created, we declare an End of Life date for the v0 version (for any endpoint not used by the app we declare an EOL date). In this way the intention is to encourage the community onto v1.
Shall we spec the performance? Since we have pagination for every endpoint and a maximum page size, we could put a hard limit on that, for example 200 records have to be served within 4 seconds. That would mean that the default 50 records are usually served in less than a second.
The performance depends on the server specs and the production database and therefore it would be best to measure performance in production. We can do that either via Datadog or add a little bit of code to measure the execution time and report to Bugsnag if a request takes too long.
We could then say something like
4 seconds is fine,
8 seconds is an S3 level bug,
16 seconds is an S2 level bug,
30 seconds is an S1 because we would like to limit all requests to 30 seconds.
Since Datadog APM seems to be already plugged in, it should be great to measure performance? (rather than adding more code to instrument API performance?).
4 seconds seems very long to me for an API call but it depends on the applications using the API. Would you have some use cases that are going to be using the API?
Hi Adrian,
Loads of work has been done getting to this point on our API discussions over the past year. On this thread you will find a lot of our journey, including notes on the adventure from React to ReactiveRails (which was key to the API discussions). Toward the end youâll find this post that summarises some explorations of use cases and some loose prioritisation.
On prioritised use cases:
Weâll implement much as funded features (more on âfunded featuresâ here, here and here)
The DFC, and in particular the v1 products endpoint, is amongst the first funded API work as the funding for this work has been confirmed. More here on auth but the next stage is products interoperability.
Just to clarify my question: if the API is used only for backoffice tasks to sync data between platforms, or maybe as the background loading step of a frontend, 4s might be ok. But if itâs used in some interactive tasks (in new frontends), I fear the UX might be poor.
Use bearer token authentication from the beginning (it makes it a bit clearer how you handle permissions). You could then put the apiUserId, tenantId (entrepriseId) and scopes inside the token. As scopes, I thought of product:read, product:write, catalog:read, catalog write, entreprise:read, entreprise:write, ordercycle:read, ordercycle:write (inspired by https://guide.openfoodnetwork.org/basic-features/enterprise-profile/enterprise-to-enterprise-permissions-e2es and github scopes)
Use open api 3 for defining the api schema and model the api (you could have the file on a specific github repo so contributors can make suggestions)
Put something like Redoc on top of this file to make it nice to read
I wouldnât use jsonapi : itâs not as âpureâ as json (adds clutter to you json), you will be tempted to hook it to your database and your API will look like your database (changing from postgresql to mongodb shouldnât affect your API). If you want the nice stuff that jsonapi brings, just go for graphQL âŚ
json schema rocks
make the json as simple as possible. API responses are used directly in automation tools (Zapier, n8n), templates (liquid templates). You donât really want to rebuild your objects âŚ
think i18n from the beginning (I work for OFN Belgium and we need the possibility to add a product description in french, dutch and german).
The notion of events !! The ability to subscribe to events such as OrderPlaced, ⌠(through webhooks)
GraphQL is super good for reads but a bit more complex for writes. Maybe an hybrid API might do the trick âŚ
Endpoints are a good start to know how to compartmentalize the API : what would OFN look like if it was build with microservices (doesnât mean you need to go this way but itâs good to identify the different contexts).
You could apply vertical design on this : like split database, separate controllers into folders, separate API endpoints (donât forget to group them in a main API for better developer experience) , build independent microservices (or use some from outside OFN)
Heyall: OFN n00b here, in process of evaluating the system for potential adoption by our local food co-op (in Portugal, as it happens ), with a little systems integration challenge i am trying to scope out.
Specifically: weâve got a central warehouse using Moloni SaaS for Stock Management and Invoicing (latter function being integrated with PTâs tax authority, is why weâre kinda wedded to this), which- tho it has some Zapier integrations with other SaaS platforms- offers no integration with OFN software as it stands.
So i am wondering: Is there any API endpoint currently available for such integration? Iâm looking at this âproducts_and_inventoryâ report (via CSV download), and thinking this could be all the data we need for OFN<->Moloni interop, but am not clear what methods/ endpoints (if any) are available for production use at this time. ?
hi @walt - all our current endpoints are considered ânot releasedâ but we are aiming to deliver a support release of products / inventory endpoint in the next few months. In the meantime Build, Collaborate & Integrate APIs | SwaggerHub is one that you could use, but please be aware it is not supported and will change
Thanks @Kirsten -good to know. Iâll be watching this forum for news of a stable API release, and meanwhile will use the prototype to explore possibilities.
One Q: in the readme, it says âTalk to us to get your credentials set up.â Who should i ask for such credentials grant (as a tester) and how -can you say?
: