Road map for OFN's API?

Tech
1

Just thinking about the structure of our API, with a view to getting everyone on the same page as we make changes to it in future.

Connected to:

Where we are now

For the uninitiated, the OFN has a few different “APIs”:

1. Spree’s existing API

  • Traditional generic RESTful API
  • Quite well structured
  • Authentication using API keys
  • Designed for external use
  • Hasn’t been extended much with OFN functionality
  • Views rendered using RABL :thumbsdown:

2. The OFN’s back-end API

  • Designed for internal use
  • Authenticates using Rails session
  • Mostly RESTful
  • Very messy at the controller level - API logic and actions are mixed up with non-API logic and actions in Admin controllers
  • Quite a few good conventions/patterns
  • Used on most Admin pages, broad scope
  • Views rendered using ActiveModelSerializers

3. The OFN’s front-end “API”

  • Not an API
  • Designed for internal use
  • Authentication using Rails session
  • Focussed on supporting specific features
  • Mainly used on the shop page, very limited in scope
  • Not particularly RESTful
  • As a side note, a lot of the data in the OFN front-end is rendered using ActiveModelSerializers, but rather than being fetched via AJAX it tends to be rendered directly into the DOM as AngularJS services on page load.

So what?

I suppose the question is - where do we want to get to. Do we need a road map?

If we’re starting to think about external interfaces more seriously, so we extend Spree’s existing API, or do we start fresh?

Should we be separating out all of our JSON/AJAX focussed API actions and logic into separate controller hierarchy?

Do we drop the idea of “internal use only” and instead re-build everything in a single coherent API that works for both? If so, how do we manage authentication while keeping the API nice and DRY?

2 Likes
2

super interested in the evolution of this.

Here are some decisions that I have made for a WooCommerce solution:

If we’re starting to think about external interfaces more seriously, so we extend Spree’s existing API, or do we start fresh?

reusing the API. There are lots of things that the current api already handles, security, roles internal queries and integrity etc.

Do we drop the idea of “internal use only” and instead re-build everything in a single coherent API that works for both? If so, how do we manage authentication while keeping the API nice and DRY?

not necessarily all, some things may stay internal thus keeping good performance and less complexity.
I try to differentiate between internal APIs and UI redering from REST APIs meant for interoperablity.

3

In order to close all of the github issues that refer to this future feature I’m going to list them here:

4

I was reading up everything I found related to the API work. I think we can use this thread to discuss the evolution of the API.
I have compiled a list of threads that mention and REQUIRE the evolution of the OFN API:

And the API can evolve in parallel with the breaking down of OFN into domains. Actually, I think the OFN API must be organised in Business Domains.

5

Hi Rob, interesting!

— Do we need a road map?

I think we first need to define what APIs do we need (public/internal, spree/ofn, etc) and how to build them (tech decisions). Then, the growth of the API should not a task in itself but be a part of normal development of other features.

— we extend Spree’s existing API, or do we start fresh?

Is it not simple to “start fresh” with our API but using adapters/redirects to the spree API?

— Should we be separating out all of our JSON/AJAX focussed API actions and logic into separate controller hierarchy?

I think this is a must. We cannot have app/views logic mixed with API logic…
I think you are saying that, for example, controllers/ShopController should be in controllers/api?
I wonder why is stuff in views/api not in controllers/api. Can we set a rule that every view or controller that is rendering JSON goes into controller/api?
Also, I am not sure how app/serializers/api are used. Are these only used in AMS injection or are they part of some API?

— Do we drop the idea of “internal use only” and instead re-build everything in a single coherent API that works for both? If so, how do we manage authentication while keeping the API nice and DRY?

I think we should drop the notion of “internal use only”.

Next Steps…
I think we first need a map of what’s currently in there, can we go into more detail about the 3 APIs you list above?

  • Can you point to the code where those 3 APIs are implemented?
  • Can you map them into these folders with RABL files in the OFN code base?
    • ./app/views/api
    • ./app/views/open_food_network
    • ./app/views/json
    • ./app/views/admin/json
    • ./app/views/spree/api (spree API overrides)
  • Can you understand and explain each of these folders and what are the good/new ones and what are the old/bad ones?
  • From what I understand there are these 5 places with rabl files plus all the controllers that also take and produce json, plus AMS injectors…

If someone can guide me in this initial understanding (answering my questions above), I can then create that map. Thanks!

6

@oeoeaio @maikel @Matt-Yorkley @enricostn @Hugs @andy maybe you have some ideas to share on the topic as well?

7

@Kirsten you might have ideas on this as well…

8

Possibly, but I don’t have my head fully around how that would work in practice. I’m sure it’s workable.

Yeah, mostly, but I think the #show action is still dealing with html, so we need to decide which parts to move across.

There are no controllers in views/api these are just templates used by API controllers to render json. That seems like the logical place to put them to my Rails brain.

They are used in much the same way as the templates. They are used to render json via APIs, but they can also be used for injection of data into the DOM. They are kind of like bloated serialisers that also perform the function of a decorator.

Ok, what I am thinking of as “Spree’s API” is almost entirely contained within the spree gem, but we are making some changes within our own app/controllers/spree/api.

What I am calling the OFN’s “Front-end API” is everything within app/controllers/api, but it should probably include a bunch on stuff that is currently sitting in app/controllers that is behaving like an API. Any action that responds to json should probably be moved over to app/controllers/api.

What I am calling OFN’s “Back-end API” is everything in app/controllers/admin/api but also HEAPS of stuff that currently lives in app/controllers/admin. In the same way as the front-end, we should probably move every action that responds to json into app/controllers/admin/api.

Kind of. The vast majority of the OFN’s API uses AMS instead of RABL to render json. There is a bit of a hangover from a short period where we started using RABL before we decided on AMS. I’m not 100% sure but I’d be surprised if any of the templates in app/views/api or app/views/admin/json are actually used any more, I didn’t even know app/views/open_food_network existed so I have no idea about that. I think quite a lot of the templates in app/views/json are probably still used by controllers in both app/controllers and app/controllers/api but I can’t be sure. As you’ve noted, everything in app/views/spree/api is likely to be an override of spree code.

All bad as far as I am concerned, apart from the spree overrides which we don’t have much choice about. I try to replace RABL templates with AMS whenever I encounter them.

I just want to be clear that at the moment AMS is our preferred tool for producing json from an object. I’m sure there are better tools but at the moment it’s the best we’ve got. We do use AMS for injection, but personally I try to avoid that practice wherever possible, and I do not consider that its primary purpose.

9

Thank you Rob for all this wisdom. I agree that it’s really bad that our API is spread out in so many places. And I also agree that dedicated controllers in api directories make a lot of sense. I find normal controllers that render HTML and JSON difficult to understand, especially when we attach different logic to the rendering, because the actions are used in different contexts. Separating the API from our normal controllers also allows us to keep it stable for others to use. For that purpose we could think of introducing version numbers, but it’s probably too early for that. If others want to use the API and we use it within the OFN views, we will need a development version that is changing with our requirements and a stable version for others that doesn’t change with every pull request. So offering an API to the public will come with additional development costs as we need to maintain at least to versions (or tell clients that they have to update their code four times a year).

10

I am finally coming back to this post!!!

We currently have 2 APIs:

  • spree api - from spree_api and overrides under controllers/spree/api and views/spree/api
  • ofn api - under controllers/api and views/api

Additionally we have json end points in /admin but I wouldn’t consider that an API as it is mixed with the html rendering code.

We will soon be removing the dependency to spree_api! See issue here.
That means that controllers/spree/api will become self sufficient (independent from spree) and can be (sooner or later) be moved/merged with the ofn api under /controllers/api - this is where all the API controllers will be.

AMS is the serialization solution that means that there should be no rabl files in OFN. Additionally, there should be no API code under /views because AMS serializers will be under /app/serializers.

So, we need to convert all rabl files to AMS serializers. For example, files under: app/views/json and app/views/admin/json. I created a new issue in GH for this.

Now we will remove dependency to spree_api and convert all remaining rabl files to AMS.
Tasks for the future are:

  • merge controller/spree/api into controller/api
  • extract json endpoints from /admin into /api
11

meanwhile I also merged controller/spree/api into controller/api

So, the only missing task is really to “extract json endpoints from /admin into /api”, this will happen as we and as we need.
With this clean up, the roadmap of the OFN API is much clearer.

I am archiving this thread. the api docs will describe what’s needed from this thread.