OFN as data source for SOLID apps

Tech
1

The Australian team is developing a discovery app for sustainable producers and wholesale buyers. Some of the data, probably most of the initial data, will come from the Open Food Network platform. We would like to use the new DFC (Data Food Consortium) API to fetch the data for this app so that it can be re-used with different data sources and re-used in different contexts.

Today we met with Alex from StartinBlox to discuss the possibility of building on the SOLID framework for best re-usability and standard compliance.

In the simplest scenario, the app would consist of static HTML including Web Modules which use the OFN API as data source. All the data would be within OFN and all the rendering would be within the browser. The OFN DFC API is not a SOLID API but the format is very close. Some tweaks to the API could enable the DFC API to output in a SOLID compatible format. That’s then consumed by the web components which are basically just HTML and Javascript.

If we need to process the data first, maybe transform OFN DFC data to SOLID and mix it with other data like permissions or additional user-supplied profiles then we may need another server which does this for us. StartinBlox uses Django/Python apps for this at the moment.

Also not in scope of this is the authentication. A public directory app can display data that is publicly served on an API. And when you log in, you can see data that is specific to that user account. But we may have a case of enterprises granting permission to be displayed in a certain directory and maybe only with certain products. We don’t have answers to how to do this yet.

2

As far as I understood - and it may be very little. Solid is a framework bathering all approved web standards. In DFC we have been so far following standards. The cases we couldn’t were cases where no consensus happened in the W3C community, yet. So - again as far as I know - cases with no agreed standards currently.

But your sentence suggest otherwise. Can we list what’s different? And in this list, differentiate between what’s currently approved as standard and what currently still up for debate?

And I will invite the DFC team to share some comments as well, it would be good to clarify this :slight_smile:

3

image
image1344Ă—674 99.1 KB

I was just watching a recording of the meeting and Alex was suggesting that this section of the OFN DFC API might need to be renamed “@container” intead of “@graph”, in order to be Solid compliant? (He wasn’t 100% and was going to check in with Benoît, his CTO and get back to us on this)

4

Thank you @rachel for the invite.

First, I would like to say that the notion of storage (POD) is very very important in Solid. A POD is a space controlled by users where they store and manage their data. This space should be decoupled from applications. So with Solid, apps like OFN should not store any users data anymore but data should be hosted in the users POD.

I discussed with some tech people from OFN long time ago about Solid. They were very enthusiastic but the removing of the database system and the migration of data is a really big challenge.

But even if one can’t follow the entire Solid specifications it’s a good idea to follow the most of them as they are based on web standards.

I think that what you are trying to achieve can be done as the current DFC standard is close to some aspects of Solid and because you don’t need authentication. You basically just need a LDP API and to agree on some client-to-client rules [1] like “where data is stored?”. I’ve just opened a discussion about data discovery on the DFC GitHub.

In this discussion, I suggest to investigate the work of the Solid interoperability panel or to use a TypeIndex. In addition, to comply with the Solid specifications, a WebID should be provided for all users to store the data registrations and/or the TypeIndex.

PS: It seems that the suggestion of Alex is mixing things. The JSON-LD document you are mentioning @dthomas is a resource graph and can’t be converted to a container in this way. It’s true that in LDP there must be containers. These containers must lists resources which contain resource graph (dataset).

[1]: In Solid, client rules are defined in a “client-to-client standard”. Indeed, Solid apps must follow the client-to-server Solid specifications + a client-to-client standard. I already started to propose a DFC Solid client-to-client standard. This client-to-client standard tells where data should be hosted on a POD, how and what should be indexed, etc. It is a draft under active development.

2 Likes
5

Hi @lecoqlibre ! :wave: Welcome to OFN Land! Thanks for your clear summary.

For me too the key shift with SOLID is from a single application database with data relating to the app for many users, to many data pods which each contain all data related to a single user. In a SOLID world, your application layer manages (async) data retrieval across multiple pods, stitches it together and displays it as a meaningful dataset to the front end.

As a relevant example - If a hub is selling products from 3 different Producers, that data would be retrieved from 3 separate pods, potentially on 3 different servers, in 3 different storage mediums, and then displayed in the front end… some of that data might not be available, some of it might be instantly retrieved, some might take a few seconds. The App layer needs to manage that, so that the end user can have an enjoyable retail experience! :wink:

My understanding is that the issue isn’t so much that the DFC Standard isn’t SOLID compliant (which it isn’t, but it’s not far away), more that OFN would have to completely re-engineer it’s Application model to fit with SOLID principles. We could move User data to a Pod provider (we could offer Pods), but we’d also need to support other pod providers, if our users chose to host their own for example - that’s where something like the DFC API interface can start to move us in that direction.

2 Likes
6

Thank you all for the explanations. Fitting DFC into SOLID sounds like a great step forward.

On the format of documents. My understanding is that when we list catalog items, for example, then DFC expects a graph starting with the user having an enterprise having catalog items. And in SOLID, the API endpoint is seen as a resource containing a list of catalog items. It’s just about arranging the graph with the same data in a different way. And we could support both with very few lines of code.

On separating data and application. It’s great to have control over your own data in your independent pod. But most web applications have their own existing database. And not every user wants to have their own pod. It’s just another concept to understand and another thing to manage. So, I’m wondering if web applications like OFN can provide the APIs to act as pod and/or as application. Fact is that we are currently doing both and it’s working quite well for us. We just want to be more compatible with other tools to enable more use cases.

7

My understanding is that the issue isn’t so much that the DFC Standard isn’t SOLID compliant (which it isn’t, but it’s not far away), more that OFN would have to completely re-engineer it’s Application model to fit with SOLID principles.

Yes @RaggedStaff, I’m not proposing to make OFN switch to Solid right now :slight_smile: But you’ll see below that the issue is related to Solid.

On the format of documents. My understanding is that when we list catalog items, for example, then DFC expects a graph starting with the user having an enterprise having catalog items. And in SOLID, the API endpoint is seen as a resource containing a list of catalog items. It’s just about arranging the graph with the same data in a different way. And we could support both with very few lines of code.

In fact, there is no standard routes (URL) in DFC so one platform can’t find where data is stored on an another platform. For now, we are using only one URL as the source of data for a platform. This is currently how the prototype knows where to find data. This URL is a RDF resource expressed in JSON-LD containing a “@graph” property like in the 1.8 example.

In term of data discovery this is not working. My point was in favor to use the solution provided by the Solid community for “re-usability and standard compliance” because this solution could work for non-Solid compliant apps too.

How it works with Type indexes (like proposed in the updated DFC Solid standard):

  • Users or platforms have a WebID (URL) referring to a public profile document;
  • The public profile document points to general or DFC specific Type indexes;
  • Type indexes lists DFC standard routes (see this example).

So, back to your data discovery problem, the discovery app would just need a list of WebID (URL) to know where to find data.

On separating data and application. It’s great to have control over your own data in your independent pod. But most web applications have their own existing database. And not every user wants to have their own pod.

Solid’s hope is that one day PODs will be used by everyone, producers, consumers, farms, enterprises and so on.

So, I’m wondering if web applications like OFN can provide the APIs to act as pod and/or as application.

OFN can definitely act as a POD provider by following the Solid protocol. Even is PODs should be decoupled from applications, they can. It’s not a jail anymore because users can migrate their data to another POD provider very easily if they want to.

9

@container is used to define a location where data is stored. There is usually one container for each type of data, but this is not mandatory. It is a specification of LDP.
@graph can contain all kind of data, regardless of whether it’s stored in the same container. It is a specification of Json-LD

Containers require platforms to implement a container-based API structure. DFC requires platforms to provide an API through which it will be possible to retrieve the root or complete data of the connected user. Containers are not required. Only LDP uris for all data are required.

10

Alex from StartinBlox now said that our DFC API is SOLID compatible but we would need the LDP compliance for their StartinBlox modules.

1 Like
11

@maikel I don’t know what exactly have been said but the regular standard of DFC (which OFN is currently using) is not Solid compliant at all, it notably does not support the Solid-OIDC authentication nor the Solid WebID profile (which SiB does not implement either).