Proposal - Global Integrations Pool

The Problem

Individual instances are creating a wealth of integrations to solve user problems. This work is being done in response to our slow delivery pipe and . In different countries, people are implementing the same things, duplicating time and effort. A problem might be solved in one place while another place is saying the users’ needs are impossible.

We are starting to realise some of the potential of OFN through integrations - a distributed commons of tools to solve the needs of our sector. However we are not managing our commons as a commons. We reinvent the wheel, wasting precious time and resources. It’s just not very aligned with what we stand for.

The Proposal - A Global Integrations Pool

Using open source tools to create a global server that connects to each OFN instance with template integrations that any instance can use to connect their server to the tools we need.

The range of integrations we already have across the community is huge, including:

  • Transforming reports
  • Integrating with accounting tools
  • Connecting marketing tools.
  • Improved email communications between hubs and shoppers.
  • Customisation of OFN processes
  • Enabling simpler collection of user fees.
  • Managing metrics in a more streamlined way.
  • Project management and notification tools.
  • And more

For the whole community to benefit from the commons we need to be implementing these such that if one instance builds it, every instance can see, understand and use it. THis will involve more than just development work. It will involve creating templates, documentation, simple videos. This means there is scope for technical and non-technical roles as the project grows. It will also create pathways for non-Rails devs to contribute to our global commons, which might be very welcomed by instances approached by technies that want to volunteer. The intention is that we make it possible for non-techie people to set up sophisticated integrations.

The Implementation

To implement this on a global scale we’ll need to meet a few requirements:

  • We need to meet EU GDPR rules
  • We will ideally make use of open source tools as much as possible

N8N

As such, the best option is to use N8N - a sophisticated open source workflow tool. We can host this tool on an EU server, meaning that data flows will follow GDPR.

N8N enables integration of any API with a range of already connected tools. It enables the creation of templates such that people can log in and create the systems they need easily from saved templates. Our own self-hosted version is free, with just the cost of the server, meaning we are paying a fraction of the cost of other workflow tools that are currently being used.

Delivery

Coordination of this role will require technical and product input. Product will be responsible for coordinating priorities, building specifications for development and ensuring proper documentation. Technical responsibility to implement the spec. Already in some instances there are people employed in these technical roles so ideally these people will start moving their implementation focus toward the global integrations pool.

We can create a Github repo that will backup our N8N workspace and use Github to manage our priorities.

Toward a Roadmap

With the recruitment of a Global Metrics person it seems appropriate that the first set of integrations are based around global metrics. Building global metrics should extend the work already done building a metrics database in metabase, using the metrics already developed by Jana as a starting point. This will provide the opportunity to:

  • Connect each instance to the integrations server
  • Build some initial integrations with OFN reports
  • Safely connect OFN databases for any data not available on reports

This will therefore create a baseline of interoperability and examples from which future integrations can more easily be built within N8N.

While this work is in development we can begin prioritisation of the global integrations tasks to follow, that will involve:

  • Understanding the most pressing global issues that can be solved through integrations
  • Moving bulk tasks out of Zapier to save on costs for instances
2 Likes

To make sure I am clear on the proposal, it is to create a central/common library of useful product-adjacent tools and templates that can be used by any instance with minimal or well-documented customization or technical expertise?

Responding to the first paragraph: The integration work is being done because of the nature of the development pipe and because instances must offer (as much as possible) functionality that’s critical to our users’ business success (like accounting, logistics/delivery, or mailchimp integration) or users will simply and regretfully stop using OFN and they will be impossible to get back. I think every instance manager has had a conversation with a hub or market that begins with “we really love you and what you do, but we just absolutely need to be able to xxxx which OFN doesn’t do and yyyy expensive software does it so unfortunately we have to go with that” or “we’ll just have to build our own software”.

Likewise, integrations that support instance management like invoicing users are likewise critical to an instance’s survival: when instance managers can’t efficiently and accurately collect user contributions (from the standpoint of managing enterprises and invoicing), the instance suffers and there is less money to contribute to global.

I realize doing something in “urgent” mode is unsustainable, but urgency is driving the duplication of efforts across the community - there is some integration functionality we need yesterday, and the opportunity cost for not having this functionality can be significant. Unfortunately, this also results in instances with more access to technical expertise/money being more able to build custom integrations (I guess this is kind of a “success to the successful” systems archetype maybe).

Keeping/getting/managing users and getting money seem like basic necessities for keeping OFN and the global community alive.

I am wondering if there is an interim step or effort that could make a few of the more common business-critical integrations that are already in use accessible to all instances now. Possible outcomes of not doing this as a stopgap are continued one-off-per-instance integrations or continued failure of smaller/newer/weaker instances to thrive. One option could be a “pool” of people who have done integrations, who an instance could contract with or otherwise compensate to implement a similar solution? This is not instead of the proposal above; rather the idea would be to help instances “get by” until the integration pool concept is a reality. I am in favor of limiting these interim integrations to functions that directly impact user retention and instance revenue. I think we don’t often lose users over clunky reports - they an important but not life-threatening pain point (I could totally be wrong here!).

1 Like

Thanks for your work and thoughts so far, @lin_d_hop and @lauriewayne1!

I agree to both of you:

  1. Having a standard tool for integrations (N8N) and a process would be fantastic!
  2. A list with all the available tools and integrations that are already used and could be copied or modified by others would be priceless as well! I am thinking of Excel tools, airtables, API(v0) tools, Google Forms and so on…

Probably (1) needs to be in product hands, while (2) could be driven by the Instance Managers circle.

Maybe its worth clarifying…
If a global integration exists anywhere it will be just as fast in most cases to add it to a global integrations pool as it will be to copy and paste that integration to any instance that needs it. I’m strongly of the opinion now that we should collectively focus on adding any existing integration to the global pool, and certainly do that over reimplementing for a single instance that needs it.

What I would find really helpful from instance managers is a list of the integrations that you would like to use that you know exist, ideally with a priority ordering. Then I can work through them one by one adding them to the pool in that order.