Bye bye Spree :-) (or Spree v2.1 and beyond)

luisramos0 · March 21, 2019, 7:58pm

From this thread we concluded that:

we are going live with spree 2.0.4 - this will be OFN 2.0.0
we will probably step directly to 2.1.0 (and may or may not ignore 2.0.5-2.0.12). Both 2.1.0 and 2.0.5 are from Sept 2013, that means that 2.0.5-2.0.12 are mostly backports from 2.1.x versions.

In this new thread I’d like to summarise the investigation I did around the next versions of spree, asking the 2 following questions: what does spree 2.1 and beyond bring? how difficult it will be to upgrade?

Spree Versions after 2.0.4

This is a map of the spree versions:

Analysis

We can see that 2.1.0 brings Rails 4, which is really the main point about upgrading Spree.
After 2.1.0 the upgrades will not bring as much value as v2.1 and they may still be tricky because of our “decorations” on top of Spree.
So, it looks like we should aim for spree 2.1.0 as part of this current effort and after that, we will probably focus on other areas for a while…

From the recent investigations we did (with @sauloperez), issue #3565, it looks like moving from 2.0.4 to 2.1.0 is mostly about adapting OFN code to Rails 4. I have no experience with Rails 4, so I can’t really evaluate how much time this could take…

Questions

1 - how much effort will it be to move OFN code to Rails 4?
2 - how important is it to get to rails 4.1 and beyond?
3 - is there anything I am missing about spree versions after 2.1 that would justify the upgrade?
4 - have we considered stop following Spree and just go our own way, for example, upgrading OFN to Rails 5/6 without upgrading Spree?

Opinion

re question 4, imo, that is the long game for OFN. why spend so much money getting to Spree 2.2 and beyond?
Because we have so much logic in OFN, the advantages of using a third party (with supposedly quick upgrades) is lost, so, going our own way seems to be the better approach, otherwise we will be on the “painful spree upgrade project” forever… and in 2022 we will switch to the “painful solidus upgrade project”… and we will keep on building “decorations”…
There are quite a few things about this question but imo the main one is the quality of Spree, Spree is a nice project but it’s not awesome, and given the current state of OFN, it would really be a lot easier to gradually move away from Spree and grow our own solution: undecorated, service oriented and broken in domains.

maikel · March 21, 2019, 9:03pm

1 - how much effort will it be to move OFN code to Rails 4?

We need a bit of time for the upgrade but most of it is simple. Rails mentions all important changes. It’s a 12 step process: Upgrading Ruby on Rails — Ruby on Rails Guides

2 - how important is it to get to rails 4.1 and beyond?

Important. As long as we keep Rails as dependency we need to stay up-to-date. Maybe it’s not urgent but the CSRF protection sound nice.

3 - is there anything I am missing about spree versions after 2.1 that would justify the upgrade?

Unblocking the nokogiri update would be great because of the known vulnerabilities.

4 - have we considered stop following Spree and just go our own way, for example, upgrading OFN to Rails 5/6 without upgrading Spree?

In my opinion that should be our long-term goal. Spree helped in the beginning to set everything up quickly but now it’s a pain. We have so much code in decorators that it makes more sense to have our own classes. My suggestion to de-deface views was the first step. My idea looks like this:

Replace all Spree views.
Replace all Spree controllers.
Replace all Spree models.

We have to do the models last because they are used by everything else. But otherwise there is no reason why we can’t have our own models working with the current database, right? And with our current code quality process we would be in a much better place than with Spree code.

luisramos0 · March 21, 2019, 9:40pm

Awesome Maikel, we agree

sauloperez · March 25, 2019, 11:37am

Thank you for raising the topic Luis. I won’t add much on why we need to upgrade now. The reasons are very well known: even Rails 4 is long dead. I’m also very happy to bring in my experience here too.

I’m particularly interested on dicussing about the Spree dependency. I also had a few table-flip moments in which I concluded to give up on Spree but after talking to Enrico and understand he’s point (back when that happened) that we would be missing things Spree community already went through missing the point of open source.

So, I think is something we need to mature and give time to. There are many implications to each approach and possibly following the decoupling process Maikel suggests might lead us to a comfortable middle ground where the dependency on Spree is not that costly.

Let’s see if we can chat about it in the gathering.

luisramos0 · March 25, 2019, 11:55am

yeah, we will talk about this in the gathering.

I just realized how much importance I give to this little detail mentioned by Maikel above: “And with our current code quality process we would be in a much better place than with Spree code.”
I’ve seen many dev teams in my life and imo we are doing an amazing job as a dev team right now. So as a group we can have an opinion about Spree: we can do better.

luisramos0 · March 27, 2019, 12:02pm

We have started the release notes for ofn v2 and it’s incredible the amount of work that had to go into delivering… nothing!

I’d like to share a few things:

Spree features are not useful for OFN, all the complicated stock management stuff in spree v2 is a good example. And looking at the following Spree and Solidus versions I see nothing major that is useful for OFN in terms of features
in Spree v2+v3 the code style isn’t going to change, and I cant forget this code from spree v3.7. Is this the type of code that we want for our long term future? (because if we stay with Spree, spree 3.7 will take years to reach). If you think Solidus is the amazing solution to this, have a look here.
in v1 we are overriding 37 spree models (~45% of the total). Spree v2 brought 28 new models…
in v1 and v2 we are overriding 26 spree controllers (~60% of the total)
I’ve investigated a bit the solidus versions and it’s not very different from the Spree story in relation to OFN, we would need to keep all the OFN decoration…

I have spent some time investigating the practicality of leaving Spree completely, copying over the code to OFN codebase and remove all overrides (this idea of ofn without decorators will enlighten the day of every single ofn developer!!!). It will take a little bit to make OFN work with the copied spree code but should be straight forward to implement.

So, long term, seems obvious that OFN needs to move away from Spree. The question is when.

And looking at this now, it feels like spree 2.1 is a very good candidate to be the last spree version OFN uses.
There are two reasons for this:

easy v2.0-v2.1 upgrade: from what Ive seen, the upgrade from v2.0 to v2.1 will be straight forward and brings spree code to rails 4 (the only painful part will be upgrading ofn code to rails 4, but that’s not something we can avoid)
painful upgrade v2.1-v2.2 - spree v2.2 changes adjustments (again!) which will be very painful for us to adapt for sure…

luisramos0 · April 1, 2019, 3:49pm

fyi, I have gone into exploring the possibility of separating OFN from spree, I have created this issue with two PRs inside, see issue.

I think the follow up tasks would be to remove the dependency from (in a suggested order considering effort/benefit):

auth_devise - routes and controllers (imo spree_auth_devise is a mess, we need to bring it to OFN and make it simple)
auth_devise - views (I have no idea how much we are using from here…)
backend views (not much is left here, mostly configuration pages)
backend assets
backend controllers
api views (resources representations)
api controllers
models…

It would be great to have your feedback!!!

sauloperez · April 3, 2019, 4:47pm

I have to give some deep thought to it but you convince me with the effort that has gone to nothing. And this leads me to money. Tons of money we don’t really “see” anywhere.

From my personal perspective, I don’t want to waste my life doing this either. Since June 2017 when I joined I’ve done little else than the upgrade and that is frustrating.

Another topic that convinces me is how much simpler it would be to develop OFN therefore quicker to improve/iterate and less buggy. Although I see solutuons for the overides of models and controllers I don’t see the way out of mixing two db schemas and the mess it brings.

However I want to highlight the fact that this as well as 2.1 are needed upgrades. I don’t want product people to get the wrong impression. Our state back in 2017 was too bad and while we acquired a ton of knowledge from Spree along the way, it might be the time to split up.

Definitely a topic to carefully evaluate in France.

lin_d_hop · April 4, 2019, 7:09am

I really feel for your frustrations @sauloperez and all the devs. Everyone in the community is frustrated by this! The biggest imaginable loss right now would be if the devs that worked on Spree Upgrade left the project with all the knowledge you attained, meaning that we’d not even have progressed the tech debt in real terms. The solution we come up with should prioritise simplicity of development & dev onboarding into the future.

Looking forward to exploring this with everyone. Much love to everyone and your mammoth commitment!

sauloperez · April 4, 2019, 8:09am

Thanks @lin_d_hop!

One of the counter arguments I have in mind taken from @enricostn words is that it feels like we give up on using open source as a common, leveraging a project like Spree.

The more I think about it the more I see it’s not the case. If we went that way, we would still benefit from the wealth of OSS under Spree and what’s more, we could release bits and pieces of any Spree-derived code as well as ours as open source libs for everyone to enjoy. So I kind of see it now as us enriching the ecosystem rather than giving up on it.

Sorry for the philosophy but writing helpd on making up my mind.

luisramos0 · April 4, 2019, 9:05am

yeah, thanks @lin_d_hop

@sauloperez makes sense, the oss point is a valid point, but we have to be pragmatic, is it helping us or is it slowing us down…

sauloperez · April 4, 2019, 9:23am

Totally agree. It is slowing us down but Spree, not the rest of OSS

tschumilas · April 5, 2019, 3:15pm

I cannot weigh in on the technical aspects of this dilemna of course. BUT - I want to stress that NOTHING has been lost in the spree upgrade process! Even if there is nothing different from a features perspective, or a ux perspective… WE as a diverse collective team (of support people, testers, administrators, funders, food system activists, software developers, designers…) are different. We are more clear in our focus. We have more skills and knowledge. We are able to reflect on different logics underlying the platform and its alternatives. We are more cohesive in our conversations. We are better at seeing things from diverse perspectives… In so many ways, we are a stronger community. And, perhaps ironically, our commitment to the spree upgrade process (and the opportunity cost of this - ie: things we didn’t do instead) is what has shaped us. So - Thank you spree upgrade! Not for one second should we regret the path we took because it made us who we are today. Of course, we now disucss the future path. And we are in a stronger place to have that discussion than we were a year ago. I can’t imagine the frustration the dev team must feel at times. I can only say that I stand in community and in solidarity with you. As the lead person trying to proliferate an instance - I can say that I feel the cohesion and support of the dev team every single day - and TONS of specific things and features are WAY better than they were 2 years ago. So don’t imagine for a second that nothing is different. Everything is different from my perspective.

luisramos0 · April 5, 2019, 9:12pm

Great to read your message @tschumilas thanks for your support!
It’s a great way to start the conversation about the future.

kristinalim · April 9, 2019, 10:03pm

My preference is also to remove Spree as dependency.

I agree with @maikel that it would be simple.

We might have to spend some time figuring out incompatible gem versions and gem version quirks, but I suspect a lot of these issues will be common with Spree so we can use the later Spree versions as reference.

I browsed through the release notes (this is not complete).

There are changes we do not need that look like they would take time to investigate and adapt for OFN (e.g. refactoring of adjustments, return authorizations).

But there are also features for the API and performance improvements which we might actually want. There are many random changes too from which I think OFN could benefit, even if they are not in the OFN wishlist.

I think it’s a shame if the team went through all the trouble of changing OFN to be compatible with the rewrites in Spree 2.0, but then not take advantage of the random improvements in later minor versions with which OFN code is already compatible.

Question: What do you think about us spending time going through the rest of Spree 2.x PRs, and picking which ones are nice to have but at the same time sensible to bring into the OFN fork of Spree? Again, even if these are not in the OFN queue. We get features and bug fixes in if they are easy to adapt, but we ignore the ones we don’t need which look like trouble. This way, we can get further value from upgrading to 2.0.

Currently, that’s my vote.

kristinalim · April 9, 2019, 10:36pm

About copying so much of Spree into the OFN code, we also need to look into the Spree license: https://github.com/spree/spree/blob/2-1-stable/license.md I don’t know how to do it, but I’m thinking we would need to get a thumbs-up from Spree, consult a lawyer, and update attribution and/or copyright.

Small example of a gray area (for me at least) is that, in this PR, we copy the spinner image from Spree into OFN. I doubt it’s technically okay, especially when we no longer use Spree.

Does anyone know this topic better?

maikel · April 10, 2019, 1:41am

This is great. A year ago we would have had no idea how much work it would be to kick out Spree. Now we could attempt an estimate because we learnt a lot about our dependencies.

@luisramos0 Your approach of kicking out the Spree frontend first is awesome. I always thought of Spree as one whole dependency but we can actually choose. Most of the frontend got replaced long ago. We don’t need the Spree frontend. And removing it is independent from using the rest of Spree. We don’t have to do the big exit decision now. We can do one step, observe and evaluate.

I still think that there is a lot of experience and wisdom in the modelling of Spree. Maybe we are not happy with the implementation but there are some interesting concepts. Maybe we will hang on to parts of Spree core for years to come. For example, if we implement The Network better and remove variant overrides, we could be happy using some original Spree objects like products and variants. But that seems far away at the moment.

As long as we use Spree code and we need to upgrade Ruby and Rails, it’s good to see how they dealt with these upgrades because we probably need to do something similar.

@kristinalim’s idea of cherry-picking is interesting. I don’t think it’s ideal because once we started ignoring some commits it will limit our selection of other patches. There will probably be a lot of dependencies within the code. But once we decided that we won’t go beyond a certain point, we can still look beyond that point to cherry-pick. I see that as a valid strategy later on when we decided where to quit.

Licensing is also interesting. As the license says, we need to keep the copyright notice in all Spree code. But we should be fine then. There are no other conditions for code. We should probably avoid importing binaries like images. In case of the spinner, it may well be that Spree doesn’t own it either. I wouldn’t worry about that one.

luisramos0 · April 10, 2019, 1:41pm

Great discussion!

----- there is a lot of experience and wisdom in the modelling of Spree

yes, I agree but I think we have learnt what had to be learnt (and copied what had to be copied) and now we can do our thing. The spree order class is 600 lines and our decorator is 400 lines, 1000 (very smart) lines of code!!! This is horrible design at the core of OFN: we need a long term solution.

I think the only way out is to copy over everything we need from order.rb and refactor to what we need. So, long term (2/3 years time), I think we should move out of spree completely.

Short term (this year and next year), I agree with investing time in pushing the upgrades forward. Re cherry-picking, it’s a really good option to have in mind, lets see!

Meanwhile in the trenches… the build is green!

link to PR

sauloperez · April 11, 2019, 11:14am

This is getting really interesting. The topic of licensing is certaintly one we cannot miss @kristinalim. Perhaps asking them is the best we can do now.

I just want to add that separating from Spree does not mean we stop benefiting from their knowledge. We can still read how they solved the problems we face or the patches they did, but this way we get to choose. That’s the big difference.

luisramos0 · May 14, 2019, 6:25pm

In the meeting in the gathering we agreed to:

remove dependency to spree frontend, spree auth devise, backend and api and leave spree core for later
upgrade to spree v2.1
see if the upgrade to v2.2 is easy or not (what was the improvement in terms of adjustments?)
remove dependency spree_core

So, our last Spree version could be one of the versions between 2.1 and 2.4

ping @kristinalim, we just agreed on what’s discussed above. Nothing new.