Spree upgrade purpose and strategy

[RohanM]

Introduction

We’re currently using version 1.3.6 of Spree, which was released about 2.5 years ago. Since that release, substantial work has gone into Spree, and a number of major releases have brought it up to version 3.0. However, the upgrade process from version 1.3 to 2.0 (the next step for OFN) has proven challenging and very difficult to scope or estimate, which has prevented us from completing the upgrade.

Recently, shortcomings in the Spree 1.3 tax calculations have hindered development, particularly when calculating the tax to charge when customer discounts are applied. Many of the underlying problems that we’re encountering have already been addressed in Spree 2.2. The friction we’re experiencing gives us a greater incentive to upgrade, to save spending our time and budget reimplementing the improvements that are already available in later versions of Spree.

Additionally, we’ve come up with a new strategy for an incremental upgrade that would make it possible to work at the upgrade in smaller portions and integrate those changes more frequently into the code base. This would make it much easier to estimate the scope of work required and to make progress on the upgrade.

Due to these developments, we’re considering performing the upgrade from Spree 1.3 to version 2.2.

The Problem

The following is one specific issue we face that is illustrative of a broader set of problems posed by the limitations of Spree 1.3’s taxation system:

We’d like for some customers to be members of hubs, and for those members to receive a discount on their orders. For example, say that a hub member Susan receives a 25% discount, and lives in Australia where there is a 10% GST on non-essential food items, which is included in their price. She buys some biscuits for $44. With her discount, we would charge her $33 for the biscuits, which is inclusive of $3 GST.

However, Spree 1.3 doesn’t allow us to perform these calculations correctly. Under our current system, Susan’s GST would be calculated against her un-discounted order, and we would report the GST on her invoice as $4 instead of $3.

Spree 2.2 breaks the discount and tax calculations down into two steps, and will correctly calculate the discount and tax as described above.

Ryan Bigg has written a detailed description of this problem (and the broader issues in Spree) on his blog. See the “Promotions and Tax” section of http://ryanbigg.com/2013/09/order-adjustments/ for further details.

Method

1. We have a fork of spree (openfoodfoundation/spree branch 1-3-stable). Extract as much of those changes out into OFN.
2. Fetch upstream and merge:

   git fetch upstream
   git checkout 2-0-stable
   git merge upstream/2-0-stable
   

3. Find 100 commits forward:

   gl upstream/1-3-stable..2-0-stable |wc -l
   # => n
   git show 2-0-stable~n
   git show 2-0-stable~n-100
   

4. Create a branch based on this commit

   gco abcdef1
   gco -b spree-upgrade-step1
   git cherry-pick fork..commits
   gp
   

5. Point OFN to this branch

   # Start a branch based on origin/spree-upgrade
   #  Gemfile:
   gem 'spree', github: 'openfoodfoundation/spree', branch: 'spree-upgrade-step1'
   gem 'spree_auth_devise', github: 'openfoodfoundation/spree_auth_devise', branch: 'spree-upgrade-intermediate'
   # bundle
   # - reports dependency issues
   # bundle update bugsnag httparty
   

6. Run specs, fix problems.

Results

Gem version compatibility

When I first tried the (incrementally) new version of Spree, it wouldn’t bundle because the Spree version was 2.0.0.beta and all our gems depended on 1.3.x. I solved this by making Spree pretend to be version 1.3.99 instead of 2.0.0.beta.

Missing features present in 1-3-stable

It turns out that some features that we depend upon that are present in 1-3-stable are not present at the merge base of 2-0-stable. This meant that suddenly our “upgraded” version of Spree no longer contained features that we were using.

I worked around this by finding where the feature was implemented in 2-0-stable and making a new upgrade branch based on that commit. Here’s what I found.

1. 432d129 introduces OrderPopulator (6% through upgrade commits)
2. 0c59492 introduces the Classification model (21%)
3. e1738cf introduces BaseMailer (55%)
4. 5bfa7fb introduces Variant#track_inventory, which replaces Variant#on_demand (90%)

For my first exploration, I started a branch based on #3, attempting the first 55% of the upgrade in one go. This gave me a point from which to begin the upgrade refactoring from, but left me with a large number of changes to assimilate in one go (the main challenge in our previous attempts at the upgrade).

As an alternative strategy, I started a branch from #1 and backported the required changes into the fork, with the intention that this would reduce the amount of work required in the first upgrade step and hopefully lead to working code much more rapidly.

The cherry-pick of the Classification model applied smoothly, but I ran into considerable difficulties with the commit that brought in BaseMailer. In the end I implemented a very simple BaseMailer class as the parent class of all Spree mailers. This was followed by cherry-picking all fork commits and then bundling OFN against this version of Spree.

Next steps would be to continue work to determine how much effort is required to get a green build against this version of Spree.

[pmackay]

@RohanM would it be possible to capture some tech notes on what customisations at a broad level have been applied to spree 1.3 to make OFN work? Are they mostly going to have to be applied to v2? Is there anything that can be done now to make future upgrades easier or more accessible to others helping?

[sigmundpetersen]

Should we move this to the Spree upgrade category?